 Hello. Everybody hear me? Great. Well, welcome to the OpenStack Network Protocol Technical Contrast. My name is Joe Stevens. I work for HP. Thank you, everyone here. It's really great seeing all this turn out here for the presentation. I'm going to be talking about Network Protocol. So hopefully, that's what you came for, because that's what we're going to get into. This is not an HP presentation. This is just a community presentation voted on by you guys. So especially, thank you, because you're the ones who enabled me to get up here and do this. So that's pretty cool. Let's get into it. Welcome. We're going to be talking about four network protocols today. Flat networking, VLANs, GRE, and VXLAN. We're going to get into each one in a little bit of detail. This is a whopping 40-minute presentation. So as you can imagine, I'm going to probably talk pretty fast. That's one of the few benefits of being born in New Jersey. The Italian family is I have that ability. So forgive me if I'm going quick, but it should be at least interesting from that perspective. So what I decided to do when I came up with the idea for this session was to come up with a buyer's guide concept to network protocols in OpenStack. What I meant by that is we've got a lot of choices. We have all kinds of different options in OpenStack. Network protocols, certainly there's a bundle. But there's not a whole lot of info out there on why I might want to use one over another. So that's really what the main goal of the session is. I'm going to provide some base information for folks who maybe are new to OpenStack, networking and specific. We're going to abstract the protocols themselves and talk in detail about what the protocols do with relation to OpenStack. We'll be getting into some of the mechanical details. We don't have enough time to go into the nitty-gritty about everything. 40 minutes, right? We're going to find out why the heck there's so many choices to begin with. And that's a question I think that plagues us from the get-go is, can't you just tell me what the best one is and I'll use that? Well, sadly, that's a complicated answer. We're going to take a look at the potential costs and benefits of each one of these options. And we're focusing in this session, 40 minutes, 38 now, on the what and why, not so much the how. There's plenty of sessions that tell you how to configure VXLAN and et cetera. I'm looking at it from an architecture perspective. I hope you all too on why I would do one versus the other. And what do I get out of it when I make that choice? So it's 40 minutes. I'm going to keep it rolling. I talk fast. I'm going to try and get through a lot of content. I'm going to skid right into the end, hopefully at 40 minutes. And so I'll ask you to hold your questions till the very end. I'll be thrilled to go out afterward and we can talk about it as long as you want. But I won't have time to stop during the presentation. So hey, guys, we're here to have some fun. I hope you are too. All right, a little bit about what we won't talk about. We're not getting into hardware specifics. We're not talking about accelerator cards or offloading or any of that stuff. It's all really fascinating. There's actually some good stuff out on the vendor floor. You can talk to folks about that stuff. It certainly goes along with this discussion. I don't have time. We're not going to talk about DVR either. That adds a whole really cool element to the idea of how the protocols interact with OpenStack, but it actually, it'll make it way too long. That's a whole other topic to itself. We won't be going into how to configure, like I mentioned, and IP version six isn't on the list. So if you came looking for that, that's not where I'm headed. Let's get into it. The first step in avoiding a trap is knowing of its existence. Come on, has anyone heard that one before? If you're a science fiction guy, all right, crap, I'm the only one. All right, well anyway, I think I just outnirted a whole room OpenStackers. Fuffer Howlett from Dune, Frank Herbert, right? So why did I have this in there other than you're supposed to have quotes in your presentation? Well, I put this in there for a reason. In OpenStack, we make a lot of choices, right? When we get down to the architectural decision making process of deploying OpenStack, we have a lot of things to think about. And sometimes, and I'm sure you've seen over this last couple of days, we write presentations on things we should have thought about before we did it, right? And network protocols is probably right there at the bottom. Once you've deployed OpenStack cloud and started having users get in there and running workloads, it's a little too late to decide you picked the wrong one because redoing it's going to be painful. So we want to avoid the trap because we know they're out there. And this is easy because there's some pretty easy decisions to make. So my philosophy in this little tiny topic is start with the end in mind. This slide will have a whole bunch of pretty words on the right side. And if you're into taking pictures of slides, I strongly suggest you take a picture of this one once the pretty words are in it. Because this one is one you can print out and use in your design planning for future clouds. So here's some questions to ask before you pick a protocol. Will you need multi-tenant support? This is a big question. A lot of people, of course, do what's OpenStack and not everybody does, right? So a lot of clients don't want that. It's an important detail. You need to know the answer to that. It will help you pick the protocol. What's going to be your scale? At the end of the day, when this cloud's in full production, are you thinking it's a little test bed thing? Is it going to have a dozen, maybe a dozen nodes in it? Not so big. Is it going to be a full-blown, kind of multi-tenant, large-scale, 1,000 nodes? That kind of deal, need to know this in advance. It affects the protocol. What type of utilization do you expect to see on your hypervisors? What the heck's that got to do with it? We're going to find out. It has a lot to do with the protocol. Do you need maximum network performance? Kind of goes hand-in-hand on the hypervisor topic. Some of these protocols are a lot happier than others with heavy workloads. And it will affect your goal at the end. And we'll take a look in depth. We're going to get answers to all of these, by the way. That's the point. How much admin time can your organization, or you personally, can you devote to maintaining address space? That sounds like fun, doesn't it? Everyone here has probably done it in their careers. But is it something you actually have a mechanism for and are capable of doing? What's the underlying topology? This is all software stuff, right? We're not supposed to know about that. Well, if you want to be successful as an architect, you might want to know what's going on behind the covers. And in this case, with the protocol, there is a embedded relationship between the protocol you choose and the gear you're running on it, potentially. How important is redundancy? Test, maybe not. Production, maybe so, right? Are there going to be multiple sites? That's a biggie, right? And that's going to drive some decisions on which protocol you choose as well. And what kind of inspection takes place? Small detail. If you work in a bank, maybe not so small. They're actually pretty interested in some sort of compliance and audit that can inspect packets that are traveling in and out of certain regions of their network. The protocol you pick may affect that ability, so your compliance guys might want to talk to you. My favorite question is, will this be a temporary test environment? The answer is nope. Never will be. OK, so just keep that in mind. Whenever someone tells you, hey, we're just going to do this temporarily, right? How many people have built something temporarily that has become then the running production environment? And I put that in there because don't think of your choices based on a decision you already know is wrong. So plan on it being production. Just plan on doing it right. I mean, unless you genuinely know you're tearing this thing down tomorrow, you're going to live with it. And you're going to live with these choices. Of course, I like to know what's for lunch today, too. So we'll find that out. And this is the worst possible speaking time to be is in between hungry people and lunch. So we're going to move on. Here we go. Oh, hold on. Wait, take a picture of that one if you're going to take a picture, because that actually, this is something that'll be handy. You need to know these questions before you pick a protocol. All right. Three, two, one. Gotcha. All right. So here's my OpenStack Protocol flimsy analogy number one. Protocols in OpenStack are much like a pyramid. Not really, but anyway. So the reason I brought this up is because I wanted a cool image to put in the background of the presentation, plus the fact that it is sort of like a pyramid. We build the protocols we're going to talk about more or less build on each other. I mean, not directly. It's a little flimsy in some cases. But we definitely start out with a base of understanding, and we add some pieces to it as we go along. And I'm going to try and do that in the next 33 minutes. So here we go. Flat networking. Protocol number one. Step to the plate. Flat networking. This is probably the most important detail about flat networking. It doesn't require VLAN aware switches. A lot of folks say, why would I even use that? That's a good reason. So I can go grab my off the shelf Best Buy Netgear switch, and I can plug it in. I can run my little lab on it. Using flat networking, everybody's happy. If I do that with a VLAN tagged network, not so happy, because the switch may or may not even ship the packets. It may see that as a garbage shucket in the box. So this is an important detail. If you're thinking about building a little lab, this is a good choice. Addressing space, we all know. IPv4, it's running out of dust. So we've got to the point where we've all managed that pretty well. We know the limits. Overlap in your addressing space is a manual problem. So this leads into our admin situation where who's going to deal with this? Someone has to track addresses. Wheel in the spreadsheet, Bobby. We've got to start filling it out. Maybe we've got some ops guys who do this kind of thing. I don't know. But it's a pain, and we need to do it if you're going to use flat networks. Now, good side, extremely easy to set up. Everybody in this room has used something that looks like IPv4 with no questions. We know it. It's older than half the people in this room. So it's easy to set up. Mature, been around since the 80s. I would ask for Shal Hansley. He was born after 1980, and I could already look around and kind of guess. Yeah, you get it. It's been around. We know it. It's the guts of all the other protocols. So if you know IPv4, you're going to add something to it as we move along. Now, a little bit about redundancy and resiliency. So I told you that was kind of an important detail. Flat networkings heavily rely on spanning tree protocol for link resiliency. So spanning tree, another very familiar age old concept, requires a loop-free topology within your network infrastructure. It does this. It uses calculation to create a loop-free topology by blocking the ports that may be causing the loop, which is really cool. It does this automatically for you, all on the background. Problem is, and then if one of those links fails, it can spin up that blocked link for you. And now you're back in business again, all automatic. That's awesome. I mean, it sounds good, at least. Well, the problem is it's really slow, right? It's generally speaking fairly slow convergence, results in network outages, and increased TCO. Why is there increased TCO? Well, those block ports aren't doing anything. They're blocked, right? So now I'm paying for some 40G port to sit there, not shipping traffic, right? Sooner or later, the CFO will probably think you're a hero if you can figure out a way around this. One of the advantages of flat networks in redundancy is it can use some layer 3 stuff out there that's for crossing boundaries, things like load balancers, multi-pathing, dynamic routing, clustering. All that stuff's kind of layer 3, works really well with IPv4. You're not going to have a limitation there. But it's important to know it happens at boundaries typically. I mean, there's alternatives, but typically at boundaries. Now, most of this stuff can be enhanced or overcome with time and money like anything else. This is kind of background, right? So if you want to spend time, you want to spend money, you can shore this up. But there are other protocols that don't have these problems to start with, so let's take a look. So the admin cost. We mentioned this earlier. You're going to be tracked and forced. Forcing the avoiding of overlap has to be done manually, right? Your VMAX has to be tracked and enforced so we can avoid overlap. We can't ship them out with the same VMAX. It's a manual approach. You're going to have a guy with a spreadsheet. Maybe it's an ops guy. Maybe it's you. I don't want to do this work. Maybe you like spreadsheets. But anyway, it's a pain, and you have to take care of it somehow. Now, as far as flat networks go in a multi-tenant environment, not the best choice. And you can see kind of why. Overlap of addresses could be a really serious problem, especially multi-tenant. Last thing you need to do is have on the same pipe someone writing two addresses, the same overlapping it. Now we have the whole network down for both of them, right? Another issue is it requires a tighter integration with your network infrastructure team. So it's really more traditional ops oriented, right? Slows the roll. I can't just jump up, spin up a new network dynamically through OpenStack, get going, and do all the cool stuff that makes OpenStack worth having. I've got to slow down a little bit and say, hey, let me go get a spreadsheet. Let me get this address. Maybe I've got to put in a ticket for somebody. And that kind of thing, right? So it's not exactly the SDN model. And address space, the address space is limited, right? And it has to be managed across tenants. Now that's just untenable in a multi-tenant environment, right? So now, if you're the actual only owner of all the tenants, that's one thing. But if you really have a multi-tenant environment, and you've got client A and client B, and they're different tenants, well, who the heck wants to now be managing the spreadsheets for both of them, right? That's not going to be fun. They'll fill out this form, please, right? Now, the upside. I don't beat up on old flat networks too bad. Generally, very, very fast, OK? So typically, least amount of CPU overheads steals less from your hypervisors, right? Doesn't do any extra manipulation. When we're shipping packets in the hypervisor, we're not doing a whole lot of them shipping packets. Typically, can achieve near-lying speed. If you have a 10G link, and you want to get 10G out of it, you can get 10G out of it right there, OK? So that's a big deal. And I think another thing is it's a known quantity. Everybody's been there and done that. We know IPv4. So again, I'm rushing through this piece, because we already get IPv4 kind of. So I want to get to the juicy stuff in the next 27 minutes. So pros, flat networking. Low overhead, really low overhead in the CPU on your hypervisor has high network performance. It'll run on all your IPv4, so you don't have to go buy new gear. You don't need VLN switching support. Honestly, put that in the back of your caps, guys, because if you come to the point where you need to do something and you just happen to have that whatever little d-link router or d-link switch sitting around, this will work, right? Negatives, scale's going to be limited, right? Address overlaps can happen, and then bad things happen from that. It's not really extensible like some of the others, which we'll see. And it does rely heavily on spanning tree. It's really kind of the only thing there that's doing anything for it. More admin required, right? And more admin equals kind of against the sort of concept that we're driving for here. So why would I use it? On the green side, single tenant, yeah, no problem. Small scale, easy to deal with. Protects investment in your network infrastructure. The CFO's happy again. And labs test small stuff. Not really big time, right? Negative multi-region deployments, it's going to fight you every step of the way if you're trying to do multi-region with flat networking. Larger scale, you get into massive management of addressing spaces and multi-tenant just makes it even worse. So there you go, flat networking in a nutshell. Groovy. All right, that was, I really wanted to do that quick because, well, 25 minutes left here, and we're not even through the juicy stuff yet. Virtual VLANs, right? Virtual Local Area Network, a little more mature. We all kind of know this one, too. It's going to be pretty quick. We'll get into the deal here. Video 2.1Q describes the standards. It doesn't encapsulate the original frame. This is sometimes a misnomer. Folks think that VLANs are like a tunnel or an encapsulation protocol. They're not. It actually is simply an insertion of a 32-bit header field right after the source MAC in the packet. So you're taking your packet, you're adding a little to it, and voila, one of those things is the VLAN ID. That's what creates a VLAN. So now, once that VLAN tag has been inserted, it can be inspected and stripped, kind of like in the airport, but probably less fun. And you have to use VLAN switches to do this, right? So it kind of varies how each vendor handles stuff. Some of them, the switch will see the frame, see that extra piece in the header and go, that's crap. Let's throw this whole frame out. We won't send the traffic. Some of them will send it anyway and who knows where it goes. But the message is it requires something that's generally VLAN-aware if we're to work right. Spanning tree, our old friend, the instances can be applied logically to individual VLANs. This is kind of a big deal. So one of the problems we said was traditional flat networking has problems with resiliency. VLAN kind of takes that and adds a little to it, right? So we have the ability to kind of spruce it up a bit, give it a little refresh. We can create individual spanning tree instances per VLAN. And there's some other good stuff like that. Still requires loop-free. Still requires blocked ports at some point. So you still have some of these problems. It's just a little nicer. It's an incremental step forward. Now, one of the other good things is you can ride, obviously, multiple VLANs over a single physical link. That's a huge plus, right, Trunks? We kind of all know about this stuff and going quick. And Max, really, punchline. Maximum number of VLANs, 4,094. That sounds like a really big number to a lot of people. I've heard that statement, well, I'm not going to use 4,000 of anything. That's amazing. I can do that in my life. Yeah, well, here's the rub on that. That may be true. Your whole environment may not use 4,094. However, your switch may only be good for 24. And that's a real rub that you need to check out, too. Individual switches within your topography, sometimes your top of rack switch that all your VMs are plugged into, they all are different, right? So each manufacturer and each product line these guys have, some are eight. Some are 128. Some are 1,024. It really depends. In the domain, you can have 4,094, but your gear may not be able to deal with anything bigger than a certain number. This is an important detail to know. So quick, little pitch on the packet itself. You've got a pretty straightforward layout. It's just, I promise I won't dwell on any of this. It's just inserting this little VLAN header. Everything else is exactly the way it came from God, right? So we just put that little header in there, tells you what the VLAN idea is, now you've got a VLAN frame. That's the end of it. Not too much to see there. Redundancy-wise, VLANs, spanning tree, still has to be loop-free. Spanning tree still calculates your topology. There's likely unused links. Most of the resiliency still comes at layer 2. There's some extra goodies that we can tie together to make things better, poor channeling and such. And some of those problems, again, can be overcome with time and money. You can spend time planning. You can spend time implementing. Tends to make your environment less flexible, that we're all here to achieve as flexible environments. So some of that stuff that we do in the background in traditional IT can be done, but it's kind of fighting your SDN model a little bit. You're kind of saying, all right, I'm not as dynamic as I really want to be. Multitenancy, hey, it's pretty good here, honestly. So it can be used. Maybe not the best choice, there's better. But it can be used. Overlap of addresses can still be a problem. Still has to be tracked. VLAN IDs have to be tracked and assigned across multiple tenants. So if you have more than one tenant, they can't just pick whatever VLAN they feel like using, because it's all running on your infrastructure. And the infrastructure will still see the same address. So we can't do that. Scale problem, 4094. It's a big number, but it's not all that big when you start adding up in a multitenant environment. Think about this, 100 VLANs in a tenant. Now I have 100 tenants, uh-oh, right? We got problems. Bottom line on VLANs. Logically, they're a pretty reasonable scale. I mean, that's not a small number, you know? Multitenant support does work. It's not bad. It uses common concepts that have been there and done that idea, like I just mentioned before. Probably a brand in the rooms don't deal with VLANs at some point. Your infrastructure guys certainly have. No real learning curve, nothing, no magic here. And it performs. VLANs do perform, right? So in a high-performing environment where you need lots of bandwidth and you've got an application that wants to crank out, you know, something over the network, VLANs do pretty close to line speed, typically. But on the downside, you need VLAN switches. Maybe it's a downside, maybe it isn't. I think everybody who deals with, you know, enterprise-level gears, typically using VLAN switches, they have to be tracked, again, some kind of manual process that I don't want to do. They can't easily extend across regions. I mean, they can, don't get me wrong, it can be done, there's a way around, but it's not easy. It's not an easy way around, right? We realized for Unspanning Tree, like we mentioned, and you may have some dead links again. Again, that can be altered, time and money, just figure it out and you can make it work. But that's not a plus. So why would we use VLANs, right? So single-tenant, small, multi-tenant, for sure. Medium, large-scale, could be large, I mean, depends on your idea of large. Moderate utilization, protecting investment and network gear. When your CFO just spent $4 million last year on multi-layer switches, they're not gonna tell you to go, you know, you're gonna tell them the pound of rocks for that stuff. I mean, you know, hey, yeah, we're using this, you know, so he's happy again. This is an important detail for the success of your career, trust me. If you're using older server gear, it could slow down with CPU utilization. VLANs are not as picky about this. It's gonna be worse later, we'll see. Scale beyond 40.94, just out of the question. We're not doing it. If you've got lots of tenants, that's a bad sign. So if you're running into that, hey, I think potentially this is gonna be a large multi-tenant environment, we're gonna have lots of tenants. Well, that's an exponent, right? So it's gonna start getting bigger and bigger fast. You may hit that wall faster than you think. And if you need to get, again, if you need to get your local networks extended beyond layer three, that's tough. Now you get into some kind of weird bridging thing. There are vendors out there who have some cool technology around making that happen, but it's no fun, right, trust me. It's not gonna be fun to integrate with OpenStack. There we are. Not too bad. 18 minutes left, you're all still mostly awake. I think that guy's not, but, yeah, all right. GRE tunnel, right? Generic routing encapsulation. This is an old-time favorite, right? It's a oldie but goodie. And here's another quote, right? Why not? Anyone guess this one? Come on, somebody. Come on, save a lot of time if I just gave up and went mad now. Hitchhikers, thank you very much. Arthur Dent, Hitchhiker's guide to the galaxy, right? So what the heck are they thinking about GRE? When you start looking at this thing, you're going, ah, I don't know. And it kind of makes sense when they got it into OpenStack and then came up with something better, you know? So we'll see how it works out though. So general concepts of what GRE, if you're not familiar with it, it uses IP version four for transmission over the existing network infrastructure. This is a nice idea, right? So I don't need special transport gear. I don't have to have some wackadoodle VPN thing. I don't have to have any termination stuff. It's all done in software and it rides over your existing networks. This is good. It does create a tunnel between two virtual point-to-point links. This is not encrypted tunnel, okay? So a lot of folks kind of get that one, somewhat sideways, just because it's a tunnel doesn't mean it's encrypted. It just means it's stuffing something and something else and sending it down the road, right? And so that data is still clear text. It can still be read by anyone who can catch the packet. The payload packet is a layer three encapsulated packet within the GRE packet and it can be sent across your network traditionally. It's kind of a fun part, right? It does allow for overlap of IP addresses which is between tenants. This is a big deal we talked about, right? So we want to make sure we're not having to go and maintain our spreadsheets and see what everybody's IP address is. And this GRE will allow you to get away with that because it'll give you some of that isolation between tenants. Does allow for full use of all your goodies, your routing protocols, dynamic pathing, you know, load balancing, all that kind of good stuff can all be used with GRE as well. So again, the transport is just regular IPv4. Does scale better, arguably, than flat or VLAN networks. Because you have that ability to do some isolation, you can have overlapping addresses and therefore you can reuse stuff, right? So you're not getting anything new out of it but you're gonna use the old stuff more. So then potentially that comes like, eventually kind of teeters out where now the overhead of running too much GRE becomes a problem and it balances back again. But generally speaking can scale reasonably well. What does it look like under the covers? Well, again, we have a transmission network, okay? That's basically all this good stuff up here which just is your regular IPv4. And then we have the payload. The payload rather than being the normal payload of the packet is another packet, right? It's jammed down inside it. That was the original source packet, gets stuffed in the other one and then transmitted. So that's how we create the tunnel. We take a packet that hosts sent, we stuff it inside another packet and then we whiz that guy across a network when it arrives wherever it's landing. It gets unstuffed and all the host sees is this guy down below. So it arrives through the tunnel. But again, this is all clear just to make sure you get that one there. The data in here is not encrypted. It's just clear text. It just happens to be inside another packet. So redundancy in GRE. Well, it uses Layer 3 technology to communicate so all the goodies that go with Layer 3. You too can use. It can use multi-pathing, equal cost multi-pathing, all the good stuff that's kind of traditional WAN technologies and all that. It does use redundant, you can use redundant routers, gateways, all that good stuff. And it doesn't rely as heavily on spanning tree. Still at the underlying fabric, you got Layer 2 that's dealing with spanning tree. But because you can really now start to lean on things like dynamic pathing, you're less beholden to it. The admin cost, kind of auto magic, right? The configuration's pretty quick. It's not a big deal to set up. Doesn't require somebody with a spreadsheet to live their life tracking your addresses, which I think is great. And it kind of fits better in SDN, right? Sort of small-scale SDNs, very easy. You can dynamic, pick a dress, do what I want, make it happen. All this good stuff comes to life and I don't have to go and find the IT guy and put in a service ticket and wait for three days to get back in a network assigned. So, we're getting there. 14 minutes left, we've got some testing information. You're gonna love this piece, so get ready. Here it comes. Especially that guy, all right, yeah. NetworkKaracy.com to this test is an entirely subjective. I have absolutely no validation as far as the quality of the results presented here. However, I can say they were consistent with most of the other tests which I read. So, I think it's good for this, okay? That link will get you there. You're welcome to have copies of the PowerPoint and you can go and read the whole thing yourself, so. Here's what they did. They set up a simple test. Two servers, right? Two physical servers, open V-switch, running on each, couple of VMs on each job there, connected with a 10 gig ethernet switch and of course they established a GRI tunnel between the two. So, what happened? All right, well. We found that the processing of the mechanics of GRE, all that stuffing and unstuffing, takes some CPU time on the hypervisors. That's not really shocking, right? The overhead's low, it's not really that high, but importantly, it's required. That's a big deal, right? So, it's low, but required. Well, what happens when you run out of things you require? Like air, for instance. Yeah, bad things happen to you. So, software has a cost, right? So, when the CPU approaches saturation on the hypervisor, the ability to process GRE starts to slow, packets start to drop, throughput starts to suffer, phones start to ring, okay? So, I think it needs some kind of QOS, I don't know what's beyond me to talk about, but there isn't any at the moment. So, here's the deal, results. Baseline OVS bridge, what you're looking at on the left side of the screen is, the test conducted between the two servers as our sort of baseline that says, what does it look like without GRE, to make sure everything's working to start with? Well, they got throughput 9.3G, right? Pretty good, 10 gig link, they're at 93%, they were able to drive that pretty hard, that's impressive. Our sending servers CPU is clocking out at 70%, which means, it's sending everything it can send, and it's even got room to do more if there was more to send it could. Receiver, 82%, again, it's receiving everything, putting it away, waiting for more, beautiful. Turn on GRE, oh, bad times at the farm. All right, so now we're down to 23% of line rate, no one's happy. We have, our CPU on our sending server has now spiked up to 97%. Well, that's kind of crazy, huh? So it went up 27%, I'd argue it probably went up more than that, it's just that that's where the needle hits the end of the fire, and it says no more, right? So the server, the sending server is now slammed. It's running at full contention, it's got no more cycles to do anything, including packaging GRE and sending it out. So packets start to drop, when packets drop, 23%. So that's a bad sign. Now, you would expect this, that correspondingly, the receiving server is now down to 75%, so it's dropped. It's not doing as much as it used to, because it can't get the packets fast enough. So now it's saying, all right, come on, send me another one, I'm ready, you know? But it's not there yet. So this is a problem. I wouldn't say that this survey is the end all, be all, we don't know what kind of actual gear, I'm sure if you used slow servers versus new servers, you'd get a different result, not that kind of thing too, as you can imagine. But you get the idea. If you hit contention, problems are gonna start happening, something to keep in mind. I am so far behind, I can't even tell you. All right, so bottom line, GRE. Logically, it can handle a larger scale, good multi-tenant support, allows for some robust layer three features, right? It's pretty flexible, it's pretty quick, as far as configuration goes, runs on your existing infrastructure, and it's kind of DevOps, right? It's DevOps, it's very software-y. Bad side, well, overhead and CPU is gonna suck. High demand, network performance is gonna suffer, throughput's gonna be lower if you're at a high utilization. If you're low utilization, maybe that's not a problem, right? So where would we use this thing? Multi-tenant environments for sure, with medium expectations as far as your utilization goes. Areas where we wanna extend layer three to other areas, if we wanna get multi-region support, doesn't do a pretty decent job of that, right? Because of that encapsulation process, you can kind of shift packets around and the servers on the other side aren't so aware of what happened. Does protect investment in your network gear, I don't have to buy anything new. The bad side on use case, well, if you're gonna look for maximum performance on the network, this isn't gonna be it, right? Hypervisors are gonna run hotter because there's overhead than they would if you were using like a VLAN. Older server gear, if you've got an environment that's made up of stuff that's a couple of generations back, well just imagine, now you've got that heavy workload and you're putting in on something that can't run fast to start with. That may not be good. And then of course, yeah, high utilization, stay away, right? Man, we're getting there. Look at that guys, nine minutes left before he pulls the hook. All right, VXLAN. So virtual extended local area network, this is the topic everyone came here to see. The future ain't what it used to be. No one's gonna get this one. Let's not sci-fi, see. Yogi bearer, right? Yeah, of course, yeah. The future ain't what it used to be. So this is the new shiny toy and the toy box, everybody wants to hear about this. Here's what it does for you in life. VXLAN, virtual extensible logical area network, local area network, defined by RC 7348. You notice there's like an extra digit in there from the original IPv4 one. So a lot of things have happened since then. And it was built largely to resolve issues created by virtualization. Here we are, virtualizing. Expands that data center limit of 4094 that was created by VLANs, which sounded like a big number at the time. And now we've upped our game to, anyone know? 16 million is the new limit, right? So that's a big damn number. Solves the multi-tenancy duplication of VLAN IDs and MACS, because it creates isolation, provides for a layer three overlay for separation rather than a layer two VLAN. So think like VLAN set layer three, right? It allows for all the goodies that have been developed along the way for layer three, multipathing, all that good stuff we talked about. That's all available. And it also forms, like GRE, non-encrypted tunnel, okay? All in the clear, just in a tunnel. How does it work in multi-tenants? Well, yeah, that's the point of this deal. So it provides network isolation between your tenant environments. That's an important point, right? Just like GRE kind of did, but better. If we use VLANs, we're limited to 4094, right? That's our kind of scale. VXLAN, 16 million VN IDs per domain. That's a lot bigger. I'm not sure you'd ever get away with that, but it's still a big number. We can use, if we use layer three networks, overlapping's gonna happen, and that's okay. Because VXLANs are isolated, so one tenant does not see the traffic of the other tenant. They can all ride over the same network, and everybody's happy. Yeah, yeah. So another problem that VXLANs addressed, which is something that maybe people don't know quite as much about, is the switch table spacing problem. Within hardware, originally, layer two switches were created for physical servers, right? And so a 48 port switch had the intention of having 48 thingies stuffed into it, and it learned those 48 thingies MAC addresses as they started to communicate. Well, then along came the idea of sticking a thousand thingies into one port, because now we have virtualization. The switch can no longer handle the amount of table space that's required for that to operate, right? Now, modern switches and where enterprise class stuff, they don't have a problem with this. But there are switches, I guarantee you, in your environment that have this problem, right? So they're gonna run into a situation where they run out of table space. Well, what happens when the switch runs out of table space? Stop sending packets. I don't know your address. That one goes off the corner. Next, oh, I don't know you either. So that's what happens. And so that's a problem. VXLANs actually solve that issue by that encapsulation process because your transport medium is the only thing that the switch is trying to learn rather than all the little VMACs of every single system in the deal there. So, yeah, we talked about, yeah, network out, you got it, you don't want that, that's bad. So, okay, VXLAN layout. VXLANs run over existing network infrastructure. This is another big plus. They can extend your layer two network by encapsulating it inside a layer three network, right? So this is really good stuff too. A little different in GRE, slightly different approach. The tunnel makes it look like your hosts are actually in the same layer two network, which is pretty cool. It creates a VXLAN segment or overlay. Sometimes you hear both those words. It means the same thing. Someone says segment, someone says overlay, that is the same piece of information. The VXLAN network identifier or VNI, which is like the VLAN number, right, but for VLXLANs, is a 24-bit ID, 16 million VNIs per domain. Only VMs within the same VNI with the same number are allowed to talk to each other. And duplicate MACs can be used as long as they're in separate VNIs, which is a cool part with virtualization because you can pick your VMAC, right? And so there's no body out there doing that for you. You could pick it. It could be the same. There's no problem. So is it a tunnel? Yeah, well, sort of. So essentially what happens is original layer two frame is encapsulated within a layer three of three VXLAN packet, right? The tunnel itself is stateless. So it is a tunnel, but it's a tunnel created every single time a packet's sent. That's a big deal. So each frame gets encapsulated individually, right? So no state. The tunnel has end points. End points are called VTAPs, right? VXLAN tunnel end points. This is another important detail to remember if you're working with those. I like the fact that it's a tunneled acronym. So it's an acronym within an acronym. VTAPs exist typically within a hypervisor that hosts the VMs. So the VTAP termination hardware's coming. I didn't want to talk about that now. Certainly not in the next four minutes. However, there's stuff to kind of help out with that. Right now it's all in the hypervisor all software, which is cool, it's dynamic, flexible, it does all kind of good things. The big point is the last item. The VM that generates the traffic doesn't know anything about the encapsulation, right? It's all handled outside of it. It just thinks I'm receiving and sending packets from a server that's sitting right next to me. Look at that. Anyway, so. There's it. You guys get the download. This is just a workflow that shows you exactly how the packets move across the network. I'm not gonna go into that right now because I've got about three minutes. So VX LAN frame format. Again, same concept, right? A little different than GRE. That guy down the bottom is our payload, right? Which is the original layer two frame. It's stuffed up inside a layer three packet with all the VX LAN goodies in it, okay? And that gets shipped out. And then when it gets received, it all comes back out again and the VM that's receiving it just gets this bit inside, all right? So that's why it thinks it's talking ethernet to the guy next to him. All right, oh man, you're gonna hang me if I don't get through this. This is the IETF posted a performance study on VX LANs and the impact using software to find networks. And here it is, right? By the way, the link to it's in there too. So if you go on there, you can actually look it up yourself. It's very similar to the last test, right? Two servers, bit in the middle, yeah. Here's the plan, right? So what happened? When we ratchet this guy up to maximum, CPU with one VX LAN ID and use jammed up to, it added only about two to 10% overhead. That's not huge, right? A little more efficient, I think, than GRE is what's happening. No memory change, packet loss. Well, as expected, you start to saturate the CPU. This is still a mandatory object like oxygen. You need it to breathe. So if the server, if the hypervisor CPU runs out of space, start losing packets, right? So they experienced 2% packet loss, which kind of drove the throughput down. As you can see, 56% of line rate, right? It was max. So it's still kind of stinky, you know? I mean, that's not really that great. Now, there is some hardware coming that can hopefully do a little bit of that, maybe offload that out of the hypervisor and still be dynamic with the software controls that we want, but not create this bottleneck, or I shouldn't say bottleneck, but dependency. It's really the issue is it's a dependency. It's a cyclic issue. So if you've got a workload that does a lot of work, it's going to affect the network, right? If you've got a network that does a lot of work, it's going to affect the workload. Kind of goes hand in hand there. Bottom line, you can handle large-scale, 16 million VNI's can't be wrong. Multi-tenant support, you can use your layer of three goodies, right? It's extremely flexible, runs on existing infrastructure. Bad side, generates some CPU overhead. Throughput's not, it's not max, right? We don't know the max-scale problems yet, and that's one of the things I'm a little interested in, is what happens when we've got lots of VNI's running on the same hypervisor? Will we have any extra problems with the VTAP? Use cases, multi-tenant, right? Oh, don't hook me, don't hook me. Multi-tenant environments, we can extend across the boundaries. I'm going to skip right past this guys because I think there's something here you want to see. All right, let's get to the point, right? Remember this slide in the beginning. If you need multi-tenant support, right. VXLAN and GRE are going to do really, really well. VLAN's okay, right? Ultimate scale, if you're going to have a big scale, VXLANs can do that, that's its job. And you can see the differences as we went along, which ones are a little bit better, which ones are a little worse. What type of utilization? If you're going to spike the crap out of your hypervisors, hmm, think about what's going to happen to them. You want to keep monitoring that and be proactive about it, because if you're using VXLAN or GRE, chances are you might see other problems. Admin time, if you don't have it, don't use the flat networks and VLANs because you're going to spend time administrating a really complex environment. What's your underlying topology? Most of these don't care except for VLANs. VLANs want to use VLAN switches and some of the other ones might use VLANs as well, so keep that in mind. Redundancy, is it important? We know how we can get there. Is there going to be multiple sites? VXLAN's going to win for that one, right? Because it's really good about pretending like it's a local system to each one. It's almost like it's VPNing to itself. Inspection, this could be an issue. Flat and VLANs are known topologies that you work really well with most inspection tools for your compliance officer might care about this. GRE and VXLAN, although they are not encrypted, might throw crap at the inspection tools that the inspection tools may not be able to read, right? And your compliance guy might come back and say, I didn't generate a report this week because you used some stupid protocol. Yeah, there's a temporary? Yeah, never. And hey guys, let's get some lunch. Thank you.