 Good afternoon, my name is John Holly. I work for VMware and while I can't be with you today I'm glad that I've got this opportunity to record that this particular talk and be able to share it with you Not only now during the the open-source summit conference, but With those of you who are going to find this afterwards. I hope you find this useful today I'm going to be talking about Bridges bonds and taps going beyond ETH0 basically a an hour-long dive into Some advanced networking topics to try and get people up to to speed on some of the things that not only exist in the networking world, but specifically that Linux can can do with its networking stack and so Let's Start by covering some of the basics Effectively that there are two types of network interfaces There's the physical kind. So ethernet and finnaband token ring Wi-Fi. These are the kinds of Interfaces that have a physical presence basically they interact with the physical world in some way ethernet obviously you plug a cable in and through some analog twiddling of Electricity you can transmit data Infiniband token ring both work on a similar a similar set of principles Wi-Fi instead of plugging a physical cable in you make a connection through literally the air through the RF spectrum and There's many more types of physical interfaces. I'm not going to try and list them all because I will invariably miss something But basically that the the physical interconnect At some point that the physical piece of hardware that is what we're referring to when we're talking about a physical interface Now there's also things like logical interfaces. So these tend to wrap more complex concepts or higher layer protocols or Just provide functionality that you literally do not need hardware for so things like a Bridge or a bond Or a tunnel or a network tap or even the loopback interface. These are going to be Logical network interfaces. They're still going to be present on the system At least in the at the very least in the case of loopback But these have a tendency to wrap different concepts into slightly bigger pieces But Let's back this up a little bit. What is a network interface? It is a mechanism of transmitting data at its most fundamental level. Basically, it's the physical layer of the OSI model And I'm going to refer back to the OSI model a lot today But keep in mind the OSI model is a complete lie Um, the only reason the OSI model has persisted to this day is that actually is a reasonable way to at least start discussing things about the networking stack but the networking stack Munges and moves Across these hypothetical layers far too much for them to actually be an actual representative Example now we're also for today's purposes not really going to get above layer four Session presentation of the application layer for the OSI model All have all tend to happen in the actual application I'm trying to talk about stuff. That's a bit lower down down in the network down in the the kernel space down in the network space Even out onto the physical wires or out into the physical Airwaves But we are going to talk about one to four pretty extensively today and as you can see I've rummaged through my own drawer full of networking cards and pulled out a bunch that People can see here and there's everything from PC MCIA network adapters here all the way through Wi-Fi adapters and token ring and All kinds of bits and pieces if you've been around The if you've been around the hardware world long enough, some of these will look a bit familiar Um But what do you you know, oh we get what an end network interface, but what about the rest? What about the the data like the the data link layers, you know, what about the pieces that exist above it? Yeah, mostly there everything is built on top of the physical layer Um things like bridges and teams bonds tons taps, you know, these pieces don't really make any sense for the most part Um without some sort of lower level physical interface So if you've got a bond well, you can't actually have a bond without Having something to bond together. So you need a physical interface to do a bond or a team You can have a tunnel, but you can't actually do anything with the tunnel if it can't go anywhere And that's kind of the the the real difference between the physical layer Which is obviously layer one on the the OSI model and the kind of the data link layer, which is Where some of these more logical? Network interfaces live So keep that in mind when we're we're discussing some things here and we're going to get down pretty far into Some pretty silly things here Okay So we've we've sort of talked about the physical layer where you physically plug a cable in and we've sort of talked about The the data link layer. We're still going to be in the data link layer with this But I want to take a second to explain The land these are things that this is a a virtual LAN Conceptually this takes a a a physical switch or a physical network and allows you to segregate it into logical different Networks and the reason that this kind of came about was you can only put so many network interfaces into a computer Even by today's standards with you know the the number of PCI express lanes that we've got you can only have so many network interfaces and It's nice in a lot of different scenarios to have forced segregation of your network traffic so Example of this let's say I have an administrative interface for an application and I have a storage network and I Obviously have some sort of mechanism to push data back out onto the the public internet Now the public internet is obviously a very scary place. You don't want the public internet interfacing with your Your storage network or your administrative network and in fact, you probably don't even want your administrative network talking to your storage network Not because it's exactly insecure. Although it could be But because the storage network may actually be very sensitive to latency or bandwidth And you don't want random traffic from the administrative network coming into your storage network and potentially disrupting things So what the VLAN does is it actually tags each packet as it Transits the network now in some cases the the machine that's actually sending the data is tagging the the the VLAN information and this is Depending on what parlance you use because in the networking world Practically every vendor uses some different definition event everything, but This is sometimes referred to as a trunk or But but really what it is is it is the the end device that is actually doing the tagging There is one other way I'm generally speaking or that you can do Tagging with and that is to tag the default traffic or all traffic from the interface You can combine these so if you don't tag anything that the the upstream network switch May tag all of that all of those packets for you. You may never know that you're actually on a VLAN Or you may be given effectively explicit permission to communicate on a VLAN an ID Which is literally just a number. It's a a number between zero one and forty ninety six there there are only 4,000 d-lands that you can put on an out on a single network But the idea there is that The machine that's sending the packet out will you know may tag its packets and say I would like my storage network Or my storage packets to go on to the storage network, which may let's just call it VLAN 757 and my administrative information, you know all of you know access to to Configuration files and whatnot that it's going to happen on VLAN 20 It may set up virtual interfaces over the top of the physical interface To be able to to tag those packets correctly and the nice thing about this is that since they are Logically distinct both not only in the Linux kernel, but in on the the network itself, obviously You can run different IP protocols over them. You can run Different IPs on them. You can do a lot of different things there for all intents and purposes once you've set up the VLAN It is another new physical interface that you can Configure and work with in fact almost everything we're going to be talking about today once you've got it set up you can treat it as a new interface to to put IP addresses against or Tunnel across or do do any of these kinds of things But I do want to specifically call out VLANs because these can be really important to a network topology They're also very very common, particularly in Marge deployment scenarios or even most production scenarios just because the network segregation you can get out of it is very very good now a Brief discussion of security. It is possible to escape VLANs Most of that has been taken care of in the intervening years, but if you are Absolutely convinced that you need the the greatest possible security Using no VLANs is the correct answer. I Don't think that that's necessarily I don't think it's necessarily a threat model that needs to be completely worried about today I think that the advantages right now outweigh the potential Security or security perceived security risks as there hasn't been a whole lot of issues with VLANs in the last decade and a half so Probably a little bit deeper type on VLANs then I'd normally go into but I want to make sure that people are aware that VLANs exist and that Sort of how these works and we'll get back to these as I start bringing up examples and start showing Exactly how you can kind of put some of these pieces together Bridges now Those of you who are doing a lot of virtual machines or whatnot bridges are going to be You know almost a given they're going to be ubiquitous and in what you're going to see now And a bridge is basically just a software defined switch inside of the computer itself So the the Linux can set up This interface that many other Things can bind to and you can assign IP addresses to the bridge itself The the bridge can just you know once you you've got it set up You could treat it exactly like a normal, you know effectively like it's zero in this case. So Um in in this particular diagram, you've got a real network switch. It's out on the network And it is connected to E0 on this mythical computer and inside this mythical computer We've set up bridge zero and inside of the computer. It's obviously a Virtual machine a hosting server. It's got a bunch of virtual machines Those virtual machines then connect to the bridge itself and basically inside the computer when the virtual machine wants to talk out to The network what it does is it talks to the bridge and the bridge Does the exact same thing a normal, you know, physical switch would do is it says I am trying to talk to? 192 168 11 Not that this is exactly a good idea. Just have to try and talk to but Um and the bridge says well, I don't know exactly where this is But I do know that the next hop is probably eat zero So it passes the traffic on to eat zero eat zero then passes it onto the proper switch and then the the the rest of the switching infrastructure does the the thing you would expect in terms of Passing that packet along so that it can get to the the final destination And bridges are inside of the Linux kernel are actually really pretty powerful There's a lot you can do with them. Most people don't actually think about them as a full proper You know switch inside of the computer, but they really they really are you can attach VLANs to them You can pass VLAN tags through them you can Run spanning tree protocol on them and you know a number of other pieces and There's a lot of concepts that you can do inside Of the Linux kernel with bridges That effectively mimic exactly what you can do on a physical switch The only real downside to a a bridge inside of the inside of a computer is that it may not be quite as fast as the actual a switching fabric inside of an A real physical switch and that's because the the real physical switch has asics That are specifically designed to handle all of this and inside the the computer just the normal cpu is handling this and it's not Literally, you know purpose built to kind of handle that kind of thing So that is what a bridge is so let's move on a little bit Bond now There are two Concepts here in terms of bonding Um, there are bond and there are teams In a lot of cases particularly when you're looking for information about this stuff You're going to find That these terms are used interchangeably in the Linux kernel. They are not Exactly interchangeable Um, so let's let's talk about bonds first um Basically what a bond does is it takes multiple physical, um Interfaces now I say physical that you can kind of this gets a little mungy. Um, it takes two interfaces And amalgamates them in some fashion Um, and the Linux kernel knows about six types of Um network bonding There is the round robin. There's active backup XOR broadcast Dynamic link aggregation i.e 802 3 ad TLB and ALB Now This is a lot of gobbledygook Um, the and a lot of words to describe these kinds of bonding But more or less these six types to find how traffic is either going to come in or go out of the network interfaces um Round robin is going to Um, basically one packet goes out one interface one packet will go out the next interface And it will basically cycle a packet until it reaches the end of the interfaces And start back over and just keep doing that literally in a round robin fashion active backup what this will do is It basically treats two interfaces as if one is actively always doing traffic until it goes down for some reason And then all of the traffic will switch over to the backup link and you can have multiple backup links and Chain those in in interesting ways if you need most people. It's just a single active backup XOR does some interesting mathematics to try and um switch traffic depending on where it's coming from or where it's going on which interface it goes out of broadcast Is a little complicated. I'm not going to get too far into it the most Common bonding that you're you're probably going to ever run into is dynamic link aggregation or 802 380 This is basically the industry standard the industry standard For what all of these things eventually all kind of merged into as a single actual standard Switches support this linux supports this Almost everything supports this if they support bonding. Um And this is primarily what you're going to see and then you've got tlb and alb which kind of do some similar things but they're again different takes on how to hash information now Several of these you do not need active switch support to actually work things like Round Robin or active backup You don't actually need active switch support for for these things to work correctly For several of the others you do actively need the switch to cooperate And some of this transaction So you can't you know if you're if you have a dumb switch And we use dumb to refer to switches that don't have any smarts Or easy configurability to them. You just literally plug your cable in and they don't They just pass traffic. There's no web interface. There's no there's no configuration of them. Um There's no way to set up several of these now The fact that several some of these you know things like active backup around Robin You don't need an upstream switch to to help you That's really quite cool Um because that means that you can get some of the advantages of a full proper, you know Something like 802 3 ad without having full switch support however There are some caveats to to um How bonds work and The the biggest misconception about how bonds work is that When you bond two interfaces together you get and you get double the bandwidth And this is sort of true and sort of wrong the real answer is is that A single stream of data will still only traffic or transit data as fast as the the the link it is on So if you have a one gig link and you take you know, or you have two one gig links and you bond them together um into let's say 802 3 ad The fastest any single piece of traffic will move in or out of the system will be one gigabit It will never get to two gigabits now That that's for single stream When you start talking about multiple streams and this is why a lot of big servers have have bonds particularly 802 3 ad bonds in them is that multiple streams Are now can are now all fighting for one or more interfaces And each one of these streams may be able to actually You know peek out the the entire interface and what things like 802 3 ad um allow you to do what these bonds allow you to do is now uh uh stuff coming in on um in this particular example e0 It may be going at a gigabit or 10 gigabits Or 40 gigabits or 100 gigabits um while another computer can be coming in um on eth1 And also getting you know 10 40 100 gigabit Of traffic into the system And that is where you start seeing systems that are able to push You know 20 gigabits if they have two 10 gigabit interfaces bonded together or you know Uh 80 gigabits if they've got two 40 gig um interfaces bonded together and whatnot and One thing to also kind of keep in mind with the way Particularly ethernet. I'm not this does not apply to wi-fi or the rf spectrum stuff usually um or certain like 10 megabit uh ethernet but Full duplex connections can actually transmit data at full speed in both directions Now what what does that mean that means that on a gigabit interface You can have data coming in at a gigabit and going out at a gigabit simultaneously And this is kind of the the the the reason you want full duplex is because you can actually talk and listen at the same time And you get full speed at the same time so um This is why when you see people calculating, you know, like pc i express bandwidth needs for an interface they start talking about Well, you need kind of a little bit more than 2x You know what your interface speeds. So if you you need if you've got a gigabit interface Well, you need two gigabits of pc i express bandwidth to actually fill the entire pipe in both directions So there you go. Um So yeah, so there's a lot of different ways that you can kind of put these bonds together Um, and they do some really interesting things. Obviously the linux kernel inside of the linux kernel This is all happening there. Um has about it knows about six different types now You at some point you're going to say but there's kind of all these other weird types of bonds and and teams And this is where the bonds and teams distinction actually comes into play And the the biggest difference between bonds and teams is fundamentally a user space configuration portion versus an all kernel space argument The way the linux kernel bonding system works is it's all in kernel space all of your interaction with it goes through the linux kernel um and Everything that that deals with the negotiation all of that happens in kernel space This has some really nice advantages The biggest of which is that almost every kernel that's out there can easily take advantage of this It's you know, even if you you have to load a module or something This is available to you Um now teams the way it is used in in the linux world Um refers to a user space configuration system Usually coming out of lib team um Where the admin tasks for setting up and configuring and babysitting the um the the bonding information all comes out of a user space data lib team in this case or team d um Why would you care about this? Well For starters the linux kernel development process goes fairly slowly and you don't want to to play with it too much just because When you play with things too much you have a tendency to break them and you don't break things in the linux kernel It's just like rule number one. You don't break things there Well, when you push something out out into user space, you can decouple yourself from the The kernel development cycle and you can run your own development cycle. So that's actually a pretty big Um advantage there for the the the user space site um, you do get some minor protections from Some of this actually happening in user space versus kernel space although You still end up configuring a lot of the the kernel space to do stuff. So this isn't super um This fundamentally isn't more secure just because it's in user space but the real big advantage that teams has Over bonds is that it can actually support a more complicated more cohesive set of features Out of all of the bonding uh systems now It can do things like 802 3 ad you can do xorg It can do all of the same things that the the kernel bonding layer can But there are certain features particularly in 802 3 ad That the kernel driver does not support But that teams does Case in point the the linux kernel driver does not support rebalancing The network streams as they're coming in and out which 802 3 ad does support A mechanism by which the the switch and the the computer can renegotiate which stream is going over which interface Um so that you can actually kind of load balance the the interface is a little bit better The kernel space does not support this Um teams in user space does support this So this is you know, uh, uh one of the the reasons that something like teams um Is interesting to to to folks building out networks Now the gacha with teams beyond this is also in user space and everything Is it is not as widely adopted as bonds? So if you're you're if you're on like a red hat or a fedora or sent us um derived System you have pretty easy access to lib team if you're on davian You do not have easy access to this you would have to actually compile it yourself and do a bunch of stuff There's not pre-built packages for any of this and it's not kind of bolted into the underlying configuration system for your um Your network stack and that can be when things aren't really bolted into your underlying configuration pieces here That gets kind of complicated to use. So this is why bonds generally this is why when you're looking around in the linux space You're almost universally going to see bonds um But if you have the the advantage to take a look at some of the teaming stuff and play with some of it It really is quite powerful and in and in some cases. It's actually um very very nice To be able to take advantage of it. It does use a completely different configuration syntax um And one of the things to keep in mind as you know, there's a little bit of a fud out there between bonds and teams teams does not actually incur a performance penalty on the the the the team to interfaces All of that still happens inside of the kernel. It is just the administrative pieces that that bounce back out to user space Get a little bit of processing and then go back out on the wire. Um, things like how do you you you explain to the switch upstream? um, that you want to to to move some traffic around um on the the interfaces and that's which It's not time critical. It's not going to be a huge deal and ultimately once it's actually gotten out The interface it's going to give you a it's probably going to give you a performance center boost anyway So that's bonds versus teams Now taps for any of you who um Have been around for long enough. You'll you'll recognize the pictures here as vampire taps for old thick net um Interfaces and that's kind of where this idea of tapping comes from and particularly if you're going to take a look at vpns, you're going to see tons and taps Uh, come up a lot taps referred to network tapping tons referred to tonnelling. Um and Really all a tap is is it's exactly like the these these old devices that you see here is they take a look at the network And they will listen to everything Um, they're they're literally just like tapping a wire and listening to everything that's coming across the wire um And when you're looking at vpns, particularly open vpn Has a mode where you can run this and run the network in tap mode um What this means is that you can actually pass all of the traffic that is coming across The network from the data link layer perspective as you can see we're still at layer two when we're talking about this Um, this has some really interesting advantages particularly on a vpn um But I think the the disadvantage is far away any of the potential advantages um, for example, if you've got um, Let's just take some sort of a castable um Device in your house you could hypothetically cast across a vpn Because the if you're on the vpn it just And you've got a network tap on both sides. It literally just looks like you're on the same network segment It's just a very slow, uh um instantiation of those network segments so I'm not going to spend too much time on this I I want people to be aware that this is out there and exactly kind of the the the physical analog to this Which is literally just you're you're tapping a cable and you're also You are literally seeing and transmitting on it the exact same fabric From a data link layer perspective tons tunnels We actually move up a layer here. Um, and instead of getting all of the raw protocol pieces Now we've actually moved into needing something like TCP IP to actually start being able to route the traffic and now we're talking about routed networks instead of just passing raw frames across um This is a tunnel is definitely more like having a a a a a virtual wire or a um Or or or or a virtual network segment. Um across the the the the pieces that you're attempting to communicate across and This comes up not only in VPNs, but some virtual machine systems actually use tunnels um to tunnel into you know bridges or out Into the main networks or various other things along those lines um But really and in some cases this is literally just putting one tunnel, you know one network inside of another network That's you know, kind of the idea of what a tunnel is is you are tunneling a network across another another network Um, again, this comes up in VPNs. So things like open VPN Use a tunnel interface um I think wire guard at one point used a tunnel interface and now it kind of uses its quasi-owned thing um, but uh things like Uh IPv6 In ipv4. So if you're sending an ipv6 packet and using ipv6 to ipv4 Gateways to to to bounce things around you actually end up encapsulating ipv6 in an ipv4 It goes out a tunnel uh to your far end point and then gets uh, um Unwrapped out of ipv4 and back into ipv6 before it can then transit as if it was normally ipv6 um That's kind of where this is all going to come off. This is kind of where you're going to see it. Um I don't think I need to spend too much more time on that But I do want to spend a little bit of time talking about loot back um, so far we've talked about um Some physical interconnects. We've talked about how to kind of munch some physical interconnects. We've talked about um, some pieces where your, um You know pushing you know pushing raw network frames or tunneling traffic But the loot back interfaces An interface that everybody sort of conceptually knows about they all know that 12700 one exists They all know that one one or colon colon one, um exists But most people don't actually know that 127 000 slash eight is explicitly defined as the loot back address range in rfc 11 22 um specifically section 3.2 point 1.3 That means that there are 16 uh, million seven hundred and seventy seven thousand two hundred and fifteen ip's um Which is slightly which is one more than a a normal class c because there's no broadcast address That you can actually fake interesting networks inside of your own computer Now if you do go and you read rfc 11 22 the loot back interface is not allowed to leave the machine. It's on and there's unbelievably good reason for this um, but this does mean that you can put together really complicated networks um virtual networks inside of your computer That only use loop back addresses Okay, so putting together really interesting and complex networks inside your own computer Okay, that's one thing to do, but why would you actually care about having? um multiple loop back addresses beyond just 12700 one I'll give you an example that I'm using in my own house. Um, so for a variety of reasons. I not only run my own My own dns, so I have a bind server that does um Change resolving In my house, but for a variety of reasons there are certain networks that I need to provide a slightly different dns to um, I've got an iot network that I need to to fun munch a couple of the um The domains so that they go to different places and I Have a couple of situations where Certain services do not like ipv6 for various reasons and I need to resolve those as ipv4 only Now there's lots of different ways. I could have done this the way I ended up choosing doing this was that I actually have multiple um versions of bind running inside my firewall and one of them binds to all of the interfaces except for um 127002 and 127003 and the 127002 And 127003 both refer to new versions of bind And those versions of bind bind to to those ip addresses onto the standard port And that they can just run normally now Why would I do that? Why wouldn't I just change what port the wrong? Well, I can change what port bind is running on that's easy but Being able to trivially just move it to a bind and bind it to a new ip address It has a different, you know, it has some really interesting advantages mainly I don't have to muck with Taking over some other port that may that I may need to use later for something else Or that that already has some pre-established Fundamental function And on top of that I only care about these resolvers inside the firewall itself The firewall these don't need to be exported in any other useful way so basically that is kind of what um Loopback addresses are for they give you places where you can actually attach things Inside the computer to the their expected ports on different interfaces Or in the rather extreme example here. It gives you um About a trillion different ports Inside the computer that you could bind things to because you have um a full 127 127 0 0 0 8 network range as IP addresses that you can bind things to each of those IP addresses you can bind You know thousands of ports to This adds up pretty quickly So that is why loopback exists not only so that you can find yourself and do various things against yourself But so that you can actually build out weird and complicated systems For networking and services inside of your machine without them having to be exposed out to the internet Okay, I've talked a lot about really vague concepts at this point I haven't actually talked about any real world potentially honest example And so we're going to jump right into an example Um, literally that I this is actually an example that I literally have running um in production right now and um This kind of amalgamates and crams a bunch of these concepts all together and I want to talk about why I picked this example The the and the biggest reason I picked this example is I wanted to show How you can kind of layer these pieces together to get very interesting functionality that you otherwise Would have to create in some very odd ball ways So We're going to kind of start to the left a little bit and kind of work our way way right as I explain what's going on here This particular machine what i'm showing you anyway has two 10 gigabit Ethernet interfaces and a one gig inter uh ethernet interface that physically connects into two various switches The um the two 10 gig interfaces use lacp, which is that is 802 3 ad that is LACP 802 3 ad dynamic link aggregation. Those are all synonymous on terms It bonds those two interfaces into um a 20 gigabit per second bond So again, I can only get on a single stream 10 gigabits per second on that But I can get across multiple streams up to 20 gigabits I also gain by having lacp I do have some failover if one of the interfaces go down all the traffic will will switch over to the other one and vice versa Um just inside of that bond Now one of the things that you can't do with um At least you're not you shouldn't do with 802 3 ad or lacp is you can't have interfaces that are different speeds This this breaks all kinds of assumptions with the way lacp works if you do that um, so Let's say that you you want a situation where you've got a switch that that has two lacp 802 3 ad Bonded interfaces on it. What happens when that switch goes down? well On this particular machine. I do have a spare gigabit Ethernet interface that I have actually bonded to the bond so Bond zero is actually a member of bond one. It is the active primary Interface for bond one It then has a backup interface of eith 2 Now everything from bond zero and bond one is all on the same logical network They're on the same vlan. They're saying on the same, you know physical effectively the same physical fabric So they can actually get to the same places and what I can do From bond one is I can actually, you know, transit out on either the the the bond zero i.e. ETH 0 eith 1 which are both 10 gig interfaces Or if those are down for some reason it will fail over automatically The eith 2 there might be a little bit of delay just as Mac addresses move around and it gets a little complicated But that that is what is expected to happen now This is where the diagram forks a little bit bridge zero Has an ip address associated to it one two three four. Don't ever actually use that ip address That's actually a real ip address for entity down in australia um, but the The bridge for this particular machine is actually the primary interface for how it gets out to the internet So bridge zero is attached to bond one which is attached to bond zero and eith 2 And bond zero is attached to eith 0 and eith 1 Got it Now bridge zero in this particular case would be what most people would see as eith 0 or whatever your default network interfaces I can assign ip addresses to it. I can you know treat it exactly like you would normally do You know exactly you know like eith 0 Except that there's these other layers of pieces to the other side of it that define how it's going to work The other upside that I can do with the bridge is as you can see with vm 1 Is vm 1 can actually talk to bridge zero so it can actually transit out the same interfaces That um that the main system is transiting That means we're on the same network segment That means that vm 1 if it needed to talk to the host machine Doesn't even need to leave the The confines of the actual computer It can just talk to the bridge and the bridge will bounce back into the main host and that entire communication will happen there Now vm 1 also can talk to bridge one So there's two bridges in this network As you can see bridge one does not actually have an ip address assigned to it. It is literally just a bridge From the perspective of the the the host machine here It is just a bridge. It can't talk to it. Um, it could technically a wire shark information out of it, but it's not going to get To talk on that interface on its own because it doesn't have an ip address on that network segment And you'll also note that bridge one not only is attached to vm 1 vm 2, but it's attached to bond 1.22 It's actually attached to a virtual or a vlan interface So once we've got bond 1 you create a new on vlan, uh tag 22 of a vlan interface and then that gets a Attached to the bridge now what this means is that When vm 1 or vm 2 vm 2 specifically wants to talk out It is going to talk to the bridge and then that bridge is going to talk to bond 1.22. That means that all traffic that um comes out of bridge one will all Be tagged as vlan 22. So when it goes out of um, east zero east one or east two ultimately on the far side there It will actually still retain its vlan tag of 22, which means it will sit on whatever You know vlan segment is 22 Okay That is a lot to take in at this point. Um Um, and again, you know, is this real? Yes This is actually running in production right now That what i'm actually showing you is a simplified version of that. It's not even um, but my actual setup is is Um far more complicated than this and and slightly deeper and weirder Um, but why would you do this? Well in the case of the bond bond zero and bond one failures happens, which is die and um computers these days particularly servers come with so many network interfaces um That you can actually do these kinds of things. I mean, I I've got network. I've got servers right now that have Four or eight network ports on the motherboard without having to to do anything By the time you slap a couple of 10 gig network cards in there, you've got somewhere between six and eight Network interfaces on a system. There's really at some point no reason not to take a more advantage of these physical interfaces um that you have access to um And and when you plan for failure and you make these things work this way This just means that when your switch dies or you have to reboot a switch or something You know things may get slower But they don't fall over and die. So this is something to to kind of think about a lot. Um, when you're you're building these things It also particularly for the purposes of this um This talk it gives a really interesting example example of how the stacking or layering of all these pieces can go together now I kind of alluded to this a little bit earlier in that The the various distributions have different ways of configuring their their networks and they all do Red Hat Fedora sent us all use One of two ways. Um, either the old sys 5 Networking scripts or they they've all kind of moved in more recent just drives Um our distro versions uh to using network main network manager Um, devian has always had its own, you know network interfaces a way of bringing this kind of stuff up um, Ubuntu has net plan and system d network d and Like that basically every distro has reinvented the wheel on how you define how to bring up your network Some of this is good that this means that there's a new and different and interesting ways of how to come at this problem This is also bad because if you switch between your distros a lot You have to figure out how this works across a number of different distros Um, and not all of the p not all of the the systems that do this configuration can handle everything. So, you know, again stuff like devian Um, don't really handle teaming very well. Whereas red hat and fedora do so. I want to kind of um Show you a little bit of what the these configuration files look like. I know that There's no way that i'm going to really be able to explain all of this. Um, but uh, uh the the Pulled from the the example. This is actually a um more the configuration file that ends up generating um those image those those diagrams from the last couple of slides Um, you've got the the loopback interface. You've got standard e0. Um, we've then got a bond interface Uh called bond admin. You'll note that you don't actually have to to stick to the zero one two three four No, muclager. You can actually just kind of name interfaces. Whatever you want um yeah, um Um, you've got bond admin. You've got eth1 and you've got some other 10 gig physical interfaces Um bond one bond 119. You've got a a bridge interface another bridge interface and a couple of vlan tags. Um so Um Yeah, so so this does a lot that's going on there I'm going to leave this to the to to people who find this To go over and take a look at this this would take far too long to try and and piece it all out But this more or less will generate the the pieces um from the last couple of slides and Not to leave red hat sentos and fedora kind of out. I wanted to show a similar setup This does not actually generate the same thing, but I also wanted to get something in here That kind of showed off teaming. Um, this builds up a a a similar set of interfaces Um using the the the system that that red hat sentos and fedora time to prefer Um, and as you can see that there's a bunch of different files involved here um, particularly Network manager will will put these together in the right order. It it will purse them down and go. Oh, well, you know team zero needs, you know This up and this up and this up before it can it can do other bits and pieces And you'll also note that I've got a bond zero and a team zero in this this example They actually do slightly different things Um, and you can have both a bond and a team on the same system. There is no reason that that that doesn't work Um, but you can also see a bridge and some physical interfaces And you can see in the bridge that you know, that's where you know The ghcp actually gets pulled in and where none of the rest of these things other than team zero has dhcp and those kinds of stuff so Now This is a lot. I know that this is a lot. This is There's no way that I I can even cover a lot of these topics in in enough detail particularly in an hour To to really do them justice. Um, the linux documentation project length the first one there has an exhaustive set of links to other documentation a lot of it's going to be fairly technical And very low level, but it will also Do a fairly good job of trying to explain what's going on Lib team is a great place to go and learn about The network teaming driver stuff And the the networking tongue tap stuff. I mean pretty much anything in the networking directory is going to be relevant to this so Hopefully I've gotten through some of this and helped demystify a little bit of this I know that this is a lot to take in and there's a lot going on here. Um, if you have questions comments, um Please by all means, uh, you can find me on irc twitter. Um, email me I I I'm I try not to be hard to to find and I I'm more than happy to Answer questions or go over things. Um, again, I hope you found this useful And with that, thank you very much