 Tom here from learn systems. Let's talk about open source mesh VPNs and calling a VPN. I guess is accurate They do solve connectivity problems of devices on separate networks and allow them to talk to each other But they don't do it in the same way a traditional routed VPN such as wire guard Open VPN or IP sec tunnels work And that's why I wanted to make this video to kind of dive into this topic to show the different use cases for them Because they're not replacements for those that are VPNs. They just solve problems differently But that different problem you may have may be the reason you need one of these type of VPNs to solve that problem and the two products I'm going to be talking about today are Nebula and Zero tier now. I haven't done any videos yet on Nebula as of the time of this video if I do I will update that and Leave it linked below. I will leave a link to my zero to video They both work on the same principle of how they function using UDP hole punching. They have different implementations of it So there's some nuances that are different, but that's not the part I'm going to focus on because they both solve problems in a very similar way That's the thing I want to talk about is how they differ from traditional VPNs before we dive into all the details Let's first feel like to learn more about me or my company head over to Lawrence systems calm If you like to hire short project, there's a hires button right at the top If you'd like to help keep this channel sponsor free and thank you to everyone who already has there is a join button here for YouTube and a Patreon page your support is greatly appreciated if you're looking for deals or discounts on products and services We offer on this channel check out the affiliate links down below They're in the description of all of our videos including a link to our shirt store We have a wide variety of shirts that we sell and new designs come out. Well randomly So check back frequently and finally our forums forums.laurancesystems.com is where you can have a more in-depth discussion about this video and other tech topics You've seen on this channel now back to our content now as I said I'll leave a link to my deeper dive into zero tier, but let's at least talk a little bit about nebula and How it's fairing and I say that because zero tears well vetted lots of clients lots of people use it And it's a pretty solid protocol that I still think is really great Nebula is a little bit different and it's only self-hosted at this time So if you stand up a nebula system, you have to stand it up all on your own from the Receiving nodes the beaconing nodes they refer to as lighthouses all it has to be done on your own And if you're wondering of about security or scalability of it I'm on the slack engineering blog here and it was born inside of slack and just to read the blur We get you an idea of how it scales What is the easiest way to securely connect tens of thousands of computers hosted at multiple cloud surf riders and dozens of locations around the globe? So it does work at scale and let's go take a look here They have a github you can download and get started But by the way hundred percent command line driven very manual way you set this up and to find networking as a newer company I actually spoke a little bit on Twitter with the developer really great things are doing here This is all early phase as far as putting an interface on it the technology the underlying of it's very well proven By slack obviously you've probably heard of them they're a fairly large company connecting thousands of devices using this and Without a front-end though that may be a little bit more difficult But it's a great DevOps tool and allows easy scripting for deployment And they have a deep dive video that is actually really great if you want to watch to talk about Scalability problems when you're the size of slack and why they had to invent this product And they are aware of zero tier and the problem some people have had with zero tier is despite It's being a wonderful product that I've recommended many times and have solved problems with the challenge again with zero tier Is the fact that it's harder to host the control yourself So you have some level of trust that zero tier will host the beacons that do all the connections That does not give zero tier immediate visibility into your network But it does mean part of that control plane is held there But there is a way I know and I'm not gonna get too off topic, but there is a way. Yes. I know you can host zero tier yourself There's a completely separate project that allows for this and like I said, that's kind of off topic I'll leave a link below in into that now zero tier does have really great documentation lots of details No way their installer works that's something right now as of this moment's January 5th, 2020 It's kind of lacking over at Nebula. They don't have a ton of documentation just this write up and just this video And if I get around to doing a tutorial on this, which I think I will Let me know in the comments below and that will encourage me to do it the System is relatively easy to set up But of course there's a lot of expanding and scripting you can do on top of that Now let's talk about the differences in these vpns and what makes them so much different than a traditional vpn solution And we'll start out right here Now first thing is privacy vpns that use routing such as wire guard or open vpn being the two most popular That's what I have listed up here That's like your pia by the way have an affiliate link down below if you'd like to sign up But if you're looking at a privacy oriented vpn The goal of those is you take all the data that's on let's say a laptop and we encapsulate it in a vpn Through one of these privacy networks and now all the hops in between such as my service provider Don't get to see what information i'm traversing and my public ip address becomes that of the privacy vpn This is not a problem that can be solved easily by zero tier or nebula not designed for that at all There are ways you can make routing work in it, but it's not the way that solution works Now let's go down here and talk about like site a site b and site c And this is a traditional vpn setup that you'll see where site a needs to access resources on site b And on site c and site b needs to access resources on site c So you build these vpns and there are routing protocols that allow this to work quite well Where things get a little bit more difficult, but yes those routing program protocols do work for this is add site D e f g put a thousand of them in there Now you have a lot of complexity and if you don't build either a hub smoke system that creates a central point of failure You create this more mesh style is what it's referred to where you have all these different vpn Talking to each other across all these different sites and all these different rules And that becomes kind of a complexity management challenge and that you know It becomes that much more complicated when you have an outside user Where does that user connector they connect here? We just bring them a vpn connection to this firewall and then there's more routing rules to route this user to this site But they also need access to the other sites and that problem scales out So this is where traditional vpns. Just don't solve that scalability problem Let's introduce you now to the way zero tier and nebula work Now zero tier nebula set up a beaconing server And I say beaconing server and it doesn't necessarily have to be in the cloud and a matter of fact in nebula If you put it in the cloud you have to put it in whatever cloud you want to put it in It's very platform agnostic. You could also just run it at one of your locations that has static IP That is the first core of the way both of these work Zero tier solves this by having their own beaconing servers And the beaconing servers are a way to know the location of all the devices And if you first glanced at this you would probably think that I'm solving the problem by everything Contacting the beaconing server and routing through it But that's not how it works and that's where the diversion comes in from a traditional vpn solution Matter of fact one of the neat things that both zero tier supports Well, zero tier actually natively is built this way with many beaconing servers globally is You can add more than one in both of these devices both of these scenarios nebula and in zero tier Well, that means they have the ability to simultaneously talk to more of these servers This is how you build the redundancy, but it's still not clear right away how that actually helps If you're thinking okay now they can route through this one or that one, but now I've created choke points That's where the udp hole punching comes in to solve the choke point problem And by the way, you notice that it's not the firewalls at all. They're only providing internet access at this point They're not even involved in the vpnning at any level. You don't do any firewall config changes They all talk to these servers and then the server figures out where everything is and starts creating udp hole punches to get the devices to talk to each other So site b has this particular resource the server over here site c And site a and both of these tools do the same thing They add an extra adapter an extra network interface standard network interface gets added to these whether it's windows linux Anything that can run their client software So the client software is actually getting loaded on every individual host not on the firewall And the firewall doesn't have to be aware of it in any way So you take the host adds the extra network adapter. As a matter of fact, I can show you what that looks like Right here's my zero tier adapter on my laptop, which has this ip address So zero tier works this way nebula. You can call it whatever you want. You can call it a nebula adapter Zero Tara starts with zt for theirs pretty straightforward though It just adds an extra adapter and then you just start doing everything by ip address But i'll get to the point of how this works shortly here So each one of these devices has that extra adapter added There is a process that is out of scope of this particular talk where they are authenticated The authentication method for zero tier versus the authentication for nebula are very different in implementation But the concept is the same you're going to have an ip address assigned to every single host a Subnet slash network and ip added to these then there's firewall rules that are applied at the host level Nebula goes a step further and has really slick group systems for security And it's one of the reasons I think nebula is a really cool installation for a dev ops team And which is really what slack is when you look at the way they have to manage her back end That's one of the reasons I want to review it as a solution because it goes a little bit further And has a different set of rules compared to the way zero tier does but zero tier does have rules as well So it's not like all nodes can instantly talk to each other on this network It does have some defined rules that make them all play nice now Let's talk about how the magic happens, which isn't really magic. It's just udp hole punching And I tried to do it as simple as possible for a very complex topic We have each of these that are going to have their private ip address assigned to them by zero tier or nebula And these devices reach out to the firewall and talk to the beaconing server or all of the beaconing servers They have a process by which they figure out which ones to notify their existence So they're notifying their existence and what it's paying attention to the beacon server wants to know what the public ip address is on each firewall So this device and this device both reach out to one dot one dot one Let's just assume we're just using some public ip addresses here for assumption They reach out to that and then The beaconing server says okay, this one is at eight dot eight dot eight and this one's at nine dot nine dot nine And then it tells a little bit of a lie Via udp and I see a little bit of a lie because that's the best way I could think up to describe it It's going to spoof it. So whenever you have a firewall open up There is a udp path that's going to open up and a high port number is assumed It does know what high port number that is going across here It's then going to bend it so to speak It's going to figure out what port was opened up over here for that particular udp piece of traffic that was initiated by this device through this firewall and find that high port number Then it's going to find the same answer over here of what the high port number is there Then it's going to take those two ports and Get them to talk to each other without going through the beacon itself Now if for some reason the firewall really doesn't like it in edge case Yes, they do have the ability to route through the beacon But I have actually tried this on quite a few different firewalls and quite a few different rules And it's pretty amazing how well this works We've even done some testing with some friends, especially with zero tier We were shocked at how few large corporate networks have any type of Defense so to speak against this and I say against this because we were doing it for kind of an X filtration Demo to can we get this out there? Can we get ip addresses inside of a very large enterprise company that my friend worked for? And he was shocked that his security team didn't knock on the door. He had the blessing to do testing That's what he did for this company on it was to test things to test their security And this one was an interesting thing that even large enterprise firewalls don't seem to even care that these udp holes happen And that traffic is you know getting back to these devices Protecting against it is actually going out of scope of this because it's not something that's easy to protect against because so many things actually use this This is not unique to nebula or zero tier udp kind of has to pass back and forth Passing udp back and forth right here between two different devices and then kind of bending the rules a little bit of udp I it's still kind of a good use case for the way this works And what makes this also interesting is it doesn't have to leave the network Let's say you had two devices and we'll go ahead and duplicate this device And this device was subnetted, but physically behind the same ip address But maybe on a different subnet so there may be rules where two different devices can't talk to each other But if you take and assign those devices to one of these networks both Works the same whether it's nebula or zero tier This device despite being on a different subnet from here is able to actually just loop through the firewall and get close to line speed So let's say we only had a connection between these two sites that was or even between the beacon That was a one meg connection But you have a gig connection internally in the network and the network is able to route because remember I said they would be on different subnets so they're able to route between them It will actually loop the internet connection between two internal devices at close to line speed There is a little bit of overhead and I demoed this with zero tier where two devices cannot ping each other They are in separate subnets that have rules that say these subnets cannot talk to each other But the ports are opened up on the device and it kind of creates a loop that that loop instead of going all the way across Actually kind of like an internal loop to get them talking It's actually really fascinating how all of that works and it's all initiated over at the beacon So to say that these solve the problem the same as a routed firewall would be Misleading because routing firewalls still have a very strong purpose For example, privacy oriented vpns like I mentioned at the beginning are definitely better suited to a routed firewall such as wire garter open vpn Also use cases I have for example My lab is remote when i'm at home And I want to be able to take my laptop wherever i'm at and get into my lab And I don't want to install a device Extra network adapter through one of these tools and load it on every single device I play with in my lab Or some devices that you may have in your office or network May not be easily Well, not easy at all to load that particular tool on there So a routed network would make a lot more sense So you vpn into the office and you have access to all the resources and things that are over there Especially like I said for devices that may not have any ability to talk to it Now both zero tier and nebula have the option for adding specialized routes to get something on the inside of a network and do that but that's kind of a Outside their own use maybe it would work for you But you can see how that may be a bit more complicated in a traditional vpn is of course probably more desirable Now both of these Do not have any type of actual user authentication either Not natively built into either system versus a vpn if you want to revoke a user you can if you want to revoke Someone zero to you do a little differently you invalidate that particular node and the same thing with Nebula you're going to invalidate a node by revoking its signing certificate So both of them have ways to do it. It's just handled very differently and by the way zero tier and nebula as of right now January 5th I'm not aware of any deep integration they have to Tie into something let's say like active directory or other User authentication databases that you might have so once again, they may not solve those type of problems But I wanted to raise some awareness because they are excellent solutions. They are scalable They have been deployed both at scale zero tiers actually got quite a few clients and I know we've solved some people's problems especially with some unique server connectivity that they needed with roaming ip addresses and different devices That needed to be out in the field and get data back Without thinking about vpnning all the time or dealing with any troubles for vpn and zero tier just solve that problem And for slack. Well, they're pretty big and I think you get the idea that yes If it works for slack it probably scales for whatever project you're working on and it's a great way for You know, you have multi cloud systems You need a lot of connectivity and you want to tie in some dev ops engineers Watch that video they did over at define networking and I'll leave a link to that Breaks down a really solid use case of how they handle it Why it's a very secure solution and how security is handled within it From a deployment standpoint and so you can manage things and they're using ansible I believe for some of the deployment demos They don't have any public facing you kind of do your own scripting right now But that's right now and to find networking is kind of a startup And like I said, it's something I'm keeping an eye on because I like having these type of tools to Solve problems people bring to me or clients may have and I go, you know, what would really work good? One of these solutions, which is why I did this whole video is just to kind of describe They're not dropping replacements for the year traditional vpn. They solve the problem differently But sometimes people have different problems that need to be solved and this is just more tools I wanted to make you aware of once again, I'll leave links all below if you want me to do that demo on nebula Which I'll probably do but leave some comments and let me know and that'll probably encourage me to do it sooner Thanks, and thank you for making it to the end of the video If you like this video, please give it a thumbs up If you'd like to see more content from the channel hit the subscribe button and hit the bell icon If you like youtube to notify you when new videos come out If you'd like to hire us head over to laurance systems.com fill out our contact page And let us know what we can help you with and what projects you'd like us to work together on If you want to carry on the discussion head over to forums.laurancesystems.com Where we can carry on the discussion about this video other videos or other tech topics in general Even suggestions for new videos. They're accepted right there on our forums, which are free Also, if you like to help the channel in other ways head over to our affiliate page We have a lot of great tech offers for you and once again, thanks for watching and see you next time