 Okay, so we're going to talk about troubleshooting networks with PF Sense and this also kind of leads into the question people ask a lot. Tom, why do you like PF Sense so much? Why do you like to put it at the head of your network? Well, that's what we're going to dive into a little bit here. So first, let's start with the network layout that I have, which is my computer is at 3.9. The PF Sense box has a WAN side of 3.150 and a LAN side of 40.1. That makes this the a 40 slash 24 network over here and we have one Linux box running and one Windows box running virtually. Everything in here, just so you know, is virtual. These are not physical boxes except for my computer here. So this is an idea of the network so you can kind of see how things are connected and kind of visualize it here. So I have this opened up on the WAN side so I can administer it from here. So anything you may see in the title bar if you're looking at the IP addresses is going to be from externals. I'm not doing it from inside. Also doing it that way so I can show you cleanly the data that's going through each of these machines and where they're connecting to. I've seen a real network and there's going to be a whole lot more of this and a little bit harder to sort through but I want to start with the basics so you can kind of get understanding of where these tools are in PF Sense and get an idea of how to start looking at the network connections. All right. It is May 2nd, 2018. We're running the 2.43 release of PF Sense. I also have in a package manager here. I'm going to go to install the packages. I have Entop 08.12 in here and I have that set up. I've done a video on Entop. It's pretty straightforward to set up, install the package, enable it, set a password. Do that just save you the trouble. You do that in the Entop PNG settings. After you install it, you check this box, enable it, come up with a password, bind it to an interface, hit save. Now, I went a step further and created a firewall rule. This is not recommended necessarily but it is for the demo here. I opened up port 3000 that it runs on and you can see 3.150 and 3000 is the port. I opened it up externally. Normal situation, you want to have that internal facing and you either VPN and remotely and look at it if you want to connect to Entop or it's running in your network and you can run Entop there. Now, I will mention with Entop, it is a little bit processor intensive so you got it. If you're running a really old slow machine for PF Sense, you may have some trouble with Entop keeping up. So just throwing that out there real quick. All right, let's take a look at the first tools. So we have the network layout. We have a couple of machines running. I have the Windows 10 booted and I have the Debian machine booted. We're going to go look at with diagnostics PF top. This is just a great go-to tool to get you started. I figured out where all the data is going. So there's just a whole bunch of established connections in here. And actually it's kind of funny if we filter them. The two IP addresses we have running here. So let's go ahead and oops, I'll put it in for typed wrong. This does it in real time. And by the way, don't press enter. So if you press enter, that will do that. So let's filter it to one host. So filter it to just this host here. You put host, then the name, wait a second, don't press enter and it updates. And now we can see every connection for this host. And if you can guess it, this is our Windows host, which has lots of connection. This is our Debian host. I booted it. Debian does not go out by default and make a bunch of connections. The one exception to that connection that you may find on a Debian box is if you have enabled the package survey manager that sends periodically packages or an automated updater, it will contact whatever server you told in there. Other than that, Debian by default is really quiet. And Windows, it connects to everything. It's got a lot of connections established just because we boot it. So let's go back over here to this host. Let's start our first connections. So I'm on my computer, as you can tell by it says Tom Spooter. My really less than clever naming, naming scream, and we're going to go ahead and SSH root 3.150. That is the firewall. But we have a rule that passes it through to the Debian box. So many SSH in. There's the Debian box. And there's the IP address of 40.50. And now we see a established TCP connection. So pftop in real time. Now you can run this from the command line as well. But they've made a really nice job of doing it in here. So when you got to start that tracing and going, all right, where is data going? What things are using what? This is what helps you a lot to do that. So you can string the commands together. So we found the host and we see the connections. And let's make some noise. So we actually have a lot of connections. I have DNS perf test on here because it will do a DNS test. So it's going to test DNS. And each DNS it does, obviously, fills up more and more state tables. Now you have a mess and you're going, how do I read this mess? So we're going to say, and port. Give me a second. There. I filtered it only for port 22. I could also filter it for port 53 to only find the DNS lookups. Now this is kind of that Swiss Army knife of tools that I like to use to get me started on why something isn't connecting, why something isn't working. If you know where it's supposed to be going, you can put host and host in there and start figuring out, okay, where's it connecting to? What's it connecting to? And why isn't it getting there? So this is one of the first tools is PF top to be able to start sorting out what's going on inside of PF sense on terms of network. Now you notice they all died. That's because the test is over. And if you're familiar with how UDP testing and that works UDP, it's short lived. So it sends out the packet, it got the responses, the state tables die. So if we go back over here, port 22, we can still see that I have an established connection here. Now, next thing I'll show you is this is just some rata here for understanding how it works. We're going to exit enter again, exit enter again. We're exiting and connecting exiting connecting. And now what's happening over here. Here's the established ones. Here's the ones that are closing, but they're waiting and then they're going to die off over time. So you're watching the countdown of them expiring. This tells you when they're going to expire in a minute. So that'll count down and then those connections will expire. Now, you can also, I believe there's even a filter for like active TCP. You can go to right here and a convenient click. And it gives you all the little options. So you can actually sort by when they're expiring. But this is one of those tools that is a great way that you can just start poking away and going, okay, let me filter. Let me find and let me sort out what is in the network, what's going on and what's doing that. And by the way, it also tells you things like the speed that things are connecting at, how much data is traverse size. So you can start really drilling down into here for this. Also, and I don't have this set up on here. If you have queues set up, and I don't like, so I don't have any on here, it'll list all the queues and the queue types. Let me see if I can pull up to my system. All right, and now here's all the queues and you can see all the data going through there. And I think we can have fun with it by doing this. I'm going to, I got my phone in my hand here. I'm going to call my office and you'll watch the queues change. And we just seen more activity in the VoIP queue. So once again, it's just an absolute Swiss army knife of tools to start organizing things. And I have queues and HFC traffic shaping turned on for all of my ports. So it's really nice because I can see in real time each of those calls and you're going to watch in seconds can refresh and these numbers are going to go back down because there's nothing in the VoIP queue. I have it set up to prioritize the VoIP queue in terms of bandwidth. So let me close that. Other things you can do when you want to really dig in and do some diagnostics is we're going to go ahead and go over to packet capture. And we don't want to just do well, we'll do the Debian host. So that was at 50. We're going to filter it like this. Enable Persecus mode interface LAN. So I wanted to make sure anything going to this address here gets captured. And we're going to go ahead and start that capture. And we're going to go in exit enter. So we now we established another connection that the packet capture can see. Then we'll do that DNS perf test. So we just filled up and now we're creating a bunch of stuff going on in the packet capture logs. Sorry, we did some stuff. Stop, download capture. And of course everyone's favorite tool, wire shark file. And now we can see data going back and forth and start tracing out packet capture logs. It's nice to have this built in. So a lot of times this is the trouble is getting the interfaces to go into promiscuous mode to do this. PF Sense makes this really easy. So if you want to dig into packet capture to really understand what's going on inside of a network between public, private interfaces, whichever interface you want, because PF Sense runs at the head end of your network and you have designed a network to pass all the traffic through it or at least I think that's the way you should do it. It now understands and has all those details. So I can do packet capture. I can do filtering. I can start narrowing down data very, very quickly with PF Sense. It's an outstanding tool for doing this. And it also just in the capture here. This is nice when you're working remotely and just logging into the client's system. You can do a basic packet capture and start going looking at the back and forth. And this is just the log level detail that was going to show up here without actually doing it. And then you can just view the capture whenever you want. Now it only remembers the last capture. It doesn't have like a system that organizes all the files. So you got to download them out. But it's an amazing tool to be able to go in here and just say packet capture that network. Let me sort out what it's doing. Let me dig into it a little bit. And it's also handy for things like when we're talking about the Windows machine being right here. I did this earlier. It's amazing how many connections the Windows machine does just on a reboot basis of all the different outgoing. So that's another tool that is also really handy. Those are kind of your tool things that I use a lot. The other thing we do is trace route and test port. Now this is great because you can do host name lookup, port lookup, source lookup, where's the source address, WAN, LAN. And if they have multiple external IPs they'll all show up in the list here. And this is nice because we get into the machine often externally and they say I can't get to this website or I can't get to this. We can start here sometimes not even looking at their computer and go can they see this? Can they see this network? And you're going from that PF sense box to wherever the destination is. So being able to test the port and of course you can do this internally. Do the host name lookup, test the port internally. You can even set a source port for where it comes from. So when you have to do, you know, is the port open? It's actually a really handy tool to be able to do this. We sometimes when we're setting up remote ports we need to make sure a port's open. It's a quick way to go in our own PF sense system and go all right can it see it? Okay I can see a response from this. Log into another box and see how it sees the response from it. So another great diagnostic tool just right there. And just standard DNS lookup, you can do this here. So if you just want to look up LawrenceSystems.com it tells you based on all the DNS servers that are available to it, its own DNS, the local DNS and external DNS that I have programmed in there and away you go. Then you can right away jump to a tracer out to it or add an alias and start building something off of that information if you wanted to create aliases. Handy having this here, I mean it's great to, I'm used to doing it from a command line, but having a web interface that you do it then you can just start building on the rules from there if you have to do something really easy to do, especially if you're using aliases to block things. You can go in here, do the lookups, do the add aliases and start building lookup tables. Let me do a video on that sometimes. A lot of people ask about blocking things. There's also diagnostics for sockets, routes and all kinds of other little details in here, but the one ones I'd use are the ones I mentioned right there. Those are the absolute really handy for starting to drill into some of the network stuff. You can do the track graphs and package logs and some of that, but where we really like is the end top PNG. I load this on there. I don't always leave it running because depending on how much horsepower the machine has, but when you got to dig into some of the network diagnostics, this tool is wonderful for that. So I've done like it's an entire video on it, but we're just going to jump into just showing you a couple of the host options and how they work in here. So here's my Windows host and right now there's probably nothing going on. Nothing really going on here. It's got one connection to there, which I don't really know what it is. I don't know where that's going. Let's take a look. This is kind of neat. So it creates one connection to there. All right. I know where that one is. I know what it's going to. That's part of ScreenConnect hosting. So it's hosted. It is connected to ScreenConnect. It's quiet. So what we're actually going to do here, and I'm bringing it over so you guys can see what's going on. Here's our Windows machine. We add it to here. It's in a VM. I'm just going to restart it real quick. Oh boy. Now it's decided an update. Of course it's got an update. So I guess what the pause this video while it does an update. All right. Now Windows is booting and what we're watching here, this is the network. So we're going to watch for when the network peaks up up here. Hopefully it doesn't take too long when it wants to update. And once we see it do the peak here, that's when we know it's starting to establish network connections and we'll jump back over to end top and see where Windows wants to call home to. Because really, there's nothing installed in this Windows other than the one screen connect session that we have. Getting ready. I'll speed this up. All right. Windows is booting. It's pulling in some network traffic. So let's go to end top and see what it's talking to. And we'll start with the flows. So here's all the places it's starting to talk to. Here's the flow talkers as they call it. And this is kind of the cool feature too. You click this and it gives you an idea of everything it's phoning home to. Let's go back over to flows. Then go back over to talkers. And then over here. And it just doesn't auto refresh. It does play, but it does not auto refresh as they come. So you're starting to see it's phoning home to Redmond, phoning home to Menlo Park. Apparently it thinks this is in Chicago. I'm sorry. Oh, that is ours. That's in Detroit. So my bad. This is the one I was looking at. It's Iowa and City of Cheyenne. This one in Kansas is ours. Our G.O.I.P. is not always the most accurate. It insists we're in Kansas on that particular IP address. It's not. It's located in Detroit. So these are all the things that Windows does. Go back over to it. Sometimes it makes a connection overseas to Dublin. Now it's actually not made. Two of them over here. So now it has more data going over here. So we got this one and this one. So San Jose. Thanks, Windows. You're just going all over the place. Cheyenne, that's us. And there's the machine itself. So here's Windows sending all the data. Now this is how you can start digging into where your data is going. You can look at the protocols. You can look at the traffic. You can dig into it a little bit deeper. Now I'm going to go ahead and open up ours. Because I wanted to show you all the details of what we have running on our network. So all the different places that we're doing data. And I chose my computer. This is the 3.9. Like I said, this is mine. So you get a better idea of it. It's also fun to see just how many connections and where they're all going. I don't know. What is going to Spain? Some website I have apparently is open in Spain. So this gives you a better idea of the traffic. And let me show you what it looks like with more data. So you can see the TCP. Since I started it, this is how much data I've sent across the network. Package distribution. The current flows of data going back and forth. Protocol breakdowns. Top peers that I'm connected with. And if you didn't notice 2.24 happens to be this. There's a lot of data because I'm running a screen session in here with Windows running. So it's actually sending quite a bit of data right now. So like I said, between all these tools, starting with pftop, just looking at the basics and end top being the ultimate, we really need to dig into what this thing is doing. These are great tools you can use to try to understand where all the data on the network is going and start breaking it down. This also is handy as well. I went over here to the time slice and you can start saying 1 hour, 10 minute and start breaking. Okay, where is the data going in terms of like putting it on a graph? And because I haven't been running this for very long, I don't have it running all the time because it just is a little bit resource intensive. So you but you can start to look in here and go, okay, let's look at the amount of data. You can also use this to look at the overall aggregate data going across the network. So whoops, I actually went to the wrong one. So this would be my data slice. There we go. So this one's on ours. This is my computer. A little more data. I've been running it for, I think I turned it on yesterday. Yeah. So you can see that when I'm downloading stuff. And if I upload a YouTube video, I'd see that part two on the cent. I see peaks in the cent when I upload a YouTube video. So the last thing I'll show you is the traffic graph that I usually have put on here on the front page. This also kind of gets you started. Is there a traffic problem? And you just go here. And it's just the traffic graph is a little service status, which I have here. And then you have the you can put the firewall logs, interface statistics, some of that stuff in here. And then of course, the traffic graph. And then just move it around, put it where you want. Save the positions. Whoops, how have I closed it? But that gets you an idea too of just how much bandwidth the system's using before you start drilling in. So let me add it back. There we go. Save. All right. Hopefully, this is helpful. And we'll start your troubleshooting in PF sense. It's also fun for a lab environment. If you want to play around with it a lot and kind of get a better idea of how the connections are going in your network. But like I said, it's an amazing tool. They have just even more diagnostics. But like I said, the ones you can start with are the PF top and top PNG for your real in depth look. But just starting with PF top and just start looking at it or looking at the cues and how things are going really are the most of the time all I need to troubleshoot a network issue that's going on. So I can kind of like, you know, start and go, oh, this is where the connections are going. All right. Hopefully, it's helpful. Leave your comments and feedback below or in our forums. Thank you for watching. If you like this video, go ahead and give us a thumbs up. If you have some feedback, leave it in the comments below. If you'd like to subscribe to our channel, hit that subscribe button, hit the bell icon to let YouTube know you'd like to know about new releases on videos. You can also find new releases on our website, lauranceystems.com slash blog, where every video automatically gets posted. So you can always find our videos, whether YouTube notified you or not. Also, if you'd like to hire us for consulting services, go ahead and hit lauranceystems.com, fill out the contact form, tell us about the project you would like us to help you with. We work with a lot of businesses. We work with a lot of other IT people who need services done. Also, if you want to help the channel out in other ways, we have a Patreon. We have right below me a Amazon store where you can check out some of the products we've reviewed and as many of them as available on Amazon. You can also check out the things we love. And that's an ever-evolving list of discount codes, offer codes, and different software and affiliates. You can find that on our website as well, or just follow it in the link below, right to the Things We Love landing page, including the hot sauces we recommend, which is always changing. All right, once again, thank you for watching and see you in the next video.