 So I'm here from Warrant Systems and I decided Greylog is the logging system of choice here at Warrant Systems. I've been using it for a little while and we didn't do it a video of how to get started because I think other people should be using it too. It's free. It's open source, but it does offer enterprise support and some really cool enterprise features for those of you looking to scale and want to buy license and support. So this is not an endorsement by Greylog, but this is endorsement for Greylog by me. This is not paid at all. I just want to get that out of the way up front in case anyone was curious. Now logging is obviously something really critical and the more you can correlate all the different despair systems that you have into one container of logs, it makes your life so much easier. So this logs PF Sense, this logs Unify and a lot of other little things that we have here at the office. It has a lot of different input support, but the Unify is one we're going to focus on today in terms of how we created some data and, well, parsed data to figure out a weird anomaly we ran into with Unify. A little bit of background. We host the Unify controller within our server stack. This Unify controller we host is multi-tenant multi-site and those sites are clients that we manage networks for. So they're under contract. These are paid clients that pay us to manage their IT infrastructure, their computers or systems and of course their network. And for those of them that have Unify installed, which is not 100% of them, but we do have quite a few sites in there. We just handle everything for them and make it really simple, but that simplicity means we have to maintain a controller and that comes with some weird quirkiness when you do the updates. And we're not sure if an update did this exactly, but it is a weird puzzle that we haven't 100% solved, but at least we can identify and understand a lot better. And that's the way Unify handles syslog. Now let's talk about what ports you need open for Unify. You need port 3478, which is used for stun. Then you need port 5514 if you want to ingest syslog. I don't necessarily recommend it if you have clients set up remotely, then pushing all the data back to 5514 may be burdensome or may cause a bandwidth issue. And it's not always needed. The times you use it are more frequently to debug or troubleshoot something, not something you need on all the time. But if you do need a lot of syslogging, we do set this up for a client where we'll put a syslog server on site for them in order to have all the data there because it's usually more than just one thing you want in syslog. And it's like I said, if they have a lot of access points, for example, because the way it works in Unify is each device talks to the syslog server, each switch, each access point. Therefore, it can be a lot of different data going back and forth quite a bit. Now port 8080 is the other one that we have open. This is used for device and application communication. I do not recommend opening port 8443 up to the public internet, but at your own risk, it's just ill advised. I don't know of any security problems with it now, but that can always change in the future. The other ports are definitely optional and they're for some other use cases such as portal and things like that. And if a portal is needed, by the way, we'll just kind of set up a controller on site for that particular client. All right, back to the logging, though, because this is the very important that first it uses 5514. And also, as far as I know, it was always turned off by default. And the place you adjust logging in the Unify controller is right here. You go over and they have the remote logging option right here. So here's remote logging and enable syslog server. Now this is my house and I actually do have logging turned on and I have it sent to this port. But if you check this box, the default option is actually log syslog in that council to this controller. Now this controller is local to me and my house is not here. So this is the same way it works for our clients. They are remotely talking through the domain name we gave them over to this server. And we don't have port 5514 open. And what we thought was this was turned off for all the clients. But what are the updates turned it on? And we're not sure exactly what we know what happened around September 2020 from our correlation data because all the clients that have this turned on appear to be created after that date. Any client created before that date seems to default with it off. But this is what happened over in Greylog and how we seen the problem. Now I've redacted all the data at the bottom, but this gives you an idea. We've gone from 928 to 106 and we filtered for just the streams that are from LTS Unify and LTS PF Sense. And you can see on average and these are broken down into four hour segments. There's between 700,000 and just under a million messages coming in. Well, they're actually ramping up slowly. And that's something you're not really going to notice that slowly you went from 7 million to 9 million messages. Not enough to even make a significant difference of load on the Greylog server. Then all of a sudden 13 million logs it. Now when you get a bunch of logs, you can set thresholds to say, give me alert if a message aggregation gets above this. But those are alerts that'll annoy you because they're often just some flood of DDoS, some new scanner that went online and it found you and thought you look like an interesting thing to poke at. Now, one important thing. We're sending all of our PF Sense logs to this, including the dropped packets. If a bunch of drops come in, I just want to know. I want to know where they're at. I don't always trigger on them because they're happening all the time. But in retrospect, this is why we did it. So in case there's something we start scratching our heads about, we can go back and look at the history of it. And of course, we keep quite a bit of logging on here. So able to jump backwards and we can see the trend. But then it drops off to 3 million logs, 3 million, 1.3 million. Them jumps back up and then gets steady. We're talking like 19 million, 17, 18 million. So there's a whole lot of data going on here. Now, the next thing I want to do is start bringing us down to, how do you turn us into intelligence? Because the only thing you have right now is clearly there's a lot of logs, Tom, but they don't really give you any intelligence. This is how great a log can really help you parse that. Now you have to think about what you want to parse. So let's parse it really simple here. Now, the first thing I did was just narrow it down to just being October 6th. Then we're going to go ahead and create and we're going to create a message count. And all right, we've got 28 million messages, but this still isn't the most action item right here. We want to edit this and let's group by source IP. So who are these? And we know they're going to be IP addresses because it's PF sense. So go here and we're going to do a count. And I want this to be a data table because I need it to group by source IP. Give me the limit for the top 15. You can change that pretty easily to even more, maybe even the top 20. Then we can say function count, select field. And I'm not worried about that. Let's just see where we get with this. All right. Okay, we see these IPs, but let's narrow it down a little bit more. What if we narrowed this down and I'm redacting the IP addresses here, but we do see one IP at the top with 26 million. So that's clearly our culprit here. And let's actually add another group by. We'll group by destination port. I bet that would be really interesting. And updating results. Oh, all right. We have a winner really quickly with just two clicks that this is the problem child right here. And we're going to go ahead and highlight this value of 5514. As I mentioned, 5514 is the syslog server. So for some reason, and if we scroll down here, it's not the only one doing it. This is how we started discovering that some systems were sending a lot of data. Now this is where things get a little bit strange because the fix for the problem was really simple. Uncheck the box. Don't send all that logging data to 5514 because we're not collecting it. But now I wanted to say why were these things doing it? So we set up logging and then turned it back on. I won't do it anymore. This has been kind of an annoyance trying to figure this out because now that I can't reproduce the problem, I only know the history that the problem occurred. I don't know what logs are in there because they got dropped. And I don't know if it'll happen again. But I think because we've turned off this logging, it probably won't. Now I had this conversation that I said with Riley Chase who runs Hostify and he manages quite a few controllers and quite a few instances of Unify. And he's got a couple anomalies he had mentioned where he's seen large spikes in data. This may be correlated data. So I did share this information with him back and forth to see if he had insight into having seen the problem. Instead, he replied with, hey, I actually have a weird issue where sometimes certain sites use a lot of data periodically that seems really odd. And he's going to investigate specifically looking at this. And so maybe you are having this problem as well. And if you're logging all the data and yeah, maybe you're having these strange issues and maybe you even know why. If you could leave that in the comments below, that would be great. I did a little bit of searching and didn't really find anything solid to answer this problem. And once again, because I can't just click a button and turn it on and reproduce the problem, I don't know if it's going to pop up again. I do know we are double checking and part of our checklist when we create a new site now is to make sure this log is turned off. But outside of that, you know, it's just one of those little anomalies that happened. Hopefully this gave you a little bit of insight of how you can take something like gray log, use data to pivot. And let me know if you want to see some more videos on gray log and other features that I may be using it for. I plan to do at least a few more, but let me know what specifically you're really interested in or what you're stuck on. And maybe I can create a tutorial around that, but do feel free to check out my gray log video on how to get started. And by the way, gray logs documentation is great. So if you don't even feel like watching Tom's video on it, just go to their documentation. One thing of note to a few people playing this out, I had mentioned in my older video that you could download an OVA file. I believe that is no longer supported here in 2021, but they do have a Docker image. So there are other ways you can quickly deploy this about going through this setup procedure for gray log. All right, and thanks. And thank you for making it to the end of this video. If you enjoyed this content, please give it a thumbs up. If you'd like to see more content from this channel, hit the subscribe button and the bell icon. To hire a sure project, head over to laurancesystems.com and click on the hires button right at the top. To help this channel out in other ways, there's a join button here for YouTube and a Patreon page where your support is greatly appreciated. For deals, discounts, and offers, check out our affiliate links and the descriptions of all of our videos, including a link to our shirt store where we have a wide variety of shirts and new designs come out well randomly. So check back frequently. And finally, our forums, forums.laurancesystems.com, is where you can have a more in-depth discussion about this video and other tech topics covered on this channel. Thank you again, and we look forward to hearing from you. In the meantime, check out some of our other videos.