 And welcome to the homelab show episode 47, CrowdSec and using open source threat intelligence. Well, I should have probably titled it like participating in open source threat intelligence because CrowdSec is a two-way street here of you give some data, you get some data and me and you are going to dive into it. So we think it's a really neat idea that they're working on here. It's definitely, we weren't sure if there was a, it's not going to be probably the longest show. It's going to be more like, hey, go use this product. But of course we're going to tell you why you should use it and what they're doing and how you can participate in all this. They've done a lot of really solid engineering. Jay, I think you've got at least one video done already on CrowdSec, am I correct? Two. Two, okay. And then I'm working on some videos too that'll be more in-depth tutorials about the product. But today we wanted to, you know, dive into talking about things a little bit here about what it is. But before we get into that, we got to thank a sponsor of the show and that is Linode. And Linode is actually a great place to test out CrowdSec. So there's a good alignment here because CrowdSec is a tool that allows for public data to come in and public threat sharing and looking at logs. And, you know, you need something public facing. And if you don't want to try this on your home machine, spin up a Linode instance, load CrowdSec on there, leave all the ports open because that's what makes this fun. And now you can let them, the internet I should say them being whoever wants to probe systems, not always, not all probing or looking at systems as nefarious, but being able to log all that. So this is a great place to host that. And we do have an offer code for you, which is go to linode.com slash homelab show to get signed up. It's a great place to do a lot of the projects we talk about here on the homelab show. If you don't have your own servers or don't even want to run your own servers, let Linode take care of all that. Save that power bill from home and trying to stack it all up there. You can just, you know, set it all up in Linode. Well, at least it's an option. It's a great way to have a public facing thing. If you want to build your own VPN, learn more about networking, tying into things, Linode has been a great sponsor of the show. It's where we host this podcast. If you are listening to this, you downloaded it, you downloaded it off the homelab.show. You downloaded it from a Linode server where it's been. Me and Jay, there was a whole other discussion me and Jay had about that just before the show. We will mention it here, tracking IPs and things like that. Well, maybe we'll do a project video about hosting some things on Linode because you don't want to necessarily host everything inside your lab. But anyways, if you don't head over to Linode, use our offer code and thank you for sponsoring the show. Appreciate it. All right. Now crowd sec, there's a, the challenge is, and we'll start with some of the commercial products. So obviously, you know, working in the commercial space and you work in the commercial security spaces, you need some threat intelligence data. We want to know who the bad actors are. The IPs that are attacking and sending these out. And a good example was when log4j was on, well, this is a hot topic just a few months ago and it's not exactly but fully mitigated because patching takes a long time and some people just don't do it in any timely fashion. So, but one of the things that companies and Grey Noise is one of them I've looked at before, you can look up some of the IP reputations for free on them. And you've seen that it's not like the internet was attacking log4j. There was a few listed IP addresses and not the best mitigation, but at least some mitigation for slowing down an attacker would be, hey, we know like these 50, 100 IP addresses, whatever they are, are looking and actively trying to exploit log4j. This is a list kept by companies and Grey Noise is an example of those companies. Like they have sensors out on the internet and they're listening all the time. Their sensors are somewhat obscure and hidden. They're going around looking for this intelligence data, Grey Noise compiles it and they're among the companies that sell these intelligence feeds. Now Grey Noise does have a free system and it's not a video about Grey Noise, just wanna use them as a commercial example of this. And this is where CrowdSec is a really interesting tool. So, while you have these threat intelligence and we can mitigate things by blocking those small list of IPs that are doing a specific attack or maybe when we know some things that command a control server, those feeds are expensive. The commercial ones, if you pay for a commercial license for using these like for a really solid IP block list, threat intelligence, reputation type list, it's expensive and it's not something necessarily a homelab person would really do. And the people at CrowdSec actually have a background in hosting so they have a good understanding of where the problem is. Now the solution a lot of people would start with in for years and I don't think it's a bad tool but you have tools like fail to ban. But fail to ban is kind of a lonely tool. You see someone poke at your server and then you ban them for trying too many attempts. That's something that they shouldn't be doing, like trying a lot of different passwords. CrowdSec takes this to the next level and they said, you know, we could probably get a community project going. We could probably build an open source project and an open source intelligence platform. And with this open source intelligence platform, they kinda do it in a two-way street. You get, and it's a trade-off for here, you load the CrowdSec app on your system. You get their threat intelligence feed. Then what CrowdSec is able to do is you can parse logs. You can even, we'll get into some minute, you can even parse existing logs. So I load CrowdSec today, but what if I had six months of logs? Well, I can actually retroactively tell CrowdSec to parse them and look for information in there. And this builds that threat intelligence feed. And these are all, you know, public-facing logs. You're not invalidating any of your privacy by doing this. Like a web server, for example, that you have running an SSH server, you have running somewhere publicly where people are hitting it. And they're only looking for public IPs, not private ones. That's not of interest to them. So you have this public-facing server and you're wanting to know, you know, was someone trying something bad? Was one of these IP addresses on that list? And then it sets up a bouncer to stop that from happening. So it's a two-way street. You have all the data ingressing in and then you have CrowdSec sitting kind of in a layer in between of I'm gonna analyze your logs and if one of those bad reputation IP addresses are in those logs, we're gonna go ahead and just block them and stop them from coming in. It's just a really clever system. The trade, as I said, is the fact that you are also including your logs, as everyone else does, but in trade, you get their threat intelligence list. And I was looking at their website. They actually have, right now, what was the, how many different IP addresses around there? I'll follow you. I forgot to- Well, yeah, there was quite a few last I looked at. I don't know what there is now, but while you're looking, I, I, I mean, CrowdSec is just one of those solutions that, you know, makes me smile because it's just like you were saying, we have all these threat intelligence companies and they could be charging tens of thousands of dollars and they sell this to big businesses, you know? And honestly, it'd be really cool if you are a homelab person and you have no problem with tens of thousands of dollars a year for a service. But I think I speak for the majority of us when I say that's not something we can do. So CrowdSec comes along and I kind of feel like they're disrupting this entire space. And I just like it when companies do that, you know? It's not that I'd like to see other companies have problems, but it's just, let's do it the open source way. Let's get everybody the information. Let's let everybody, you know, benefit from this and use a community approach to it. I think that's just, you know, so awesome that they decided to do that because how easy would it have been for them to just say, yeah, we want a couple of grand a month for the service. Yeah, and this is really, as Jay said, turning it around a lot is an interesting way to look at it because I understand completely why large businesses that sell these, and this specifically is like, you know, IP reputation feeds. Well, the reasons it is expensive to do is getting all those sensors out there and analyzing all the data and bringing it back. That's a, I mean, granted, yes, we have a lot of automation, but there's still a large intensity of labor and your expense of setting up all these servers, monitoring them, ingesting all that data. And then you have to set up because no one's gonna poke at something that doesn't look interesting. I mean, they might, there's always probes and scans, but the real threat actors, they poke at things that are interesting to them. They go, oh, look, a WordPress site running this version of software. Oh, look, there's, you know, on a patchy server that's not so patched. And that goes, that's what we wanna text. So these guys have to set up honey pots that look like something interesting to gather this data. Now, this is not a replacement for advanced levels of open source intelligence threat detection. This is more focused around IP reputation. Now, currently they have 2.1 million rogue IPs that they're keeping track of that look like they're not doing good. That's on their site right now. So this is really cool. Now, the way they're turning it on their head, though, is you gotta look at it as a distribution of labor. If you write a really solid piece of software, and this is what they did, their installer is stupidly easy. There's a little nuance. We'll talk to the developers at some point about it. Me and Jay found something that in certain scenarios we've had a problem with with some ports they use, but for the most part, if you have engine action or patchy, common software, open light speed is what the exception we'll talk about shortly is, but if you're doing one of these things or even WordPress site, WordPress is where this platform, many, many people choose, or maybe this is very directly related to HomeLab, you're running engine X and you're running NextCloud on it, which by the way, Jay just released an entire, how to get going on NextCloud video. So- Brand new and completely updated for 2022. Yeah, so pretty in depth. But any of these things, you know, a lot of people, I prefer to keep as much as possible behind a VPN, but at some point for ease of use and access, people open up the ports to things. And this is where CrowdStack says, hey, if we write a software, it's really easy to install. And then the distribution of labor has changed. You're just adding yourself as one of the sensors. You've already did the hard part. You've created something that would be interesting for a threat actor going home. We want to look at these, we want to look at things like a NextCloud server, we want to look at things like a WordPress site, but the distribution of labor is you loading some software, you're taking care of that part of it, and you then send your log data, parse through the CrowdStack app. And it's, by the way, completely transparent what they're sending. This is why it's important because no one wants to load some app that sends data back to the mothership because they could completely spy on me. But when they do it all open source, you are implicitly understanding what they're sending. So I'm perfectly comfortable putting this on my servers because I know what they're doing. They're very clear on it. We parse these logs, we look at these IPs, we see the ones that are doing something bad. Then the data goes up to the mothership and it's compared because if a group of disparate websites, all these different homelab users, all these different commercial people like myself using CrowdStack on our websites, me and Jay both have it loaded, we have it loaded on the homelab show. If a threat actor attacks these completely unrelated sites, those are those intelligence points that CrowdStack goes and analyzes. It's not one attack that will do this. It takes a reputation building. This is why they refer to it as a IP reputation system. But by doing this, now they have that intelligence going, all right, we just seen these different sites all get attacked in this same methodology by this singular IP address or maybe these two or three IP addresses. So by doing that, they go, oh, put this on the ban list, then their system puts on a ban list and it sends that data back down to the bouncer side. So you've got the ingestion of logs and you've got the bouncers that you set up. And it says, hey, this IP bad, block it, get rid of it. So it then puts that on the list. It's just a really simple, so to speak. I look at it, it's just a simply orchestrated, beautiful way of doing it. And it's very, very effective. This is made CrowdStack popular very quickly. And wow, what a cool idea. Now, of course, someone's got to ask, but how do you pay for it all? Because there's some programmers on staff and there's some people and there's still a mothership server that's collecting data, which by the way, you get a dashboard, we'll talk about in a second. But the way they monetize it is it turns out some businesses go, you know what, you can't have my data. We, and they're offering as a service some cool upsells for monitoring for the dashboard. They have, I think that's under roadmap. The other thing too, if you want a feed only, but you don't want to participate in giving, no problem, they'll sell you a feed. And a lot of large companies go, yeah, we don't let anyone touch our logs. End of story, that's where it ends, which is fine. That people who have that stance on it, you can't have our logs, we only want your data. Well, no problem, they'll sell you a data feed and that's how they're funding it. And it turns out because the, as I mentioned, some of these large commercial companies have licenses that are as much as $25,000 a year. That's a lot for a Homeland user. That's nothing for these large companies that are huge going, oh, you can't have my server logs, but oh, you're only 25 or $5,000 a year. Well, these other guys are $25,000. And you don't have to have but a handful of these companies to see that's as a pretty sustainable business model. It's really interesting, diving into it. I mean, it's something I may take the time to do an interview with some of the people at CrowdSec too and bring them up on my channel. I've been engaging with them, we've been, me and Jay both have, they're very community focused, very looking at all the different things they can plug into. And when I say things like WordPress, they went as far with WordPress. I'm demoing the, well, not really demoing, we just put it in place, the actual WordPress plugin. So they wrote a plugin completely supported within WordPress to not just get your log data, but to go a step further and get within WordPress. And if a bad reputation IP or someone tries to attack my login page of WordPress, it'll actually send them a block page right through WordPress. It's part of the insert in there. This is really nice, because if you're not familiar, WordPress has been under a pretty heavy attack. It's nature of one being a popular product too. Others been a few different add-ons for WordPress that were, well, very vulnerable. PHP everywhere was the recent one. Because boy, it put PHP everywhere and it may even made accessibility everywhere in a way it shouldn't have. So having different tools like this that would help block those things buys you some time, not a guarantee, but helps buys you some time to get these blocks in there. Yep. And I wanna take a few minutes to just kind of round the edges a little bit because there's a few things that I think, we should probably mention that probably goes without saying, but there's always those people out there that are new. So these types of tools, obviously if you have a weak password, right? And someone brute forces your password and they get in like at the third attempt because your password's that easy, there's no security solution on the planet that's gonna flag that as a bad actor because they entered a valid password. So if you have bad hygiene, then obviously no security solution is going to save you and no security solution is 100%. Now, another thing I wanted to mention too, fail to ban for those of you guys that have never used it because I think kind of going over that a bit might help people understand the difference between the two because fail to ban and CrowdSec are compared like all the time because they kind of do something similarly even though they do it different. Fail to ban is something that I consider that everyone that exposes something to the internet they should have at least fail to ban fail to ban or CrowdSec. I'm not really sure I'd recommend that you have both cause I really see a point there but what fail to ban does for example is it looks at attempts. So if you have a login page and your website is exposed and someone just keeps hammering the password you could basically tell it, okay after that seventh attempt because I'm a klutz and I could probably mess up my own password six times but I never mess it up seven times. If you see anyone going beyond that just go ahead and add a firewall rule automatically that just blocks them and you could set it to a permanent ban four hours, 15 minutes or whatever. You don't have to make it permanent because at that point it just becomes so much harder for them to brute force because they can only do so many attempts wait an hour, so many more attempts wait an hour it just lengthens the time that it takes them to get in. Now with fail to ban it's very easy to look at SSH or have it watch SSH but it kind of gets difficult because you have jails which is what they call each config that it can block and those jails you have to make sure that they're right you have to keep an eye on it because for example, what happens if you change your SSH port because you just want it on a different port now is fail to ban watching that port we probably got to go in there and let it know that, hey, by the way I kind of moved this over to different port and if you have Apache you can set up a jail for that there's a lot of them that are pre-configured but if you want to do anything more than that you have to look at regular expressions come up with your own jail maybe I don't know if there's one for next cloud built in yet but in fail to ban but if there isn't you have to basically manually tell it where the log file is, where the verbiage is for errors that you want it to look for so that it knows to block things and what happens if the logging parameters change if the new version of an app changes the verbiage of alerts then it's no longer watching that so you have to kind of just keep an eye on it but it works great though I mean at a minimum fail to ban works for that purpose now CrowdSec also does the same thing if it sees a bunch of attempts it's going to say, hey, wait a minute no, you really shouldn't be here you really shouldn't be doing this but the difference is that it's benefiting from the information that it finds online and or not online but in the database and the feed that contains information from other people that also have CrowdSec installed and then as you're saying it comes to a time where people are generally like, yeah but it's collecting information, it's bad by default which I don't really like that mentality because the way I look at it if a company is honest about what they're capturing then it's fine, if you're fine with it then it's fine the problem is we've seen so many times where companies like Google are saying, yeah we collect A, B, and C but we don't collect X, Y, and Z at all and then someone does like some kind of a packet capture finds out that they actually are doing the thing that they say that they're not doing so I totally understand the lack of trust because the big companies have been bad stewards of personal information especially Facebook but when it comes to open source tools you can see this information and the best example I'll give is Ubuntu because when you installed desktop Ubuntu nowadays and Canonical got a lot of anger about this that it comes up with the, I mean they ask you it doesn't even do this on its own it asks you, would you like to send us information about your computer? I forgot the verbiage and people get upset with that it's collecting data but you can click on it and look at it and see exactly what's going up there and all it is is just your model number, processor GPU, the things that are compatibility related so keeping that in mind because I know some of our audience might be under the impression well it's collecting information as bad not necessarily, yes I know a lot of companies are doing it wrong but CrowdSec just look at the information you know what it's doing it's an open source tool and obviously if it ever has a problem later on down the road we'll have to let you guys know about it so for right now, I mean we've looked at this thing I don't know how many hours you've spent or however many hours I've spent on CrowdSec but it just keeps checking the boxes and going back to my fail to ban versus CrowdSec comparison I feel like CrowdSec is easier to implement obviously fail to ban doesn't send any information to anyone as far as I know I've never seen anything in the config file for it to do that and CrowdSec of course benefits from that information but CrowdSec in my opinion I mean I've never had to learn regular expressions and I still don't know regular expressions so if you're having trouble with that I get it but you know just wanted to go over some of those differences between the two which might help people understand fail to ban or CrowdSec which one's better for them Yes and one thing worth noting is the absolute minimal amount of knowledge you need to set CrowdSec up it is the documentation is not only good they do and I know some people are hesitant about copy pasting a bash installer but they have one if you don't they give you all the details if you like to manually do things but their automation level is great and that's really a testament to the level of engineering that goes into something because the easier something is to use especially when it's a more complex product the more engineering it takes to build this product to be something easy to use and I'm really impressed with how easy it was to set up and from a usability standpoint like you can just copy and paste the bash script in there and in a WordPress instance and I'll cover how to set that up when I do my review of it you just say hey generate a local API key because they did this right they don't just tell you to point at it and just accept any connection it even uses its own local API keys so local things can talk to it by the way they have an entire API interface for people that want to build things on top of it they have a pretty large hub of different features different options of things you can integrate with and this is really interesting how they're doing this and it allows for and this is from the ground up how they wrote this a more modularized experience so people go hey I'd like to put this in my set of devices or the tools that I'm working on and how would I communicate with it? Well it listens by default on port 8080 you can queue up the API key, set it up and build your scenarios and applications there to talk to it so it's a very extensible system. Yep, yeah I think, yeah it's easy to install like you said they have a workaround command if you don't want to do the curl pipe to sudo bash or whatever their equivalent of that is I've seen different flavors of that it's becoming a very popular thing but yeah you could get it installed and I think what's really cool about this is that if you use CrowdSec on your home lab and let's just say your company also uses it if you do IT for a company then it's the same CrowdSec both ways unless your company is paying for the access to the feed without providing the information but it's the same thing like it's very cool when we home lab people get to use a tool that enterprises use the actual tool, right? Cause some of them need, Red Hat needs a support agreement unless you get the developer license so that could be a little challenging but we have the actual CrowdSec it's not like CrowdSec Lite it's like oh let me see if my company can subscribe to CrowdSec Ultra Plus Mega Edition or something like that but it's the same thing so we have at our disposal not something that's going to make us 100% bullet proof I keep saying that because I'm trying to go against all these thumbnails I see on YouTube that says maybe not about CrowdSec make your server hack proof I cringe every time I see that but with the responsible admin CrowdSec just adds a lot of value to your home lab and it's free and in my opinion you may as well benefit from the information Yeah. Now something interesting that they did recently they announced an integration and I have not tested it yet but it's something I'm working on as well with open sense now they started there whatever reason that's just the firewall platform they chose first but that integration is going to be really interesting I'm probably going to start looking at it there and of course open sense is a fork of PF Sense and they're both based on BSD yes they have packages not just for Linux but BSD as well so I'm going to be doing some testing about how CrowdSec could be integrated within PF Sense so it is on my roadmap because I think this is where the next level is going to come is not just integrating it into the servers themselves I mean that's definitely a great place to start and where most of the intelligence data is but in addition to having it at the firewall levels that way if you have different ports open on a firewall they can first be evaluated through CrowdSec lists and go nope these are a series of IPs that just never get to pass traffic through this firewall that way you're now this is not an outbound thing this is an inbound thing so it's not going to really matter to people who don't open any ports so it's specifically more about that I mean it's not that you can't stop talking to those bad IP addresses but generally these bad reputation IP addresses are on that bad reputation for them bringing traffic to you not the other way around. Yep so there's a couple of things too about this setup now that we mentioned well I mean it's an easy setup and it is but you know the ecosystem around CrowdSec is a bit different because like with fail to ban it's failed to ban you have the defaults you could turn on you know whatever you want it to monitor I think on most builds of fail to ban SSH is being monitored by default but with CrowdSec it doesn't block anything by default it when you install it all it's doing is just keeping an eye on things if someone was trying to hack your server oh that IP is trying to hack the server logs it that's it done it doesn't do anything more than that you need a bouncer installed in order for it to take action so CrowdSec itself monitors what's going on and the bouncer is what takes action so if you don't install a bouncer then all you really have is something that's just watching it and logging information you need a bouncer installed and running in order for it to actually take action and they have different bouncers like NF tables IP tables bouncer they have a WordPress bouncer like you were saying that can integrate it into WordPress so you get one of those bouncers or maybe more than one if you have more than one thing and you make sure that they're enabled and then that's what gives you the complete solution so I don't want anyone who it was like me and has ADD is oh it's easy to install okay I installed it done I'm gonna move on and go mow my lawn or something you need to do both things absolutely and that's very important to keep in mind thankfully whenever you install a bouncer and you verify it's working it's done and then you have a whole command line tool that you could use CSCLI that you can use to interrogate things you can get a dashboard like you were saying but should we talk about the port issue that we ran into? Yeah this is worth noting so the weird challenge and this is a me and Jay thing because we're weird well we use open light speed and it's not that open light speed isn't popular it's just niche open light speed is a optimized web server for WordPress so I actually because of here we'll give you a little bit of backstory the reason Tom went to open light speed is because Tom's website and WordPress is not coded extremely well there's some messy things like when you build an automation tool that puts 1400 blog or 1500 blog posts now of all your site and has a lot of repeating code in there your site gets slow on index solution use open light speed or I could find some programmers to update my WordPress site we went with the technical DevOps thing that Tom knows which is we switched open light speed which my website actually quite fast it's a good engine there's nothing wrong with it but crowd sec doesn't natively specifically support open light speed and open light speed has its own quirkiness and because of that you have to make sure you're not conflicting ports and open light speed seems to have a port conflict with the port 8080 that crowd sec default binds to now crowd sec once again all open source is the files are easy to know you just go through Etsy crowd sec and change the port numbers it's not like this is hard to do so to speak it's well documented where you change these ports Jay of course built an automated deploy script and Ansible that as he sets up crowd sec he goes and moves it from port 8080 but it's something you have to be conscious of when you're looking at it but this is nice because the crowd sec people thought about this they have it defaulted there but there's nothing wrong with changing it and once you change it you can use it whatever and bind to whatever other port and you're probably gonna run this there's only gonna be someone with a use case such as open light speed where the default bind ports have to be changed but in case you're running into those issues and by the way, var log crowd sec.log you could look through and the log tells you hey this port's already in use really easy error to troubleshoot just kind of a weird one there so it's one of those little things that you have to be aware of if you have something bound to an existing port and now I think about it I wanna say unify binds to that port as well so if you were to install crowd sec on a server running unify you would run into that's a potential problem you run into is something bound to that port easy enough to change though it's not just change from default Yeah and to build on that a little bit I have run into like quite a few apps that use port 8080 for the local port one example of this is I used to work with Atlassian software a while back I'm not personally a fan of it but that's what the job was for so I just you know it is what it is I'm pretty sure it was JIRA that mapped to 8080 and then what people will generally do is put like a proxy in front of it so that way you know if you're going to the JIRA site for your company or I don't really know many homelab people run this but you would just put colon 8080 at the end I've run into a lot of apps that use 8080 so I think that this is going to be something that a lot of people might run into and like you said it's very easy to fix but just keep this in mind because it's you know it is what it is so this port that we're referring to is for local communication it's not a port that's open up right it does open up to the internet right right so it's how CrowdSec communicates with its bouncer locally so if it's going to tell the bouncer hey bouncer you need to do a thing well I'm pretty sure that's how it does it but it's a local port so if you spin up JIRA for example or anything else that uses 8080 and then you put CrowdSec and a bouncer on it then what you're going to find is that the bouncer won't start because it's on 8080 so what you have to do and this is you know just to save people the trouble is I counted three places to update the port so if you want to change CrowdSec to use a different port which is you know what I do then you have to change it in three places and like you said you could interrogate the log files it'll tell you that this is a problem it's really easy to fix you just change the port number in these three files I believe they're all in Etsy CrowdSec if I'm not mistaken and inside there you just update the port restart the services and then you're fine that's really all you have to do but I just mentioned this so if someone runs into this they'll know exactly what to do Yep, actually we have in the comment section here one of the people who's part of the CrowdSec moderators and one of the cool things they mentioned it is a great way to put it so IDS Intrusion Detection System in IPS is Intrusion Prevention System so sometimes they're the same service but they have a mode of operation and the way they said is you can compare the IDS of CrowdSec's bouncers like in IPS the official CrowdSec team can explain it better is what they said in here but let me expand a little bit on that the CrowdSec IDS Intrusion Detection is the engine itself so CrowdSec's analyzing and having the data but that's just in telling you data it's just not doing any actions the bouncers are the IPS part of this and I think Tom might have froze can someone in the chat room let me know if you see or if you still hear me just a little bit of a lag in the chat see if Tom comes back okay so I would just give Tom a few minutes to see if he pops back in we could always I guess delete this in this section in post when we upload it to the podcast networks how cold is it in Michigan right now Tom's frozen solid I don't even know like I don't even listen to the or watch the weather at all because I kind of feel like predicting predicting the weather in Michigan is just a yeah we can't do that oh I think Tom's coming back now all right yeah there he is sorry for the disruption with the edit that in post so yeah so it was like just before 32 or 31 minutes in or something like that if you want to make a note somewhere around there well you probably see the stop before the audio form yep all right so nonetheless hopefully I'll finish my sentence there so I know where it stopped I think and the crowd sec part is the intrusion detection system the bouncers are the prevention system so the bouncers themselves actually will block the different things that may be found within the engine system and of course the bouncers are also listening to the feeds I think the last thing I mentioned would be the dashboard great Jay yeah I want to talk about the dashboard and there's actually if I remember correctly two dashboards there's one that you can run locally on your mission on the machine and there's one that I don't know is it out of beta yet I don't remember the I think it still says beta but we'll use it it's great I it's a really functional dashboard for being beta how's that yeah yeah it's really cool because it's just really neat because if you have like more than one machine that's externally available you can have one dashboard and then you can see that information right there or if you would rather not do that you can have your local dashboard so you can view it locally per machine yeah the it lets you register each of your instances on there and let you see the alerts it's a nice dashboard they keep adding more features to it remember when I first signed up to it was pretty basic it's gotten a lot more advanced and this is a nice feature for being able to aggregate all that data and it's kind of interesting how this works because the aggregation of that data isn't automatic you have to implicitly enroll each one of your crowdsack agents with a API and it's it's got a nice two they made it really easy to do you copy and paste these things in there to register each of your instances to register them to your dashboard so it's not part of what everyone can see and you can then narrow down to what your specific instance at each of those instances you set up see so they did a shop in a dashboard I think it's a nice value add being able to have that I'm I also like from the completely command line driven because I'm you know as this HMI web server I want to be able to quickly look at something you can type in their command line tool which actually is very functional which allows for very from extended scripting and pulling data out of it so you can look at it run it see that what they refer to as their decision trees for how they did things look at the logs look and see what API calls are being made and interacted with overall it's a pretty robust tool yeah yeah I think it's great to it's an option so it's one of those things you know you don't have to you don't you know obviously you could just run the dashboard if you want it or you could just you know if it's not for you you could just do everything via the command line so it's truly the the Linux way right you could use the GUI or you can not use the GUI and and that really pertains to crowdsack as well you could just be all command line or you can install the GUI if you'd like it it's just really cool to have that available I think yeah but all of our testing I haven't had any problems I haven't had crowdsack break anything it hasn't blocked anything and they have in the part I wanted I'm not going to be able to explain this here but they do have mitigations for a really interesting attack scenario that will make sense to you in a moment like you're thinking oh wait how could someone attack crowdsack well what if a group of nefarious actors wanted to do something nefarious like block someone else's services or things like that what if they went and registered a bunch of crowdsack instances to try to poison the feeds they've actually got and I this is something I want to dive into and I think they have some write-ups on there's well I have a lot of write-ups so I find probably find exactly this one but it was in one of the interviews I listed they've actually really thought through that and have mitigations that stop for threat actors from poisoning the feeds themselves or trying to get things added to a bunch of whitelists and things like that and this is where crowdsack is such a crowd and people oriented service where the more people that are using especially the more good people are using it the better the data becomes the higher the fidelity becomes of the threat intelligence data they gather when we have a lot of people plugging this into their servers and helping to identify traffic and things like that so it's it's one of those things that the product just gets better with the more people that use it is the bottom line and I'm pretty sure I could be wrong on this but I thought I read that you know I'm sure there's better technical terms for this but that there's a BS detector almost like like there's a scoring system when someone what is getting information from a source that's reporting some kind of a bad thing going on like has this IP ever reported anything before is this the very first time was there no you know false positives from this or good information there's a lot of logic in the background to prevent this because I think it's literally the number one thing that I've seen or at least within the top five that people ask about CrowdSec is okay well that's great but what if someone just really hates Google and then just sends like a bunch of BS information about Google and then no one using CrowdSec and or I don't know I was just a made up scenario there but there's a lot of different scenarios and logic that they put in there to try to prevent that kind of thing there's a scoring system there's also the I believe they call it the leaky bucket system where they have to have a certain amount of negative information before it you know leaks over the bucket so to speak so that way like the first person to complain just because they you know want to complain isn't necessarily enough but they're going to keep an eye on it but then there's a reputation system on top of that too that keeps it honest so I think that's a really important thing to have this is great because like I said it's a it's a multi-tiered approach to this I like the fact that it's community participation but one thing I'll mention because this may come up later is well you know actually one thing that did come up someone asked about what about cloudflare web application firewalls or any other in the comments and this is something from the team over there at CrowdSec one of their moderators that commented here that yes you can use it you of course need to be forwarding the original IP through the web application firewall to whatever you're pointing at because obviously you don't want it to just look at CrowdSec's IP address and I'm sorry cloudflare's IP address the second that I didn't surprise no one asked is can you use this with sericata or snort or some of the other inspection tools actually yes and the reason why is they they operate at different levels so if you have other tools that are doing different security for example and with port forwarding that also gets in traffic inspected by something like sericata because you have another firewall in front of it those are doing analysis at a different level so when you look at something how sericata and snort work they're a rules-based system that look for patterns that are known so they download their feeds they're not just bad reputation IP feeds or more specifically like look for this type of attack pattern this type of scenario coming through that's a little different than what CrowdSec's doing so yes they can generally work in concert with each other when properly configured without any overlaps so to speak there may be certain things that they will have a bad reputation list but they're two different products so this is another piece of security in your stack but you don't have to get rid of the other ones they can such as sericata or snort yeah and that that's a good a very important point I mean assuming that the people listening obviously they probably are you know following proper hygiene they have a randomly generated password if a service must be public facing and if there's no need for it to be public facing then it's not and they're installing all the patches they're closing down ports that don't need to be open and just kind of limiting things there and following all of that then CrowdSec gives you another layer on top of that that even though you've done your due diligence in securing your server you know sometimes things happen there's a zero day there's something that is going on that hasn't quite made the news yet then you know maybe CrowdSec is one of those things that will give you that extra layer of protection that'll just eliminate a potential reason to rebuild your server tomorrow you know unless you want to because we have an audience of people that you know I'm bored I'm just going to rebuild it why not I need something to do that but assuming that's not you and you're not rebuilding it for that reason then and you don't want to do that then maybe this will help you not need to do that yeah overall one of the reason we talked about these tools in security as a whole not as an absolute but as a layer everything's one more layer the goal is always when you're building things secure anyone who tells you it's absolutely secure is probably just salesperson walking out the door but in a more realistic stance here security is a series of layers what you want to do is put all these layers in place to make anyone who is trying to attack your systems have to work for every step of the way they get there you know blocking their known IP addresses blocking c2 servers whatever those layers are crops that can be one more in that layer that's why I brought it up as a tool and we know with more and more people and we we know you're going to public have things public facing we know you're going I know Tom said use a vpn but that's inconvenient so you want to have these defense in depth type of situations all these layers in place to keep you as secure is possible so I mean the only real way to be secure is just go back to living on a farm turn off the internet but yeah I don't think that's reasonable I like the internet on so I kind of I kind of do too and I'm also glad that I'm not a sales rep for a security solution company because I'd probably be fired within two weeks and I go to a potential client I tell us how your solution is going to help us is it going to protect us it might should we go with your solution versus another I mean they both might help I don't think that's going to work I don't think I'll last too long yeah yeah this never speaking absolutes when it comes to security and if anyone does like I said there are probably some shady sales rep or they're just painting a target on their shirt you know the Mars solution is the best ever and it's really going to make you bullet proof and the next thing you know like the entire security industry is attacking just that one service just to show them how untrue it is yeah that happens it does all right well thank you for joining us I think we've covered it all and it's really easy to get started with CrowdSec they have plenty of installers for different situations read through their documentation easy enough to find huge in terms of huge in terms of just a great service that we highly recommend and free to use of course if we didn't say it enough at the beginning it's not just open source it's also free free to you users to try it out and play with it so and it's not free for the first five installations like if you have no thousand VMs in your home lab well first of all congratulations second of all if you can install it in all 10,000 and you don't even have to fill out a contact form so I want to make that clear this oh yeah sure you don't need an account no you don't yeah I think that's one of the best things about it because you don't even have to introduce yourself to them you don't have to you know submit your email and phone number then have you know just arrest someone calling you and bothering you at dinner time to buy something you don't even have to talk to them you just install it and just put it in your automation system and just spit it out to every single server you have and that's fine you don't need an account like you said you just have to have it installed and it does its thing you can create an account if you want a dashboard so you know that's the reason to do it but they don't tell you that you need to do it and they don't limit you by how many installs or anything like that I hate contact forms I hate them so much like I can't remember the name of the product but I did a tutorial and a review and I loved the product and I said that but in the same video I also said but shame on them for putting it behind a contact form it's a free video but or I mean a free product but you know we really shouldn't have to you know fill out a contact form for plugins or something like that yeah crowd don't have to talk to anyone at CrowdSec to start using your product that's probably something we should have started with because I like you I like the ease of very like it is you don't have to talk to anyone to sign up for their app dashboard you can that obviously you do at the register and account for the app dashboards you can log in to the website but to set up the instances no formal no paywall as I seen someone just put that's a great way to say it so simple as that so we could obviously if you didn't tell me and Jay are excited about the product because why we have videos on it check out Jay's videos on the on there just head over to LearnLinuxTV look for CrowdSec he's got some there'll be some videos in the future that I'll be doing as well so awesome and there's also an interview with one of the or some people actually I think it was just one person from CrowdSec on the Enterprise Linux Security Show so there's technically three videos on the channel about them so you could get probably more information there although I think we've covered everything you know people need to know to get started it's really that simple so yeah we're trying to make it easy on you all right well thanks everyone for joining us and thank you bearing with us while we temporarily didn't have internet but you're hearing that in the edited version it happened and you listened to it and posted it what was taken out so all right thank you very much take care