 time here from Orange Systems in October is Cyber Security Awareness Month. And I'm also acutely aware that there's going to be a massive amount of information coming out this month talking about cybersecurity. And I think the best way to kick this off is with a topic I've been wanting to cover for a little while. And that is how to know when a security researcher is fully know this is not an April Fool's video, it's more of an October full of fools who will be posting way too much information. Now, there are a lot of wonderful cybersecurity researchers out there. And there are a very small number but much more annoying group of them that are just full of BS. And what I want to do today is me and Jason are going to be breaking down the rules we use because we're both known for calling out some of these vendors. Jason is especially well known for calling out security researchers who, well, just kind of post a lot of BS on there. And I want to offer you and we want to put together some of the rules we have kind of that we go through and see if a security researcher is a good participates in the community wants to help things out and teach you about security or raise some awareness or just someone who's sensationalizing things and creating a lot of noise and wasting everyone's time in the process. But let's dive into this as a topic because I thought this is just an interesting one. We're also going to be posting a series of business related cybersecurity videos. You'll find those over on our business technicalities channel. There's a link right down below to that channel. You can go to Lawrence.video.biz to get there even quicker. All right, let's dive in today's topic. This is exciting. And I've been wanting to do this for a little while because people full of BS and you were just at a security conference and you got the reputation occasionally for throwing rocks at vendors and throwing rocks at bad research that gets poked at. And I've got this as well. And by the way, all these rules we're going to talk about because I want to offer the insight that me and Jason used to like the rules. These are some scientific methodologies and commonalities. So why we would call someone out. And I don't want to call out a person, but I want to make sure you understand the methodology we use and how we think about it. And by the way, always feel free to use these rules against us just in case there's a future where me or Jason become so overwhelmed with ourselves or we plant our heads up her butts and start breaking all these rules. These are these are some pretty solid rules for making determinations of whether or not someone is just full of BS. Yeah. I mean, I have my opinions about a ton of researchers out there, positive or negative, right? But like I feel like people should pass their own judgments. So set of rules that we use is a good one. Yeah. Having rules and define this because I mean, it's easy to point at some people because it's one of those things you know it when you see it. But let's start with the first one. And that's people who lean on their credentials, never their accomplishments. Yeah. This is just, oh, it's like all those things where you greet people and they're just like, why, you know, why should I listen to? It's never like this whole history of all the different things they've done or different cool things they've taken apart, reverse engineered, etc. It's a string of letters or telling them that, oh man, don't you know this or that or I have a million followers. And by the way, I never brag that followers on YouTube is a reason to follow me, because it turns out if you sensationalize you can get a lot of followers. That's not the same as actually being, you may be entertaining, but you really need to be educational if you want to teach people about how to hack something. Yeah. And YouTube, especially, I think there's a certain amount of, you could have more followers by being more entertaining, right? You're taking the long tail of slow growth, which is I'm just going to say the truth and keep doing it. And eventually people will show up and I think that's good. You see people, I mean, I have my stuff after my name and my email signature. There's nothing wrong with that. And I don't on my LinkedIn tag, right? And I've let, honestly, I need to update my email signature, because I've let most of that stuff expire, right? Like it has limited lifetime value. What I'll say there, so KD8BFN, I'm a ham radio operator, and I definitely memorized the question pool to pass that exam, right? Like I understand the theory. I understand all the things behind it, but not to the level that I was going to be working out the answers to some of the questions they ask you during the test. Like I wasn't going to pull out a calculator. I just memorized the answer. They give you the question pool. It's published, right? Like the same thing exists with a lot of these tests. If there's no practical test, right? If it's only a multiple choice test, people come out, they brain down pump it for, you know, 80 bucks online, you can buy a brain dump, which is the answers to the questions. Then you just memorize them and then easy, easy. Now you pass something. Yeah. And like I said, nothing wrong with having the credentials. It's not like that's the disqualifier. It's the people who only lean on that and then never have anything. Because sometimes, like Jason said, some people are just really good at regurgitating things. But it's a combination of all these things that we're going to talk about. The next one is going to be has a history of big coming soon claims about security that get attention, but never come to fruition. Oh, this one. There's like, you would think the boy who cried wolf story, the old tale would apply, but somehow people have a really short term memory. So when someone says, you know, oh no, big security problem coming, but then you can follow that same person realize they do it all the time. And I'm like, how does this person still have 2 million followers on Twitter? I don't get it. There's one particular researcher who I will not call out because we're not doing that here that this is just a repeated theme and over and over again, oh, I got this big, huge thing against this big company that all these people use. And then they releases it. And it's like, that's nothing. You chain together several low vulnerability things and it's still just fishing, right? Like it's not, there's nothing to it. And yeah, there's a certain hype cycle. And I understand that when you're trying to grow and make a name for yourself, maybe some amount of that hype is necessary. I go the other way. I'm an anti hype. Like I, I don't know, I think a lot of the legit researchers have a certain amount of imposter syndrome. So they're not out there, you know, being big and bolsterous, right? And there are a few legitimate ones that for sure are, but they usually follow it up quickly. You know, you could say I have something coming in 30 days because I'm still under embargo. Yeah. And that's a different type of discussion. For sure, things get embargoed. Like the, the CVEs I had against ConnectWise, I sat on those for six weeks until I could publish because, well, actually I sat on them for 10 weeks before I could publish because you end up, it's a negotiation with the company to be able to release your data. And, but I wasn't out there big, huge ConnectWise announcement coming out, you know, in six weeks. No, it's just, it is what it is. And it's a lot more about looking at the pattern of behavior. And that's why I'm, sometimes it's one of the things, if I see someone that says something, I will look and, you know, does this person have a history of saying these things? It's kind of just establishing a baseline. Are they overhyping or is this a pattern they've always overhyped? It's just one of those little things I look at to try and make these little indicators. Maybe we should assign a rank scoring to these, which each one level, but they all kind of have some meaning and the next one's going to be, does not have an evidence-based history of write-ups regarding past findings and awesome gets evasive and defensive if you talk to them. That's a big one. Yeah, there's a skill set for sure where people are very good at changing the subject. And I talked to some of those people I would say last week that, you know, they're talking about something really cool. So I start, I'm a nerd, right? Like, so I start nerding because that's what I do. And so I start asking technical questions about it. And they very quickly divert the conversation back to a high level or away from the topic completely. And it's like, my immediate brain goes to, you don't understand the words you're saying, you're just repeating them. So we can't actually have a more technical conversation about it. Yeah, there's a lot of people, and this is especially in the YouTube world, YouTubers who always talk at that high level, but they really aren't understanding what things occurred to reverse engineer to get that exploit or how the mechanics of it worked. And of course, you know, just coming back, you just got back from Gercon the other day. And that's why we go to these conferences, which I didn't happen to go to this one, but we go there. So I missed, I did miss out. But we go there to talk about the nuts and bolts at the most detailed level, the most of the talks at any hacker conference are going to be at an in depth level. That's one of the reasons they're excited. They jump right into it. And when you start finding people who are evasive and just can't really explain what they did, or, you know, it just becomes that Hollywood hacking mashing on the keyboard and a bunch of commands go by. And it's, they're clearly not understanding it. And the write ups are important. It's a really important as kind of to give all those details. Whenever I cite things, I'm always finding the source. Who did the original exploit? And often they have a really nice chain baking on there. Yeah. And I learned a ton from those write ups. Like it's one of the most useful ways for me to learn new techniques is to review write ups of what people have done in the past, because you come across something and you're like, Oh, I never would have thought of doing that in that way. Right. And so having that detailed history is super, super, super important. You know, you didn't have an original list with citing the sources. I think that's huge because I always cite the sources where I get information, but also you'll find, and this was where I went over one of the hacks that occurred with Fortinet, the people who did the write up of Fortinet, they talked about the other researchers that came before them to find these other vulnerabilities. So they chain them together to create the next vulnerability. I think that's awesome. When you see security researchers, they're showing an engagement in the community about that. So I think that's another, you know, tip of the hat. It's not all about them. And matter of fact, any good security researchers, all about the community. They may be the one presenting it, but often they'll give a shout out. Hey, thanks so and so for finding this exploit that turned out to be the piece I needed to get to that next piece of it. More importantly, like you can always tell when somebody's doing this well and is releasing their write ups well, because there will usually be a bunch of follow up vulnerabilities that are either the same class or in the same product, where somebody was able to take the researchers research plus their own knowledge and push it a little bit further to discover other vulnerabilities either in the same software or similar vulnerabilities and other software. Well, it's the where their smokers fire. If there's a buffer overflow in the VPN of a piece of software, they may patch it, but you'll probably find you can buffer overflow another if they didn't, if one field didn't have proper input sanitization, I bet there's another field that didn't either. And unfortunately, some of these companies fixes are only fixing the field that you pointed out. You probably should look at the rest of the inputs. Yeah, we had that for a long time with ConnectWise, where I submitted mine, they fixed the immediate SQL injection I gave them. And then I told them, okay, I'm going to submit two more tomorrow. So you probably ought to just go to the file and fix them all because there's at least 80 of them in this file, right? So they to their credit, they went through and did a reasonably good job of whacking all of them out of this one particular place where we are just digging them up at will. But yeah, a lot of companies, their ticket says to fix X and they fix X. They don't audit whether or not X is present other places inside of the code base. And I think this is an important next one here is makes claims about having disclosed security issues with large companies, but then says they can't provide them due to NDA. And here you are talking about CVs with ConnectWise, we know there's embargoes, there's time, but over time, this is how the security community gets better. So if you're actually a good security researcher, you negotiate your ability to release the write up as part of it. So my next question to that was what was the payout, right? Because you pay me or I publish or a combination of the two. And if you want me to not publish and you not want me to not highlight the research, it's going to cost you a lot more money than if I'm if I'm able to actually publish, right? So I just use that. Okay, what they pay you, what they pay you to not be able to disclose because there's value to not being able to disclose it cost you money, as far as advancing your credibility in your career to not be able to highlight the work that you've done. Yeah. And the really top security researchers, that's some of the stuff, you know, me, I mentioned like Tavis Ormandy, whose write ups are legendary, George Hots, Geo Hots, you know, these are people that I don't even think they I've never heard of mentioned credentials. Like if you go if you listen to any of their talks or reading their write ups, it all just dives into the technical details. But they follow a very clear disclosure timeline and process by which they do this. Because like you said, this is how us other security researchers, this is how we learn. And that's just good participation in the community. People either they don't have the chops to write it up, or because it's an unprovable thing, they can brag, Oh, I just closed the company, but I can't tell you do the NDA. Yeah, that's not it. You start losing a little bit of credibility there. You know, it gets somebody to help you write it up. And you just check your PC to help you write it up these days, right? Like or if you're a legitimate red teamer, right, like a lot of what you do is report writing and it's writing. So you should have or at least have access to somebody that can help you with those write ups. And they are part of the game there. I know a couple that I think are legit that they just don't write up, they do the background bug bounties, they take more money to get stuff to get vulnerabilities. But I mean, they're iOS hackers, right? And I know from the Pokemon Go community. Well, and there's also security researchers that they're probably the quiet people. There's plenty of quiet security researchers. It's the person that wants to get up on stage. This is the security short talk about that started with all this noise they created online. Then you got to determine is it news or noise and kind of whittle it down. Those are more than what it's all about. There's certainly some brilliant people that you meet that are the quiet ones that are just out there poking away at things, but they're not, they're not, you never call knocks, they're also not bragging on LinkedIn. They're not out there. Yeah, correct. You won't unless you find them, you won't come across them and none of the supplies to them because they're not doing any of these things. Yeah, they're doing none of these things that make you, because this all starts with that whole, how do you know this is even a good publication? That's what all these rules really come down to. These are how me and Jason look at these people and go, is this good? Is it bad? Or is it a whole lot of BS? And I think the last one is claims to know people at cybersecurity companies and participated in their internal meetings, but never offers anything verifiable or even the names of said acquaintances. This is pretty specific because of one security reason. Well, I don't want to call them. I don't know what to call that person. An annoyance. But there, but it's not like this person is the only person doing this, but it is an annoyance that this person, oh no, I talked about it. And they started by calling me out. And this is, oh, and it's been more than one of them. It just occurred to me who you're talking about. Yeah. He certainly claimed to have, as much as I'll tell you, it's a he. So he claimed to have quite a bit of information. I had deep conversations. In the corporate meetings, he claimed to participate in his said they'd never happened. Yeah, I don't think these meetings really happened. It's posturing. There were contacts. I mean, but like, yeah, I'm out there, you don't necessarily brag about it because bragging about it's a good way to lose those contacts. Yeah. It's one of those things. Like if, if I have, you know, and this frequently happens, and I will ask someone, Hey, can I quote you at all? And they're like, I didn't tell you this. And I said, that's fine. That means I never mentioned you told me that's off the record. Yeah, off the record. That's perfectly fine. That's just integrity when you're presenting something. I have talked about a lot of details, but there's details when I can't talk about something. I just stop and just say nothing. I never go. I could tell you, but you know, I can't reveal my sources. That is when people start doing that, you start going, are you really a serious researcher? What, what, how did that help the conversation move forward? How did it help the security community? And it doesn't, but it's kind of just a, it's got to be just a narcissistic thing. Those, some of those people do. Yeah. I mean, there's certainly a class of people that this is their 15 minutes of fame, and they're trying to milk it for everything that it is. I have one more to add. Yeah. People that do the same thing over and over with regards to class of exploit, because I got a couple that I come across that like literally, Oh, I've got, you know, 25 CVEs to my name. They're all the same CVE. You just went from the same bug in the same library that 30 different vendors are using and then filed 30 CVEs against it. That's not, I mean, yes, that's research, but it's not. Yeah. Yeah. It's, it's all those things where they're overstating it because if someone say, Hey, I'm great at finding these bugs, but then it's one class of bugs, you know, the SSL vulnerability. Well, guess what? SSL is an everything or the more recent one we just talked about the WebP vulnerability. So if you went and filed a CVE everywhere you find WebP, did you find anything new? You just went through all the things you knew that use WebP. It reminds me of the, what do I, they have a name, the basically bug bounty scammers, essentially that, you know, I get an email a quarter of you then blah, blah, blah on your WebP or you don't have a DMARC record. I'm like, well, first of all, I do some bug off, but second, and then they say, well, I expect to be paid for this finding. It's like, well, one, you're wrong. I definitely have a DMARC record and two, like, no, that's not, no work went into this. You're in a scanner. Clearly you got it wrong. And then you're just emailing everyone without a DMARC record or you're emailing everyone using a particular version of a WordPress plugin. At some point or another, like bulk scanning and then emailing stuff in or bulk finding those vulnerabilities and then reporting them. There's not, I don't really know what the benefit is to having a CVE every single one of those products that uses WebP. We'll pick on WebP, right? Like I don't know if it's worth it, right? It's, every one of those companies should release a new version of it and they should call out in their own internal security advisories that they have a security thing because they use this library and they updated it. But I don't know that every one of those rises to the level of requiring a CVE. Right. And I, I, some of them are doing it because I know one of the more recent ones that we blocked them from filling out the contact form anymore. They keep wanting to date date for the awesome price of only $300. They will fix the vulnerability problem. So they, they had an ask and it kind of goes down to sometimes there is with some security researchers, look at the motives they have. Are they engaged with the community and sharing and moving security forward? Or do they want those deals? Because honestly, a, you can look this up. It's a good security researchers are paid quite well to speak. They might get 10, 15, $20,000 a speaking gig. And it's kind of fun to be up on stage. There's no doubt. But if they're, there's a reason they may do all these things that we mentioned to kind of bolster themselves up to a reputation. And then someone always says fake it till you make it. It's a great deal. No, it's not. Please don't fake it till you make it in security to stop. No, not with security. Not with security. Not with tech in general. There's enough bad things we have to clean up already in tech. We don't need more of it. So that's kind of in a nutshell. This is some of the things that me and Jason will look at when we see something both that we have verifying whether or not this is real, we'll go through these right here and we'll come to the conclusion. And now you can too. And if, like I said, feel free to always call out me or Jason who do take the time to publish research and things like that, mostly in video form. But call us out if we see us do any of these things here. And if you see somebody that's violating these rules, feel free to tag us into it too. Cause at least I'm not afraid to call a space bait. Yeah. You'll find, you'll find that me and Jason have no problem calling people out because this is the thing. So yeah. All right. I will leave those in written form down below as well. And thanks.