 Good afternoon. My name is David Horvath and I'm a Product Manager at Hacker One. Today I'm very excited to talk to you about Hacker Powered Data, specifically why the most common vulnerabilities aren't what you think they are. A bit about myself, I've generally worked in the security industry for a bit of time. I worked at Okta which is an identity and access management space for four years and I was the first Product Manager hired on at Hacker One. I'm really excited about the work that we're doing at Hacker One. One of the things that gets me most excited as we're focused on real and not theoretical security risk. So one thing that I strongly believe is that data will transform security operations even though it seems like that statement might be often overhyped. And what I mean by this is not that we necessarily need to use AI and machine learning for everything in security, but data can really help us in certain areas. It can help with better anomaly detection, help us make improvements to our scanners and help us make important decisions like whether to patch vulnerability and whether not to. And today I'm going to talk about a somewhat ancient technique involving data and that is counting. Counting has been around for 40,000 years and at Hacker One we've done our fair share of counting on the bug bounty programs that run on our platform. And we think that the day that we have is interesting or can be interesting to everybody and can help us all make better decisions about risk trade-offs. Before I get into those details, I'd love to learn a little bit more about everybody in the audience. So I'm going to ask a couple of questions and if the statement applies to you, please raise your hand. First question, does your company or project have a channel for vulnerabilities to be reported? Cool, that's pretty good. How about a process for dealing with them? Okay, maybe some fewer hands. How many of you know what a bug bounty program is? Everybody, most people. How many of you have one? Somewhat less. And the most exciting question, how many of you have ever participated as a hacker? Okay, we got a few. Cool. Thanks for sharing. So it seems like most of you are familiar with bug bounty in general, and some of you have probably heard of Hacker One before, but here's a brief overview for anyone that doesn't know. At Hacker One, we work with a community of over 500,000 ethical hackers. Security teams can create either public or private programs, which hackers submit real vulnerabilities to. And most of those programs will pay out a bounty to reward and incentivize the hacking on the platform, and those bounties can range anywhere from $50 all the way up to $100,000, so there's quite a large range. And the Node.js community actually runs two programs in Hacker One. This is the program for third-party modules, and there's another program, a core program, for security vulnerabilities on the Node runtime. Both these programs have pretty good activity from our hacker community, and they're helping to keep this community safe and strong. So over time, we found that companies working with hackers is less bleeding edge than it may be used to be. As you can see from the Google search auto-complete results for the phrase, should hackers be? Two-thirds of the responses are related to hackers being hired by companies or them being protected, and only one-third is related to them actually being punished by companies. This ratio was probably flipped even a couple of years ago, and this is really exciting to us at Hacker One, because it means that it's becoming best practice in most industries to listen to hackers and to work directly with white-hat hackers. So Hacker One is in a pretty interesting position. We have lots of cool data. We have 1,700 programs that are running on our platform. Those programs have found over 140,000 valid vulnerabilities and have paid over $75 million in bounties to hackers over the last seven years that our company has been in existence. And I think this is one of the most comprehensive data sets about what types of vulnerabilities hackers actually find. And when a hacker finds an issue, it represents real risk. This is something that a criminal hacker could find and actually exploit today. On the vulnerabilities, we also have some interesting metadata, like what is the skill and performance level of the particular hacker that found it, what severity did the team assess it with, what type of asset was it on, and what type of industry is the program in. So at Hacker One, our mission is to empower the world to build a safer internet, and we take this mission extremely seriously, which is why earlier this year we released the Hacker One Top 10 Volns report. And this is a list of the top weakness types that are found by our hackers and are actually paid out bounties by programs. You can access this full report today on our website, but I'm going to go through a lot of the highlights and the rest of my talk. So here is the Hacker One Top 10. Of note is that companies pay out for cross-side scripting far more than any other vulnerability type. It accounts for over 30% or about 30% of the total bounties that are paid out platform-wide. Some companies we know are not as interested in cross-side scripting because it doesn't always expose user data and mass like other vulnerability types do, but from our data it seems like it's here to stay at the top of the pack. The second most common is improper authentication, followed by information disclosure, then we have privilege escalation, and then SQL injection and code injection. The next one is number 8 spot SSRF. We think that that's going to grow in the coming years as more and more companies move to the cloud. So in the rest of the talk I'm going to dig a little bit deeper into what's behind our top 10 and what implications that has for all of you. So let's start by zooming into cross-side scripting or XSS. As you can see here, not all cross-side scripting volumes are created equal. There are a couple different types. Stored XSS is either a higher critical severity over 33% of the time, whereas generic cross-side scripting is only higher critical about 6% of the time. And accordingly the bounties paid, the average bounty paid to those is pretty different. Stored pays out $481 on average while generic only pays out $288. Stored is harder to find, which means it only accounts for a smaller portion of the total cross-side scripting volumes found. I think it's about like 18% right now. So why does this comparison between the different types of XSS matter? I think it matters because if we look at how often a vulnerability type is found, we can infer how easy it is for a hacker to discover it. And if we look at how highly a vulnerability type is actually paid out, that can imply how valuable a company sees that as being the tangible amount of business impact to an organization. And if we think that risk equals discoverability times impact, then we have some real-world bug data that can help us as we're making risk trade-offs. So in a world where security teams are low bandwidth and also embrace risk, we think this data can help prioritize whether companies need to patch a bone or they can leave it or at a very minimum what needs to be patched immediately and what might be able to wait a couple of days. So another interesting thing that our platform tracks are hacker skills and performance over time. This is Pete Yorsky. He's one of the top hackers on our platform. You can see he's got really good stats. He's a 93rd percentile signal hacker, which means he's in our top 10%. And I was interested to look at whether there's a difference in the types of vulnerabilities that our top hackers like Pete find and whether there's between sort of less skilled newer hackers. Is there a difference? And the answer is yes, there is. There are some differences. So this shows the top bounties awarded split up by our most skilled and least skilled hackers. You'll notice that for both of these groups, cross-site scripting pays out more bounties than any other vulnerability type. I thought that was pretty interesting and surprising. The other thing to note here is that our most skilled hackers tend to focus on pretty specialized and difficult to find vulnerabilities like SSRF, code injection, and IDOR. These typically pay out a lot higher bounties but are more difficult to find, which makes a lot of sense. So another thing that our report looked at were differences by industry. And you can see that there are some differences. For example, aviation and aerospace as a percentage sees three times more sequel injections than some of the other industries shown. But the general trend here is that we see fewer and fewer differences over time between these industries. Over time, aviation and aerospace and probably like travel and hospitality look more and more like the profile for computer software. And we believe this is because the world we're living in now is effectively that most companies are software companies or at least we have core software parts of our products and offerings that make us vulnerable in very similar ways. So I also looked at some trends too. What is the growth or decline in weakness types over time? And we started tracking this in 2017. Before that, our data was not as great. So on the downtrend side are vulnerability types like violation of secure design and CSRF. But probably more interestingly is that our top two vulnerability types cross-site scripting and information disclosure, those have still been on the rise the last couple of years. But the highest growth volume types are business logic errors, IDOR, SSRF, and code injection. And what these all have in common is either they're associated with companies moving to the cloud and there are also things that humans are really good at finding and scanners are not good at finding. So I've talked for a bit of time and you might have noticed that I haven't mentioned the most famous and popular weakness framework, which is the OWASP top 10. I assume most of you have heard about the OWASP top 10 and are maybe fairly familiar with it. And right now it's sort of the security community's only option. And many people sort of for better or for worse consider it the list of weaknesses. But there are like a lot of reasons for a list of things. And if you'll humor me for a couple of minutes, I'd like to talk about music. So there's a very big difference between an editorial list of the most popular or the best musical artists of the year and the most streamed artists on Spotify. For example, at the bottom left you have Kendrick Lamar and Kendrick Lamar received eight Grammy nominations in 2019, more than any other artist, but he was nowhere near the top of the Spotify streaming charts. On the other end of the spectrum, Ariana Grande was the most streamed female artist in 2019 on Spotify, but sadly to some myself included, she only received two Grammy nominations, both of which I don't think she won for. But you'll see a similarity between these two lists and that is Canadian and former Degrassi star Drake. So you could say that Drake is kind of like the cross-site scripting of music. He's both highly acclaimed and highly popular, just like cross-site scripting is popular among the most skilled and least skilled hackers on our platform. I guess what I'm trying to say with this comparison attempt or the comparison is that the OS Top 10 is an editorial list and there are lots of good reasons sometimes for an editorial list, but the Hacker One Top 10 is based off of what we actually see in the real world, which makes it a little bit more akin to the top of the Spotify streaming charts. So here's a comparison between our two lists. You will see a lot of similarities. One major difference is the relative position of cross-site scripting. So at the top of our list, kind of middle of the pack on the OS side. So I'm going to dig into the data a little bit on our end, but as an ethical data minded product manager, I wouldn't be doing my job if I didn't explain that there are some limitations in the data that we do have. The first is that there's selection bias on behalf of the programs and hackers that we looked at. These are all companies or organizations that have programs on Hacker One, which mean that they're probably a little bit more security conscious than your average organization. The other assumption that we're making is that the types of vulnerabilities that white hat hackers find are the same or similar to the types of vulnerabilities that black hat hackers find. We think that this is probably a reasonable assumption to make, but wanted to call it out. So with those two caveats, I'll get into some more comparisons of the data between our list and the OS top 10. So I think this is one of the most interesting charts in the deck. And it shows the market share for vulnerabilities found or the discoverability of them as classified by the OS top 10. And what you can see here is it shows that of all of our bounties that were paid out on the platform, the OS top 10 has a coverage rate about 50%. That's pretty good when you think about it one way, but thinking about it in another way of 50% covered means that you're 50% uncovered. If you only care about the OS top 10, that's maybe not so awesome. On the other hand, the hacker one top 10 matches 90% of the vulnerabilities that are found on our platform. So this chart is the same idea, but instead we're looking at the market share of bounties paid, not just vulnerabilities found on the platform. And here you can see that the OS top 10 coverage is even higher and it's at about 30%. For the record, personally, I thought this was surprisingly high coverage. It was great. But again, 70% covered means 30% uncovered. So how great really is that? A couple of other things to note on this graph that are interesting are even though cross-site scripting is on the rise overall, its market share of bounties paid is declining over time and compared to all the other options. And then you'll see a little green blip in 2018, and that is XXE, and I will get into that vulnerability type in just a couple of minutes. So another question I had was what do programs pay out a lot for that's not on the OS top 10? Those vulnerability types are CSRF, SSRF, Open Redirect, and IDOR, and conversely, what is in the OS top 10 that is much lower down on our list? And that's XXE and security misconfiguration. So I wanted to understand a little bit more about the disparity specifically with XXE, so I will get into that. XXE stands for External Entity Processing, and it's been on the rise over the last couple of years. It's that purple line at the very bottom of the graph. It's been doubling over a year since 2016, and it's gone from pretty much nothing to a little bit of something showing up on the graph. It's not an often found vulnerability, but when it is, it's almost always a higher critical vulnerability based off of impact, and it's our highest paid bounty type on Hacker One, coming out to 1600 US dollars a pop. A hacker has to have a lot of skill to find an XXE vulnerability. Typically when they do, they can find it in a number of places, which is why we see a lot of our top most skilled hackers specializing in this type of vulnerability. But it's this lack of discoverability, which probably accounts for the reason why it shows up on the OAS Top 10, but doesn't show up on the Hacker One Top 10. And what this difference really underscores is that not all vulnerabilities are created equal. We typically look at things like the technical exploitability and the severity and the impact of a vulnerability, but we also need to consider its hacker discoverability. Just because a volume is technically exploitable does not necessarily mean that it will actually be exploited by a criminal human hacker. And what this means is that what scanners find is very different than when hackers find human criminals use hackers or use scanners too. So we're not saying that they're useless. They definitely have their time and place. But more generally, what's the point of sharing all this information about the data we have? Why does it matter to you? And I think it matters because data can lead to better conversations with development teams. We run a survey of our customers every year and 90% of our customers say one of the most important things for them to be successful is for them to have a better relationship with their development teams. We find that even if security teams know what to patch, they can't necessarily always get the buy-in for the development teams to work on it. Typically, the security development team conversations go something like this. They say, hey, you've got to patch this now. It's important, and the development team comes back with. Well, you say that every time. How does this fit in with our current priorities? And what we really want to do with the data that we have is to enable some mutual understanding between these teams. So the conversation goes something more like this. The security team says, hey, this vulnerability is highly exploitable by a hacker of low skill. And when exploited, it's often critical and impactful. And the development team says something like, hey, I understand. We'll fix it today. So we hope that with this better understanding comes ownership and with ownership comes patching, if not everywhere, at least towards most needed. Thanks for listening. That's all I had. I have a couple of minutes for questions if folks have any. Thanks. Any questions? The folks that are hackers, do you have anything to... Oh, cool. Well, when you're done with your course, check this out. Yeah. Yeah. For companies that use our product, they can track, or even you, you can track what types of vulnerabilities you're receiving are the most common. We have some details around what those are typically found by. So we expose some of that in the product to customers to help them make better decisions.