 Bon morning, thanks for coming. In this talk I'm going to talk and hope we all talk at the end with questions about SASTI. Doing effective SASTI, Secure Code Analysis CICD. This is what we are going to see. Not going in a lot of detail here because we will see during the presentation, but just a high level view. Juan Main, I'm Florencio, this is my e-mail, Twitter account, I work at Red Hat, I'm a Secure Architecture Team Lead, I work with the Product Security Architects that work with all product services in Red Hat to try to help with the security, I'm a Principal Product Security Architect. I'm in Information Security Science 1999, I saw a funny Twitter thread, I think yesterday or the other day, telling okay, tell me your age with a comment, so this is my age, adding plus plus to this file, so the people who know will know. Some background, since 1999 I have done a lot of different things in security, always in cyber security, pentesting, had my own startup for some years, been pentesting forensics expert witness, I attended like 10 trials defending my forensic analysis, IT, not opening course. I was also an auditor, third party auditor, ISO 27001, GDPR, with DAI, a size of a big company in Spain, supermarkets company, and now proud security architect at Red Hat. Just a bit of context, well now, what is SASTI, this is an acronym for static application security testing, when we talk about static, we talk about analyzing an application without running it, so usually, very frequently we talk about analyzing the source code, I'm going to focus on analyzing the source code, but we can also talk about static analysis when we analyze the binary without running the application. Infact, there are some mechanisms, strategies to analyze the binary code, statically for sure, but here we are going to talk about testing the source code and testing it to find security vulnerabilities or potential security vulnerabilities, we want to analyze the source code of an application to find security vulnerabilities. This is one of the core practices in the security development framework Red Hat is working on, you have more information there and there, and I'll share the slides after the talk. Benefits of doing SASTI, why communities, why companies, why orgs, would want to do SASTI, because at the end any security activity we do needs time, right? Needs time, needs effort, needs resources, so why a community would want to do SASTI. It allows to identify vulnerabilities early in the development process, it points to specific lines of the code where the vulnerability is, versus other kind of security analysis, dynamic application security testing for example or a pentest, maybe we know that there is a vulnerability, we see the results of sending some pilot and see the error or the behavior of the application, with SASTI we know the specific line where the vulnerability is present, it is developer friendly if it is implemented correctly and this is a lot of the reason of this talk, right? SASTI can be implemented in many different ways and there are some that are very, let's say, developer friendly and some ways of implementing SASTI are not. It has a low entry barrier, right? It's not expensive, there are open source tools and free open source tools that we can use to start, at least start doing SASTI. It has a high coverage, SASTI is usually done with a lot of automation, there is manual work for sure to analyze the results and act upon them but the scan itself can be done automatically against thousands of lines of code covering 100% of the code, right? So no other kind of testing does not have that coverage in the same way with that level of effort. Also it's easy to extend, not only to extend to adapt to the company. When doing SASTI we have the possibility and we will see of customizing the rules to the way the company develops so that way we have better results. Also in general this is true for any activity that improves the source code of an application. We are not talking only about confidentiality, right? So why does my front end need to do SASTI or why this application needs to have better security. This is not going to be used by the military or the NASA or whatever but more security is also more quality, is also more resiliency, right? We cannot have resiliency, we cannot have a good quality software if we don't have security. When we improve the security of a code base we improve the reliability of the source code. So a lot of benefits if implemented correctly. And now this is the other side so if this is so good why not everyone is doing SASTI, right? It's easy, 100% coverage, we no vulnerabilities so we have solved the security problem with SASTI. No, why because we have many problems when we implement SASTI. In general too many false positives. We run a SASTI application and we have 100 findings that have to be reviewed and many, many are false positives. It's not a security issue, the tool does not understand what we are doing, we have methods, functions that are cleaning the data. So the SASTI tool thinks here there is a SQL injection but there is not because previously there was a function that we have coded that cleans the data. Too many out of context detections, a bit different in the sense that in this case it's true that's a pattern that can be dangerous. We are using MD5, right? 50 findings of MD5 being used in different places of the application. Is that a security issue? Well it depends. If we are doing the hash of the password and storing it, it does not sound like a good security practice, right? But if we are using it maybe because we are implementing a hash table ourselves and we need this identifier that is unique, it's okay, right? We can assume we don't need a better hash algorithm for that. So we may have many findings that are not security issues just because the application, the SASTI tool says there is a vulnerability there. Because many times we implement SASTI without integrating it, integrating the tool in the developer's workflow. If the developer needs to go to another application, to another dashboard to see the results or needs to do a great effort to analyze the results, maybe it depends on the company, right? But probably they will work on that the first weeks but slowly that friction in the process will do that the developers stop using the tool. Okay, yes, we are doing SASTI because we have these results there but who is reviewing the results and acting upon them. That happens a lot. Or maybe we are using the wrong SASTI tools. We cannot customize rules, they are a black box for us, that's a problem to optimize the process. Rules are difficult to understand, they are there but we cannot understand them. It's difficult to create, modify, remove the rules of the SASTI tool or require compiling the source code before we can start doing SASTI. This is not really bad. There are great, great SASTI tools that require that the code compiles. Usually they require that the code compiles because they have a way of working that reduce the number of false positives which is really fantastic. So it's a trade off here. I prefer to start doing SASTI since the first line of code or when the code is incomplete but there is a legit approach that is using this other kind of tools. But that can be a problem for the company if the workflow does not understand that the code needs to be compiled. I'm going to talk about a tool just because I'll explain the characteristics but any tool with these characteristics would work. It's not the tool, the important thing here, what I want to transmit is the process is what is important. But why for example this kind of tool? It's open source, we have the opportunity well. You know the benefits of that, it's fast when scanning, can be run since the first line of code. It's easy to write the rules, modify, we can have even our own rules set from scratch. But they provide many open source rules already which once we can start. And can be integrated very easily in I guess any kind of CI CD pipelines. Trivia also to install and to execute. This is just an example of an execution. This is the rule set we are using, ANOVA's top ten rule set just for the example. It is provided by the community and there is also a company behind this tool. It's about security because what this tool does is analyze code and it also has other kind of rules like LinkedIn rules, quality best practices. I'm going to focus on the security rules and that's why I'm using this rule set. It is, we are indicating that we want SARIF results. SARIF is a kind of formatization, right, but with some specific attributes focused on security. And it's becoming a standard, if we have our results in SARIF format it's quite probable we are going to be able to upload directly to ingest these rules. It's this file in a different tool, for example DefectDoyo to manage findings, to manage defects. It can be ingest by DefectDoyo and all the data is parsed and put in the correct places. So it's as easy as cloning a repo and executing installing and executing this command against this repo. This is the first thing we have in the output. So here it is indicating the kind of files I'm using for this example PyGoat. It's an open source vulnerable application written in Python, so it is on proposed vulnerable. So I wanted to have findings for sure and I use that as an example. And here Semered is saying, ok, I have 140 rules for Python, 65 rules for JavaScript. So this is for the rest the same. On this number of files, so I have 100 files in HTML, 53 files in Python. So at the end, 597 tasks. I think this was running like one minute, two minutes. Not a very big project for sure. And it found 23 findings. So this is a section of a sari file of the output. This is the kind of results that we get with a sasti tool. We get a hash. This hash identifies the finding and can be used for managing false positives. If we have this, for example this hash, we put it on a file and we use it like a post process. Just remove these findings from the sari file the next time because we already know they are false positives. That's a way of managing false positives. We have that information in one. In number two, we have the place in which folder and file this vulnerability. This is a section for one finding. One potential vulnerability. In this case, pgoat, introduction, APIs. Then we have the exact place in the code. A starting line, end line, from 59 to 65. This is the code. It is not well parsed, let's say, so we can see the code directly in the source code. But we know the exact place. And then we have the text of the vulnerability. Found the user control request. So a request data that comes from a user and it is directly used in a write function. Sounds like a security issue. Something that comes from the user is dirty and it is used without cleaning. That's the kind of vulnerability this finding is about. And we have an ID of the rule. Python, Django, security injection, blah, blah, that identifies the specific vulnerability that has been found. This is one of the 23 in this execution. As it is JSON, we can process the file quite easily and work with it. And something maybe you miss is the severity, the tool things or the tool assigns to the finding. In this case is an error. Many rule sets in the same graph has a severity of error warning and informational. Not that good in my personal opinion. I would prefer critical importance, something like that. But that's the rule set. If we don't like it, we can do a different one. So we have executed a SAS tool. Are we doing SAS T? No, we are not doing SAS T. Just running a tool against a source code, a code base is not doing SAS T. Just by running a tool, we are not improving the security posture of the application. So we need to do something else. SAS T is a process to implement SAS T for it to work. It sounds like more effort. But if we don't establish the process, the workflow, we won't get the results. And if we don't get the results, we will stop doing this very fast. So how an ideal SAS T process would be? We need to do these scans automatically. If we want to put the results as close to the developers as possible and is what we want, we would do SAS T for each merge request. Maybe for a merge request we don't have a lot of context of how the application works, but for sure the findings are close to the developers and we will have more probabilities than the people are acting upon them. But we also can do, for example, a whole code base scans and schedule intervals. It's another approach that maybe works as good as putting the scans in merge requests. Then we need to analyze the results. There is no way we can avoid this step. We need to understand what is the finding and we need to classify the findings depending on the severity. Because it's not the same to have critical issues like low impact results. We need to track the results and this is also a key thing to do if we want a successful SAS program and not a failing SAS program. We have to put the results as close to the developers as possible. And this is where we need to adapt to the company, to the org, to the community where we want to use SAS T. Is the community using GitLab issues to manage the findings, to improve the code. Then there is where we should put the results of SAS T. Are they using Jira? Are they using Baxila? Any other kind of system? We need to put the results where the developers work. That's essential if we want that this works. And we need a final step that is improving the process. We need to remove the false positives and modify the rules. These two parts are essential if we want a successful SAS T program. If we don't have an easy and agile workflow to identify false positives. Maybe it's okay to have 100 findings the first time we run SAS T. And 90 of them being false positives. Maybe it's fine for the first scan, but not for the second. For the second scan we need to have classified these 90 findings as false positives so developers don't get these same findings again. And they need to review these 100 findings once. If we try to do that, that process with time will stop. Also we need to adapt the rules. Sometimes we cannot just say this is a false positive. Identifying false positives in the code is sometimes complex. How do you identify a finding? We have a hash, it's true. But that hash is probably related to the place, the lines in the code, the structure. And maybe we have a line and the hash changes. So it's difficult to always say this is a false positive. So the other option we have is to remove the rule or change the rule. So for example maybe in our project it does not make sense to have this rule that identifies MD5 as a vulnerability. Or maybe we want to reduce the severity of that one to low because of our context. We need to be able to modify and work on the rules. That's also very important. So now I'm going to show briefly some examples of github and github are already providing to communities the possibility of doing sasti quite easily. It's just enabling these options and knowing how they work and start working with them. So try them on your own projects. It's my suggestion, let's say. In github there is an option, just set up code scanning in one. It is a workflow and at the end in github it means a github action. They even have an existing github action that uses semgrip. This github action that we are enabling and uses semgrip behind we can just also customize it if we need more functionality because at the end github has some different licenses and it offers more things in one license and less in others. But we can use our own github action with our worker that uses semgrip or other static analysis tool and that's whatever we need to adapt to the process of the community. That is what is really important. In github very similar they have included already a static application security testing just when you create a new project and you have the results automatically for each merge request. You have a sari file and get that by default again uses semgrip. Well not in github it was not by default it was if you choose it but in github uses semgrip in this case. And we can also write our own integrations. For example if we have a community that works in Jira or an org that works in Jira it's trivial to parse the sari results and using the Jira API just create an epic with all the findings extracted from the sari file. In which file, a summary of the vulnerability and which line and the content of each one more information. So the important thing here is to adapt to the way of working and put the results as close to the developers as possible. Managing false positives it depends a lot on the tool that you finally use. Semgrip has some options ignoring files and folders for example if there are files and folders that we don't want to scan because for example are coders that come from a dependency for example and we don't want to analyze them or examples or testing code. We can ignore those folders or we can do something custom to remove the false positives from the sari files. This is just syntax from semgrip. And listing having the false positives identified with the hash is something that the semgrip open source option does not include but we can do something custom with the code. About custom rules that is very important and a strong point in this kind of tools in this case they have some commercial rules but they have some open source rules with LGPL in a repo. We can fork, mirror and work on that to remove ad whatever we need to adapt to our case. This is the syntax of the semgrip rules as you see. Really easy, they are YAML files with an ID, the languages this rule applies, this helps for the scan to be faster because semgrip only applies the rules to the language. They are full, we can have also generic rules but in general it's better to have rules adapted the message and the pattern is the most important part of the rule. Here is like a trivial rule, it says it's not just a grep, although it has grep in the name, it has parsing functionality. It says here some, this is a placeholder for any vulnerability, a constant token is, so the word is and then this means any valid statement in python. So if we have a variable is and any valid statement, this rule triggers a finding. This is a more advanced rule and the key why we need or the kind of rule that shows how we can adapt this to our custom code. Here we identify patterns that are sources, so a source means where we get information from the user so the result of that is dangerous. So if we have this, monitor the result of this, then we have the sync, if we get some data here and we use it directly here in one of those patterns, we have a vulnerability. So if we get some data with this and use it here, we have a vulnerability. But if the result of this pass through this function, it's ok, that's not a vulnerability. We have sanitizing functions, cleaning functions, so this has this functionality, this is called painted analysis, that if we use our own framework, our own cleaning functions, we can customize our rules to reduce the number of false positives. Let's see it with an example, Xembre has a playground, this is the rule, this is an example code and another example code. In this case we get some data, this data is dangerous but it is clean here, so when it is used here there is no finding. In this case this is highlighted because data is get here and used here without sanitizing, we have a highlight, this is a finding. This is the way we can create our own rules and reduce the number of false positives. So this is the last slide, then we can do some questions, so at the end we have bad safety and good safety. Bad safety when results are not processed, good when they are processed and we act upon them, so we really reduce, we really improve the security posture of our application. Bad when results are far from developers storing a different application, good when they are close to the developers. Bad when there is no clear way of managing false positives, good if we give that possibility to programmers. Rules are hard coded or difficult to change, that's bad, good if we can customize. Bad when it is slow, so 2 days or 12 hours to stop a merge request, we cannot do that. We cannot stop a merge request for being available hours, we are talking about minutes if we want to do this in merge request. Good safety when it is fast, few, few minutes. And in general bad safety when we think safety is deploying at all and having results somewhere. And good safety is when we have results that we use to improve the security of our application. Much, much more better, less results that we use to improve the application that many results that we don't use for anything. And thank you, that's it. Yes, so the question, repeat the question, yes, and also to see if I understood it. The question is if we can use it for other languages or other use cases. Yes, totally, for infrastructure as code, as terrafor, there are already existing rule sets. It has already rule sets and for sure we can use our own custom rules too. Yes, I would say it's semantic but not official answer. Yes, we can establish our own severity for the rules. And yes, that should be, that is not only about the tool but the process that is something that GitLab should allow us to do. And in fact they do, again, depend on the licensing but it's exactly how I think it should be done. If it is critical or maybe important or critical maybe block the merge request and if not let it go. There is another approach that is very interesting and some orcs do that is having two sets of rules, the mature and immature ones. So the mature ones are they directly used by the developers and the mature ones are only used by few security people. So they don't generate the mature, generate many false positives. So we have, there is a process to promote rules from the mature set to the mature set. So when they stop generating many false positives because they are tuned, they are moved to the mature set of rules, that way we reduce the noise in the developers team. If there is a security and a developers team to separate that. If there one is a developer, like many cases in communities, maybe it's a different approach but this is also done sometimes. Yes, we thought of where to and find more new, more new errors, these errors come from the changes. Maybe if they help you, you only need to analyze the new errors and reduce the work of this. So as I understand the question is what happens if we have to analyze, let's say, we see a false positive in an area of the code but then we have the same issue in another area and also what happens if code changes if we should analyze. We can analyze only the difference. Ok, we are just finished this one, we are out of time. Just if it is a false positive, it can mean it is a false positive in this specific section and we use, we do that, we stop saying this is a finding here but in other place we say it is a finding and it's ok. If it is that the rule is not working, it says it is a vulnerability but for us it is not, the option we have is modify the rule or remove them so we don't have the result anywhere. And yes, we can do partial scans of the things that have changed. That's for example if we do the scans in merge requests that would be the approach to scan only the thing that changes. Please send me any questions to e-mail or catch me in the hardware, that's fine, thank you.