 Good morning and I'm extremely happy to see so many people in the room. Normally the room is full when either the talk is AI or you know it's Dan Walsh. So I am none of that but I'm here to give a brief talk about what secure development is and what secure development is from an open source point of view. My name is Hosefa. I am a lead security architect. I work for secure development in product security. I am a Fedora contributor for a very long time probably for the last 10 or 12 years. I have been speaking at DevCon. I have almost spoken at all DevCon except the last one I think and I normally speak about security. I speak about security practices. I've spoken about heartbleed, shell shock and you know I've spoken about those topics. So why are we basically here? We are basically here to talk about secure development and when people talk about secure development they normally talk about you know how I can write my code in a way that is secure but that's not really always the case right. Secure development is not only secure code but it's the process of making sure that your system is secure, your project is secure from the time when you design the project to when you actually write the code to when you build the project, when you run the project and when you develop your code, when you deliver your code and the binary to the customers. So the question which most open source projects will usually ask us is that you know people think that secure development is for large companies and enterprises like probably Red Hat or IBM because you know their developers and their customers are normally asking about security but that's not really the case right because most of the projects will start small. So if you take the example of say open SSL or you know if you take example of Mozilla or Bash or you know whatever, most of the projects really start small. It's just a few people who you know get together, they have an idea, they work on it. In some years it's common to see these projects being used everywhere. So if you take example of Bash, Bash started quite small but if you look at where Bash is used nowadays Bash is used in a lot of projects which you can't even imagine. When we were working on some Bash security issues we figured out that you know Bash is used in network attached storage devices. It's used in your television set of boxes. So you know the cable operator can give instruction to the set of box and you know you can see those channels. So the moral of the story is lot of projects start really small. So when you start working on a project it seems that you know my project is not not big enough to have a secure development process but that's not really the case. And when the project is small, when you are actually trying to work on a project and design a project that's the right time to work on secure development. And by the time your project is very very large right. So if you take example of open SSL again if when your project is very very large it becomes really difficult to inject security into the project because you know the design is set in stone. You have developers and you know you have customers and everything. So the right time to do secure development from an open source point of view is when you are actually trying to start the project. And remember one thing security sometimes can be expensive when it is when you try to add security later on in the project. So when you start the project that's the right time at that time security is cheap but you know as the project progresses as you have customers as you have users it's really difficult to have security after that. Small small projects right. Small projects don't have the inclination or don't have the money to do security because the general thought is that you know security tools are very expensive. So if you look at off-the-shelf security tools like Coverti or you know these kind of scanners or something like that these scanners are extremely expensive. So if you're a small time project it's very difficult for you to have that kind of money or you to have that kind of manpower to have the security process. So at the end what happens is that you use each a process in your project in which your project is mature enough you have people who use the project but there is no security at all which is a big problem. So we are trying to look at those problems. So what I can do to fix this issue right. So there are three things which I can do and you know when I say I can do it means basically if I if I have an open-source project I have a few developers I have a few users maybe I don't have a lot of money to and I don't have a lot of manpower to invest on security. They are three things which an open-source project can basically do. The first thing is learn. Learn what secure development is. Learn how secure development can can be done and learn what I can do as a leader of the project or what I can do as a developer of a project or you know if I am the QE person for for for the project I need to learn what I can do. The second thing is change the mindset. Security is a lot about changing the mindset then you know doing the actual work like like I mentioned it's a mindset and not a process. There needs to be security in every stage of your software development life cycle and very very important observe what other people are doing right. There are a lot of people who are doing good open-source security try to understand what they are doing what resources they are they are using and how I can adapt that to my project and how I can how I can use that for for for my software. Last but not least they are they are immense amount of free tools which are available right. You don't need to you you need to go away from the mindset that you know I need to buy this or I need to buy that. They are immense amount of open-source and free tools which are available which you can use and you can use that to do a lot of things in the rest of the talk we'll probably look at some of the tools and we will see how those tools can be useful to us. There are a lot of other open-source projects which are available like you know we'll take a brief look at Google OSS first and how you can use that to improve the security of your project. So they are huge amount of learning and application resources which are available to work on on the security of your project. So trying to reiterate what what secure development is secure development is not just code audit and patching and stuff like that. It is it is security of the entire life cycle of the of the proper project right from the design phase to you know when you develop and you know all all of those things and think of security from what I mentioned earlier think of security from design to the delivery phase when your project goes end of life what happens after that do you tell your customer that you know on so-and-so date my project will be end of life so if you are still trying to use it use it at your own risk. So all of those things are related to secure secure development. Now they are eight things which I am basically going to talk talk talk about and I know we we don't have a lot of time in this talk but they are eight things which are normal open source project can basically do when you design your project right get security not get security not knowledge as as much as you can and a lot of this security knowledge is free there's OS they are other resources also which I'll probably talk about later on in in in the presentation know what a threat model is right and we'll very briefly talk about threat threat model later on know how a threat model can be done they are various free tools available to do a threat threat threat model so know that as well when you're storing your code right know where you want to store your code if you want to do it on github if you want to do it on gitlab what are the risks which are which are associated with that if you want to create your own git repository on the internet and you want to do it over there know what are the pros and cons of doing it so know where the code is being stored and what are the security aspects associated with storing the code at that particular place when when you write the code this is the third third part right when you write the code learn what secure code development is learn how code can be written in a in a secure way if you are using mem copy is it safe should you be using something else if you are using string copy is that safe or should you be you should you be using some something else if you have multiple projects who are working with you if they are multiple people who are working on a project see if you can get a peer review before you come you commit the code right there are a lot of people out there on the internet who are willing to help you and criticize you both right which is probably good for for your code so if you write code see if somebody is is able to help you trying to review your code trying to review your pa pa pa patches and see if there are any issues with that when you build your code right then sasti is very very very it's very very important right before you build build build the code my my colleague of flodentshow gave a very good a talk on sasti and same same grep right they are other tools available as well we'll talk about some some of them there is a lot of integration with github gitlab which will allow you to automatically scan your code before your code is being built or you know after your code is being built while your code is being built so there are a lot of again free tools available which can do static analysis understand what secure compiler is what are the secure compiler defaults which you want to use so if you are delivering your code in the form of a binary to your customer or to the user know what secure compilation is and you know what are the different defaults which you can use when you deliver your project whether it is in the form of a source code or a binary figure out if there's a way to sign your source code if there's a way to sign your binary so that your customers or your users will know if there is a compromise around 15 years back I'm not sure if you if you remember vsftpd was compromised right the source code was compromised and somebody inserted a backdoor into vsftpd if you give a smiley command to vsftpd you will have full root access and the person who did did did did that he got onto the vsftpd servers he put in the backdoor he recreated the tarball and he put the tarball back onto the server and the problem was at that time vsftpd was not signed so anybody who used that tarball probably got compromised as well the good thing was that the author realized this in a couple of hours and he could remove the backdoor and you know from from that point onwards he made sure that the tarball was signed so you know all of those things are very this very important when you deliver your code to your customers or users in form of either source code or or binary supporting your code okay make sure that you clearly advertise on your website where security issues need to be reported if somebody needs to report a security issue then please use this email address do you prefer the emails to be encrypted or you do do you prefer plain text emails what is the timeline which the customers and users are looking at all of those things are very very important so that people know where the where the security issue needs to be reported and how fast or slow you are when those security issues are reported to you when your project is end of life this is what i discussed earlier make sure you clearly mentioned this on your website log4js is a typical example right the earlier version was end of life it was mentioned on the website in very small words nobody cared people still used it and and then you you know what what happened after that so clearly mentioned on the website in clear words that my project is going to go end of life December this year if you continue to use it then use it at your own risk there will be no security patches which will be applied after that last but but but not least what i mentioned earlier there are a lot of free options available so research them there's osfuzz if your project is applicable for osfuzz then it's a very good tool it does fuzzing for you free of cost and you know if you know what fuzzing is then fuzzing is very expensive so if somebody is able to do it free of cost then that's an added benefit to you so we don't have a lot of time so i'm just going to talk about things which are very very very important threat model is number number one right threat model basically means you try to decompose your application and put it on paper try to figure out where data is flowing from one end of the component to the second second end and what are the threats to the model to the design of your application in threats from inside threats from outside so we are trying to figure out what the threats are there are many ways of doing it osfuzz has got a lot of exhaustive information available in threat for threat modeling osfuzz has got a lot of automated tools as well right so you can use those tools there is a curated list on this github site so you can look look look at that it contains books it contains resources it contains free tools which which you can use so the threat model is not impossible you don't need a lot of security knowledge to do threat threat threat model so that is some something which really can be done again when you write secure code learn audit and repeat okay the trick is to be paranoid with all the input any input which goes into the application please be very very paranoid with that input you don't know what will happen especially if that input is is not processed if it is not sanitized in the way it should be then you should be very very paranoid with that input understand that everybody in your project is not at the same technical level as you are or probably you know you are not at the same technical level as everybody is so if somebody writes the code and if you feel that code is not written very well then make sure you tell that person so that you know any future code which you write is written in in the right way learn from each other do code reviews do code code audits as as much as you can use sasti right this is very very important its static analysis can be used to find flaws and even bugs in your code there are a lot of free stuff available which can do sasti github has got free sasti which which is integrated they are like 40 or 50 different scanners which are available there's a github has got its own free language called code ql which is which is thereby by default but there are a lot of other scanners which you can also also enable i think it provides you some 1000 seconds or 10000 seconds per month or per week or some something like that which you can use in github actions so you can run these tools with that provides very easy integration with cicd pipelines as well so if you are using cicd inside github then sasti gives you very very and github gives you very very good integration for fuzzing like i mentioned earlier all the cool kids are are doing it but fuzzing is computationally very expensive you need a lot of computational power power to do for fuzzing signal to noise ratio is very very low which basically means that you know if you fuzz for a couple of hours or a couple of days or a couple of weeks you probably find one or two flaws because that's how fuzzing basically works but there are a lot of again free tools which are available they are free resources which are available if your project is eligible for os s fuzz then nothing like like it what os s fuzz basically is is a google project right in which you know you give your project to google and you know they fuzz at their their ends they have a high end fuzzing cluster which 20 000 nodes or 40 000 nodes at their end and whenever they hit anything they will automatically file a bug in your bug cc system and you can look at it right so they are a lot of free free tools which are available there's hung fuzz and you know they are a lot of free free fuzzers also available the only catch is you know those things need to run on your machine but if you use os s fuzz it runs on google infrastructure so you know it's win win win for for for everybody like like i mentioned earlier make your security stance known okay clearly notify on your website what the security address is who is responsible whether this project is only your part-time project and you will get to it when you have free free time so you know whatever your security stance make sure it is it is available secure code is money in in the long time right if if if you if you write six secure code more people will use it probably you can productize it you can monetize it as well but if you write crappy code which is insecure then you know probably nobody wants to use it that's my talk yes yeah i i i i think the question is that you know what i mentioned earlier that secure code is money so especially for startups where customers are more interested in features and probably less interested in in in in security so the question was whether the landscape is changing right i think it is changing a lot nowadays right and the thing is that you know if you have a startup and you know if you have a project or something like like that which your customers are buying or you know they are going to buy there is going to be a time probably in the near future when your customers realize that you know your project is not as secure as they would like to or you know they are new threats which are which are there in your project and by by that time it's going to be too late for you to go back to the drawing board and to change the design and you know to change the code and stuff like that so i think the customer mentality is changing as well plus what what you need to understand is that you know what what secure development basically means is security baked into your lifecycle which means you don't really need to spend additional resources or additional cycles trying to do security you don't need to hire security engineers you don't need to get your code audited by a third party auditor who is probably very very expensive and is going to charge charge you a lot there are a lot of free resources and they are free workflows which are available like i mentioned github earlier github has got integrated sasti integrated malware integrated dusty all of those things github basically has and you just need to enable it right and it needs to be a part of of the work workflow while you are while you write your code wouldn't it be it'd be great if your ide your integrated development environment tell tell you that you know you know you just wrote a function on line number 50 but that's not safe so would you like to revisit so it's a part of your development process no additional cycles are required later on to to to to look at it and from a startup point of view if you feel that you know there is no advantage for me to do security right now because the customers are not asking us or you know because my priority is to have more free features then then then security it may hurt you in in long term and you know we have observed that with a lot of startups that initially customers buy your product because you know it's a new thing in the market and you know they want to do it but later on there's some other startup which is doing the same thing but they they basically say that you know we are doing the same thing but we are more secure so it it hurts you in in in the long long term i think yes when you have a lot of money so his question was do you feel that a bug bounty program is useful from from a security point point of view there are a lot of conflicting views on that right my personal opinion is that like what i mentioned earlier you need money for for for bug bounty pro program to run there are a lot of companies who give you free amazon vouchers or some something like that if you find a bug they are not a lot of hackers who would go for that that kind of thing right now people normally need the money need the money you can be associated with bug bounty projects like hacker one and stuff like that they are useful for a particular kind of thing when you are consumer project right so when you are a project which is a web app or something like that or you know a mobile application or something like that it may be useful when you figure out that you know there are a lot of security flaws in my project but you know my my team is not not able to find out what what what those security issues are also remember bug bounty hunters are you know after money they are not after improving the security stance of your project so you know if you if you have money and you know if you feel that you know that's going to be useful i know a lot of companies who started bug bounty for six months or ten months and they realized that you know it's a waste of of of of effort because you know the researchers they write ten pages of research reports saying that you know with screenshots and videos and you know stuff like that and it was not very useful for them so i think if you have the resources maybe you can you can give it a go and you know see if you get any valid things but then it really depends on how much resources you you basically have probably with that money i can buy a good secured buy i can hire a good security engineer and i can get more output out of him then you know trying to get it from a bug bounty program yes so couple of projects which i have been i have been working on okay the the question was that i mentioned that you can look at other open source projects for for for inspiration as you know what those projects are doing i have been working with a couple of projects in the last 10 years and some some some of them which i would like like to mention is jillip see okay jillip see is currently doing a very good work of you know trying to they they have some some some time back i i saw a white paper which is i think public on the internet which talks about all the security features jillip see currently has what the roadmap is what they are trying to basically implement by when they are trying to implement those those features and you know what are the resources which are which are required so this this is a good thing right i mean you are basically talking about what current security features you have what you want to do what your roadmap is so that your users clearly understand what your security stance is jillip see also has a page which i think open ssl also has which says that if you find this this this kind of report it is not security please don't bug us with it right which is very very important because you know people will tell you everything that you know i found a dose and this is security i found this and that is security so this is very very important because it it it increases your signal to noisy the issue there are less people reporting stuff which may not be secured i think that that's one very very good example open ssl after heart bleed is doing a lot of good security now they are doing fuzzing with os s fuzz and they are doing a lot of useful things things things as well they are doing they are they are doing auditing as well mozilla is one very good example mozilla runs a bug bounty program as well so that that's a very good example as well but you know those are those are large projects they are very small projects as well which which are which are doing good work i think yes okay yesterday i spoke with a person who is who is writing a lot of code in rust and he told me he is creating a a collection of how you can write insecure code with rust right so the the general understanding is that because i'm using rust i am i'm not affected by all the classical issues which normally cc plus plus so you know these kind of languages have so i i i think even if you try to use a secure language like rust or something like that you should still understand that you know you can write insecure code with with with rust right and you should be familiar with all the different compile time run time things which can be enabled or or or disabled irrespective of what kind of language you basically run come on there there has to be questions my my teacher you should tell tell me if you don't have questions writer means you have understood everything or you have understood nothing yes thank you i i would like my my government to spend more more money as well on these kind of things which which is great actually you know the thing is normally government agencies are not really aware of you know that you know there is open source number one there is open source security as well even though you know a lot of open source is being used everywhere including government installations and you know there there there's also the servers and everything so this kind of awareness is really really important and if there's a government which is doing it then it's it's a really great great initiative if you are running an open source project and if you if you're eligible for for this kind of incentives then you should definitely definitely you use it i would love for other governments and you know other government agencies to be aware of these kind of things as well and to be to be able to spend this kind of money to to support these people it's a really great great great initiative i think yes so your question is how safe is it for us to use code written by ai your question is how safe it is for us to use code written by ai kind of yeah so how how how easy or difficult is it for people to understand you know and you know kind of the view what ai is doing wouldn't it it be great to say chat gpt please fix all my security issues right so i i think the the the thing about ai is that you know it really depends on the data which is used to teach coding to that particular model right and if you if you if you look at if you look at code on on the internet and if you know if that kind of code is is trying to teach ai how how to write code then i i don't think the output would be very very good very good as well right but that that being said i think there are a lot of projects a lot of security projects for which ai can be put at very good good use and one one thing is you know for for example when you do sastry scanning okay one thing about sastry is there are a lot of false positives which come out of sastry it would be great to feed those false positives into a ai model and a ai model be easily able to figure out if there are any future false positives which come from from your scanning right you know so that that that would be great right so it's a it's a feedback cycle in which you feed the false positives into the model the model figures out that you know if this kind of information we get in future from sastry scanner it's a false positive as well so it feedbacks feeds back into the sastry database so these are some of the useful things which you which you can use but right now the state in which i think most ai models are if there's a code which is generated by ai and if you want to use that code i would be very careful about trying to use that code i would my personal opinion is i would have at least one human look at the code to figure out that you know it it has been done in the correct way yes so what i feel about sastry tool is that it's it's it's like an antivirus right if you if you scan your system with one antivirus it it may not be able to detect something which other antivirus can so i i feel and this is my personal opinion i feel that a combination of sastry tools for your project may may be more important depending upon the complexity of your project what code base your project basically has plus my second observation is there is no sastry tool which can scan all languages it's very difficult to find a tool which will which will scan everything in the same way so a tool which can scan cc plus plus probably won't be able to scan ruby python go go lang java something like that so in the end if your code back if your code base is very complex and consists of multiple different languages you may end up using different tools because you know one tool is more efficient in this language the second tool is more efficient in in in this language so right now i i think the the right recipe is a combination of different tools if you have the resources for it would would would be much better than using a single dedicated tool yeah one one one problem with using multiple tools is like you know you have duplicate number of issues which are which are reported by each tool and then for the person who is looking at the issue it becomes very difficult to figure out what the actual issue is right they are there are there are a lot of tools available on the internet free of cost paid different kind of things right which can do something called deduplication and what deduplication basically means is that you know it sucks things from the scanners right and it is able to figure out that the same issue has been reported by scanner one scanner two scanner three so instead of showing three issues it will show you one one one issue so you know those kind of things are really important because the other thing with sasti tools are is that the output is very chatty which basically means that you know they are they are they are lines they are pages and pages of logs so for a for a developer or for a for somebody who's looking at those logs it may be very very difficult for him to understand what is actually going on that is number one thing number two thing is the code base is very large so if you are scanning colonel liberty office mozilla or something like that you have like 10 000 flaws or 20 000 flaws so it's very difficult for one person or even for a team of three or four people to be able to actually look at it and try to figure out if something is wrong i'm out of time thank thank thank you very much for for coming i'm there in the conference if you have any questions then we can probably meet in the hallway and we can chat thank you