 Good afternoon everybody and welcome to IOT Lockdown Bowling the Botnet Builders. I am Adam Englander and I work at Iovation actually here in town and hope everybody has a nice little break and they're ready for some some security. It's a little sparse crowd so if you have any questions in the middle feel free to bring it up. It's pretty interactive if you want it to be. I don't have a problem that at all. So the first question I have for everybody is what happened on October 21st 2016. Anybody know? That happened. This is an outage. I was actually in Las Vegas so I was right about there. What happened? My company's data center here was about there. Our other data center in Seattle was there. You could not connect to our services. We could not connect anything to manage it, monitor it, do any of these types of things. This was one of the first DDoS attacks I've ever seen or any kind of attack that the entire world noticed, right? Because you couldn't get to Netflix. You couldn't get to Gmail, right? You couldn't resolve names and some places had issues with traffic, other places had issues with just not being able to resolve DNS. This was a tough day. I was in a little conference when this happened. Trying to figure out what the heck's going on, stuff flying around. It's a good time. What this was is this was a distributed denial of service attack on Dine, which used up to 380,000 IoT devices. This is why it was so hard to stop. There were 380,000 non-related IP addresses slamming Dine. Generating up to 1.2 terabits per second of DNS requests. This massive flood of data coming in. Producing, even while they were mitigating it, it was still producing 50x renewable traffic. This was one month after basically the exact same thing happened to Krebs on security. Krebs on security was attacked by the exact same botnet, maybe not the same devices, but the same basically the same software, the same attack, except they weren't doing DNS flooding. They were doing HTTP requests. It was double the worst thing that Akamai had ever seen. Akamai actually dropped Krebs on security from their network because they were doing a pro bono and they said this cost us too much now that you're being attacked like this. We cannot afford to deal with mitigating assaults on your site. It is our job to prevent this from ever happening again. This was 100% IoT related. I'm assuming that you care because you're here. There are a lot of other people that aren't. So I'm going to try and help you out a little bit and maybe give you some information that you might not have, hopefully a couple tricks and tips that you can use. I'm assuming based on this conference that we're all doing embedded Linux development. Some of these things may apply to non-embedded Linux, but embedded Linux is where these attacks are coming from for the most part. Because there's been a wave of IoT development. IoT development with embedded Linux, it's made developing and manufacturing more accessible. We can rapidly prototype something and just do it on a very easy development board, move it over to a production board, running embedded Linux super easy, super quick. It's reducing time to market. It's becoming more and more popular. You have more access. You don't have to be able to write stuff onto a chip in C to be able to do IoT development. You can do that with things that actually go into production now. That's the exciting, the brand-new dawn of a new day for IoT development. There's also security benefits for embedded Linux. Linux provides facilities for fantastic security. The Internet runs on Linux or a Unix variant for the most part. There's some of the world that runs on Windows, but it's not a lot. It's not a lot of things like Facebook which runs on Linux, Google which runs on some of their own stuff, kind of weird, but mostly some sort of star edX type variant. These people have, and over the decades that's been around with the other conference that we're kind of joined up with, you can tell there's a lot of stuff we're making it fantastically secure. Because of that, there's numerous resources that exist already for implementing super secure Linux because people have been doing this for a very long time, hardening these things, making these things very, very, very secure. But in all that, we have to remember two things. I'm coming from security background, so I know that, number one, you will be attacked. If you have something connected to the Internet, it's being scanned right now. There are good actors and bad actors right now that are scanning every single IP. They literally just go through the numbers and they connect to it and they start knocking. Is this here? Is this here? Is this here? Is this here? And it just records it. Sometimes they're looking to see, just be able to researchers to find out information. How many DNS servers are out there? How many people have an Ovo and FTP port? They may come back later and say, do you have security on FTP? Do you have security on SSH? So just being on the Internet, you will be scanned and the likelihood that you'll be attacked is 100%. And you will be exposed to a zero-day vulnerability. If you don't know what a zero-day vulnerability is, that is, if you have good actors and you have a good ecosystem, someone will find an issue, a security issue with your system and they will alert the community and say, hi, I'm not going to tell anybody else, but I found this really terrible vulnerability that you have. And I would like you to fix it and I'm going to give you X number of days to fix it before I tell the rest of the world because it's a really bad vulnerability. And the world is vulnerable and they need to know this. Sometimes they don't fix it. They decide it's not a problem or they decide that they're just not going to do it. And sometimes it's just not found and it's not a good actor that exposes it. And you find out after a hack happens. A year and a half ago, Heartbleed was one of those. If you were doing anything with Linux, everybody started freaking about this thing called Heartbleed. It was a vulnerability in OpenSSL that was a zero-day vulnerability. You found out the day that you needed to fix it. And so you had to apply all these crazy patches to prevent the problem before OpenSSL actually came out with a fix. So these things are just going to happen. You're using Linux, it's just going to happen. It's going to be way better than what you can do if you wrote your own operating system, but it's going to happen. It's part of the system. The other super important thing is you have to know your adversary. And when we're building small products that do small things, we don't think very large. We don't think about, I mean, there's the lone gunman. There's someone who's just trying to hack things and they hack things for the sake of hacking things and they hack things so they can say they're really cool and they broke into stuff. They have no really other ulterior motives. They're just kind of seeing what they can do. These people range from the age of 10 to 90. They're usually not terribly dangerous, but they probably don't follow the same protocol as an ethical hacker does of letting you know before the rest of the world. They might find something and just spit it out to the world. They found this terrible vulnerability, but not tell you beforehand and give you a chance to fix it. There's local criminals. IoT has a very specific, different problem that most of your other Linux brethren don't have is that you're not giving information about a specific individual. When you think about what you're doing, if you have, let's say you're writing a thermostat or some smart home or smart office piece, one device by itself doesn't do much, but a lot of devices throwing out data allows criminals to have a profile of tracking the garage door opener, the thermostat, the lights. Oh, I know when they come home. I know when they leave. I know when they're not expecting to be coming back because they have different parameters on what they set things. So if I'm not going out for the day, I'd drop it 10 degrees. If I'm going out for a week, I'd drop it 60 degrees on the heater because you're not that worried about it. And if they drop it really far down, that means they probably don't have a dog. So there's a lot of things that one little piece might have anything to do with, but as you put all this stuff together and you build a profile of someone that a local criminal just leaching off of information being sent on IoT devices, they have the ability to create profiles to be able to know when they can do an attack. Whether that's a high profile individual or even just a low profile individual, we're just kind of standard theft. Hacktivist groups. I haven't seen that a lot in IoT, but it does happen and it's going to happen. If you have, there are groups that think that certain things shouldn't exist or that things are bad. There's a few fairly recent examples. Ashley Madison was a big example of that where they got hacked because someone said, you're a terrible person, you're doing terrible things, and you shut down or expose you, and they didn't shut down and they got exposed. So if your devices are being used in some place that someone doesn't like, they may use your devices to do terrible things to people. And whether that, whether depending on what side you're on, you may be really upset or really happy about it, but the likelihood that it's going to be used on something that you're going to be upset that it happens, that your device was responsible for it, is pretty high. Because there are angry people about everything. Competitors. So competitors, either competitors of yours, right, that can find these things, they'll find these bugs, these vulnerabilities, and inform the world that you're not secure, or competitors of people who use your devices, using your devices to gain access to their network and gain information. Organized crime is a big one. For organized crime tends to be something where they're doing extortion, where they're finding out information about individuals, and using that to extort money from them so it's not released. And then nation states. So nation states, it's a really kind of weird thing. Nation states just collect a ton of data, because they're just, they just want to collect all the data they possibly can, because they have the computing resources and power to use that to make determinations. So they just use just massive, massive data collection, whether it's our government or somebody else's, they're all doing massive data collection. And if they can use your devices for doing those data collections, you're probably not doing your users a favor by giving them the privacy that they're expecting, that you think that you're giving them. So now that we kind of know that, we start talking about security. So security is like an ogre, it has layers, and I hope somebody gets that joke, because it's one of my favorites. And it's good, it's layered security, it's defense in depth. This is not something that is specific to security, this is something that is specific to defense. If you go take a look at a castle, it has a moat, three walls, and a keep, because they know that any one of these things can fail, but all of them probably won't. And the way we do that in defense is that we do it through layers in the system. So your applications running on an embedded Linux system has layers, and what we have to do is we have to understand these layers a little bit, and we have to understand how each one of these layers really takes place, and what the things that we can do to help protect them. At the network, the application, service, file system, and operating system level. If you're like me, you're probably not a great sysadmin, but you're writing things that run on a system. Whether that's you're doing IoT, there's another huge problem that's coming up with containers, right, the Docker world. You now have people that are writing applications that are running in containers that run on Linux, they don't really pay attention to the Linux side. So containers and running in these systems on Amazon and Google have their own problems, the same that IoT has, because neither of them are really good at, none of us are really good at being system administrators and being Linux security specialists, but now we don't need those other people to run our systems, and not having them is creating some havoc. So at the operating system level, you have to have a patching strategy. So there are vulnerabilities that you're going to have, and you're going to have a device that's out there in the wild somewhere. You have to come up with a way to be able to patch the operating system, and patch the operating system in a manner that is secure. It's something that a lot of us don't think about, right, if you, I don't know how many drones I have, that have no way to update their firmware or to update their actual operating system, right, they're running on a Linux system and they're always going to be on the wrong Linux system and it's whatever vulnerability it has when I bought it, is the vulnerability it's going to have to the end of time, because they did not give me an upgrade opportunity. One way, one of the things that you can do that's becoming, actually, we might be able to leverage some of the container world, is minimal distributions. So there are Linux distributions out there that are super tiny. They're meant to be super tiny, because they're meant to take up as little memory as possible, because in the container world, if you can reduce the amount of stuff that's rolling out there in containers, you can help application developers stop doing as many terrible things. We can leverage that in the IoT world as well, right. One of the things that you normally do is you figure out I've got a standard Linux distribution, whether it's Yachto or Raspbian or whatever the case may be, and what you want to try and do is start paring down. But the other option is, is you can start with nearly nothing and then add a couple things that you need. Because most of us, we don't need SMTP, we don't need FTP, right. We don't need all these other services that come with it, we'll normally provide that in the application layer. The number one problem with, if you take a look at the statistics on how these botnets actually happened, you actually take a look at a lot of the early drones and how they were actually hacked, it's because they used a default username and password for every single one and they never changed and there was no way to change it, right. You've got a limit Linux system that has a user root and that either root user either has no password or it has the same password for every device, which means all I have to do is get into one and I can get into all of them. So my suggestion to you is to find a way to do some randomization with that. If you can print out a serial number, you can associate a random value with that and that value can be the random password. So that if they managed to get into your source code and find the default username and password, they don't find everybody's password. They have to go crack your database and go find a few things, right. That's that defense in depth. Make it a little harder for them to get through, make it more expensive, make it more difficult for them to get in and require strong passwords. So authentication modules allow you to, inside of Linux, allow you to require strong passwords. Don't let your users hurt themselves. You probably hate it as much as everybody else, right, when someone requires a strong password that you don't remember. But we're talking about devices and if the users probably should never be able to connect through the terminal, right. We were supposed to be building applications so they never have to connect the terminal. So why would you not want to have a super secure password for that one case where someone might have to do that just to recover their system somehow. At the file system level, named application users are a really big deal, right. When your application is running on this device, it should not be running as root. Depending if you're using something standard, right. If you're using like a Python server or if you're using node, or maybe you're running a JVM or if you're running C. The other ones are a little easier, they're pretty well stand out, but you can basically you start your application as the root user, connected to port 80 or whatever port is a high level port that you need root access for and then you just spawn off another fork and that fork is a much lower level and that fork is going to process your request. It's the same thing that Nginx does, the same thing that Apache does, the same thing that the the Java servers do. You start up as root, but when you're actually running the code, that fork is not running as root because if you give somebody root access they can do terrible, terrible things. Basically anything that you can possibly think of, they can do it. Restrict your application to only the files necessary to run. It's super easy for development to give yourself access to everything so that you don't have to start figuring out file permissions and that's fine for development, but once you start actually getting ready to put your application out there, you've got to harden that up and say what does this actually need access to? Should I be able to write to Etsy? Should I be able to write to the configuration files for the entire Linux system? Absolutely not. If your application needs to write there and it's not for just the initial setup, it's probably a bad thing. Avoid write access wherever possible. There may be certain issues, instances where your application has to write, but limit those as much as possible because giving your application write access is giving it the ability to change things at the operating system level, at the application level, possibly even overwrite your own source code. If you're running something that's not compiled, if you're running something like a Python or JavaScript, someone can just rewrite your code and now you're executing all new code, which is a really, really bad thing that you don't want to get people to do. Once you get outside the file system you get to service security. This is a problem with application servers everywhere. And one of the things that I've done over the years in application development and more recently in IoT development is reduce local dependencies. And the way that you do that is you don't process things locally if you don't have to. A lot of IoT devices we're finding the best way to do that is you're going to have some sort of proxy to go do all these things. You're connecting to an external service to manage. Don't use SMTP locally. It may be easier. But now you've got an SMTP server running on your IoT device. And that IoT device is going to be accessed and if you've done something bad someone's going to go great. Now, I've got a Linux computer I can start sending spam from. We all become part of the spam problem whenever we do that. FTP, SMTP, right? If you don't need to get SSH access just don't even install it, right? Like I said, most of the time we don't, users should never have to go to a console to run our applications. If you do have services and it's something that you do have to have, so you have something that's completely separated. It's not going to be connecting to anybody else's systems. It's not, it's a single standalone device that needs to be a single standalone device. Require authentication for those services. Your application knows what the authentication is but someone can't just get out of the box and start using it and setting it out. It makes it a little more difficult on your side because you have to pass credentials through to SMTP or whatever the case may be. But just make sure you have them so that someone can't just start sending things to your IP address on port 2727 or anywhere they want to and just start sending 25 SMTP. Turn yourself into an SMTP transport. It would be a really bad day. And be as secure as possible with service data. Understand the data that you're putting in these services, understand that most of the stuff when you're talking about the Linux side, it goes into a file that is text and can be read. SMTP goes into a queue. How does it go into a queue? It's a file that just gets written to a directory. So if there's super secret information there, you've got a problem because it's accessible through the file system. So be as secure as possible with that data. If you've got data that's secure that you're going to be sending, encrypt it. We'll get to that a little bit more. On the network security layer, if you can do it and if we're all talking about it seems that IoT for the most part is moving towards kind of dumber devices with a smarter, central brain at the location. If that's your case, you don't even need connection bound connections. Everything's outbound. You should be connecting to whatever your brain is at that particular time, and you're going to proxy through that to talk to the user. That is the most secure way to do it. Not everybody's going to be able to do that, but that is absolutely the most secure way to do it. Do not allow direct connections to your device ever. You have a device that you're going to tell it to connect to and you know what that device is and you're always going to connect there, no reason for anybody to come in. But again, that might not be your situation. If it is, restricting inbound and outbound traffic. If you don't have port, let's say you have to have an SMTP server for whatever reason, you've got to have an SMTP server. Don't allow inbound traffic to that port. You just don't want to do that. Because you don't expect anybody to be sending SMTP to you, you're just sending it out. If you're not going to be having, if you're not sending out SMTP, restrict SMTP going out. It's a very common thing that just restrict the data going out so that you don't even if someone is running code on your system, if they found a vulnerability, they can't do bad things by going out on something that you shouldn't be using anyway. That your application doesn't use on that device. Prefer pair Bluetooth. For some of our stuff, we need LE. It's absolutely essential that we have LE. If it's not absolutely essential that you have LE but you want Bluetooth, using standard Bluetooth creates a very secure pairing connection that cannot be someone can't connect to it without having the correct keys. There's a key exchange that prevents someone else from being able to do that. When we're talking about LE, you just get a device ID and any one of us can bring up light blue on our phones. We can copy that ID and we can say hi I'm this ID now and start talking back and forth. When you use standard Bluetooth the pairing process is a key exchange that actually encrypts the data going back and forth so that you can't sniff it, you can't pose yourself as that device, it's much more secure. Not everybody can use it, not everybody has the ability to do it, but if you're doing Bluetooth, think about why you're using LE and not just standard Bluetooth. And if you are using standard Bluetooth, challenge response so that you at least have, you know this isn't happening remotely, that somebody is not trying to spoof Bluetooth or they're not on the other side of a wall trying to connect to your device, they have to be able to see an actual value that they're going to have to put in to be able to complete the challenge. Which brings us to what we all do, right? Now we've got through the kind of the basic stuff of here's how we will protect the system but now we get to our application. I assume that we're all application developers, we'd be in the other rooms with all the Linux folks. So we have to know what we're preventing and do as much as reasonably possible. And that's always a difficult thing in security is that we want to be as secure as possible, we also have to be easy to use. That's always a struggle any time you get through that is making it easy to use but still be fairly secure. And again, with your application you have to have a patching and update strategy that is secure. So you have to be able to update your code on your device. How do you do that to make sure that somebody else can't do that on code that you don't want on the device? We'll get into that in just a minute. So here's the things that you're preventing. Account hijacking, right? Sensitive data exposure, escalation of privilege, denial of service and remote code execution. These are the real big ones. These are the things that are the terrible things that are going on every day. Sensitive data exposure, the way that you prevent that is you have to start getting into the terrible role of crypto. Crypto is confusing and it's very hard. I may be giving a talk on crypto on Thursday. I haven't got the response back. They said they wanted me to do it but I haven't heard back yet. But that's how you protect your data. Protect your data in transit. Now an easy way to do crypto in transit is TLS, also called SSL. You should not be using what's actually SSL anymore. You should be using TLS. Because SSL and all of its versions is no longer secure, but it's just something we call it. We call it SSL. Those protocols are available to you. Straight through open SSL or whatever library you want to use on your Linux distro. So if you're setting data out and you have the networking capability and the processing capability, crypto uses power. Crypto uses CPU. Crypto uses memory. You're talking about chips and embedded code on chips that is really hard. But when we're talking about embedded Linux, you probably have the memory and the CPU to be able to do that. So think about it. Think about why you're not using SSL to connect. Back and forth. It's a little more difficult and you're going to have to figure a couple more things out. But it's not that hard. There's a lot of information out there, along the lines of your distro, whatever distro you're using. And it makes the world of difference. It means that when you're going and connecting somewhere, you can do validation to make sure that you're getting the actual response. Someone hasn't stuck a little router in the middle of that, and they're just changing the responses coming back to you and putting code that you don't want to execute on the system. And protect data and rest with encryption. If you have things that you don't want to be able to get on the box and see, you want to encrypt the data that's sitting there. You want to do your best to protect as much as you can. Again, it's defense and death. You're protecting it at every single layer, even down the application, and then your application has multiple layers. You have a network layer, you have memory, you have file system, all those types of things. So one thing that you can do, which again, it's a little more advanced, is certificate verification of your fingerprint. When you're doing SSL communication back and forth with another device or outside on the internet, when you're going over SSL, you're using a certificate. This is a public-private key pair. And you're making certain assumptions. It's telling you that this is the authority that gave me the certificate, and you go and validate it. Everything's great. But if I'm going to perform a sophisticated man-in-the-mill attack, I'm going to give you a valid cert that came from a certificate authority that somehow, through the chain, proves out to be okay. Or if you've done development with self-signed certificates, and you forgot to turn off the verify SSL flag, super easy to hack. One of the ways that I do that, we do this a lot at my company, is that we use pinning. Or we say that I'm only going to allow a certificate from this certificate authority to validate my communication back and forth with a server over SSL. Again, it is a little more advanced, but if you have a device that's got information going back and forth that is important and secret, you really should think about it. Yeah? Oh, sure. All day long. Right, so a lot of times when people are developing, now, let's encrypt has made this easier. Right, and so if you're in development and you don't change your code to verify after self-signed, then you're wide open. But even if you validate the cert, even if you validate the authority, if you have an old, so what ends up happening is if you've got an old distro, these old distros have these old certificate authorities. If you're using an old distro that hasn't been updated and someone has gotten the private key for a side authority that's in the list, they can now impersonate all of these different things. So that happens from time to time. It's very rare, but it does happen. And those things go for good money on the black market. To be able to say, hi, I'm a certificate authority and this certificate is good. But like I said, it's when you're talking about this is kind of more nation state or very organized crime is going to be doing these kind of attacks. But it has been done and you can't protect against it. Yes. So certain protocols are going to allow some sort of TLS. You can either do some sort of you can create a tunnel over TLS and then use UDP over that point-to-point UDP. You're kind of losing the whole idea of UDP at that point though. The other option is that you can encrypt your data that's going across. So that you encrypt the data before it goes into transit. And if you're like me and you're really, really concerned about that stuff, if you don't just use TLS you encrypt the data on top of it. Because, again, there's going to be a zero-day vulnerability. So I was going to say I found a vulnerability in SSL open SSL where I can just bypass all of your validation on your certificates. So you could use some of the things that I use as a paranoid individual that you could use in every day is that you encrypt the data before it goes out. And there's some stuff at the end of the on this where if you're doing that and you're doing kind of more binary data which is more prevalent in IoT because you want smaller packets you can do kind of Seabor, a binary object representation and there's an offshoot of Jose, which is the JavaScript object signing and encryption that keeps it at a binary level for being able to do, provide libraries to do encryption of just binary data and pass along and validate it. That is your question? Okay. If I get way too complicated, you don't understand what I'm saying please just say, just let me know because I forget sometimes that not everybody understands all this stuff. I apologize. So encrypting data. If you don't want attackers to have access to it then encrypt it. Sometimes it's super, super, super secret and you don't want anybody to have it. If you're recording information that should not be available to anyone except for the individuals that are collecting it, they may care enough and you may care enough to encrypt it when it goes across and use a strongest encryption that's feasible. IOT, we all have restrictions. We only have so much CPU. We only have so much memory. Encryption takes up CPU and memory and we can't just you have to do what's reasonable. With most of the chips out there that are using embedded Linux you can do something that is secure. You can do AES-256 which is perfectly secure, perfectly quick. You can use RSA 2048 on most of them and you can kind of use the combination to get super secure and super speed to do that. I always remember that if you're sending data and it's important it could be around forever and if it's something that's going to be more along the use of user information person identifiable information that's collected or things biometric data that's collected biometrics is like you really need to watch out for biometric data to hold on that really well because that never changes. And just because you're encrypting it today what we thought was safe five years ago is not safe today. Everybody got these little warnings on our browsers and everybody had to update their SSL search recently to 2048 keys instead of 1024 because Google wouldn't allow it. There's a reason for that, it's not secure anymore. So when you're taking something like biometric data that has to be safe forever. It releases forever as you can make it. So always use the best you can the highest level of encryption that you can because it's got to live a long time. Just try and think of that when you're doing this stuff. So digital signatures digital signatures are used to verify data. So one of the ways that you can keep people from changing your code and being able to run it is with a digital signature. So if you send your code for a patch you can sign that code you can stick a file on there with the signature and then you can verify that signature against the code whenever you start up to make sure the code has not been modified and there doesn't have to be any information on your device that actually gives you the information to recreate a signature. So when you're talking about asymmetric key encryption a public key is enough to verify a signature but the private key only has enough information to create the signature. So you can create the signature from your central repository where you're getting your updates from with the signature and you can verify that it hasn't changed. There's no way that anybody is going to be able to recreate that. So that's super awesome way to make sure someone hasn't modified your code gotten into your system and given it something else to run that you don't want. Or they haven't found a vulnerability in your application to write new code to execute. You can also do that in memory if you're really worried about those types of things. It depends on what level you want to get at but there's a lot of things that you can do. And then sending data in transit. How do you know that your data hasn't been changed in the middle? If it's not secret data, you're not worried about whether anybody can read it but you are worried about whether or not somebody changed it on the way, you can sign the data as you're sending it and you can validate that signature that someone has messed with the data coming in. That it's good, valid data, that's what you're expecting to be, that's what it was supposed to be. So account hijacking. This is actually number one in the OWASP top 10. Again, application passwords. So, and just user credentials in general. So we call it password hashing. We call it password hashing because that's what people understand. But if you're using a hash password, it's a bad thing. If you're using MD5 or SHA or the SHA1 or SHA2, even with a salt, salt makes it a little more difficult. You should really be using some sort of key derivation function. Key derivation functions are PBKDF2, Bcrypt, Scrypt, Argon2i, these are all available and basically every Linux distro with OpenSSL can get PBKDF2 out of the box, which is not as good as the others, but it will do. It's way better than using a simple hash password. The ability to crack hash passwords has gotten so fast that they've stopped making rainbow tables. A rainbow table is just like, would come up with every permutation, so they go, oh, there's the password and there's the actual value it's supposed to be. Because that is so large the amount of time it takes to just actually read them in and go through them and find it, it's faster just to crack it now. So, yeah, it's pretty great. So do your users a favor and make sure that you're using some sort of key derivation function. Like I said, PBKDF2 comes on almost every distro of Linux with OpenSSL. Another way to prevent account hijacking is the initial setup via hardware or force wipe. So there's only one time when you can reset a password. That's either going to be that you have to physically be there with the hardware and you can put it in that state or, hey, if we reset this, we reset this. You're not going to be able to gain access to the information by resetting the password because there's not going to be anything left. So as a user, you better know if we're going to change the password on here, it's going to be completely gone. Like I said, you still have to kind of use usability versus security. What information is on the device? What are they going to be able to do? You have to really kind of think about that. But if you've got something that's super important, think about what can happen when that password reset happens, right? Everybody forgets their password because now you're doing strong passwords. So now nobody remembers their passwords. So now they've always got to reset their password. And what are you going to allow your user to do and what threat are you putting out there to be able to just do that for them? One of the super popular things on the world right now is just, oh, I will send you an SMS message with a special code. And it's super easy. There's a lot of things that you can do just through social engineering where I can take over your phone number today in an hour. I can take over your phone number and get SMS texts. So it's important when you think about security to think one level up. Don't think about yourself. Think about somebody who's important. I'm just some peon. I come here, I write code and nobody wants to hack me personally because I have nothing for them to take. But the code that I write could be used by somebody who actually does have something people want. And that individual's life could be completely ruined by me writing bad software. And that's something that I have to take into consideration every day. And an important thing too is alert for changes to accounts. If someone goes and changes, if they do a password reset on your system or they change the configuration and you have the ability to do that externally send alerts saying hi, I got changed. If you change me, that's cool. If not, you should really like freak out because somebody's doing something and doing something weird. Yeah. That's an actual picture. Google images is an amazing thing. Yeah, you put in any question and it'll come out with stuff. That's really crazy. So nonces are super awesome. And what a nonce is, it's a really weird word. I have no idea what the origin is. I'll probably figure it out one day. But it's something that can only be used once. But it's a single use token. And if enforced properly, it does so many things. It can make sure that that request only happens once. So one of the things that you can do is you can basically keep running the same request over and over and over again. And finding out what bad things can happen. Or you can try and hijack a cross-site request for a jury if you have a Web interface. Also, not having to go get the nonce first before they process the request. It's half the processing time they need. There's a lot more processing ahead. If you just need to go start slamming some Web page and trying to see what combination works, that goes really fast. If you've got to read the Web page, parse the Web page, get a value from the Web page, stick it into your thing and then process. It's incredibly more expensive to a hacker. And a lot of the just default hacker tools won't be able to do that. So you can protect yourself a lot just by using some sort of single use token if you're having to submit forms so people can't just try and just wail on your forms all day to find out what's actually broken and the one spot that you forgot about that gives them access. But the important thing for them is they have to be cryptographically random. This is a really kind of weird thing. It's cryptographically secure, pseudo-random number generators. That's a very big word that they shrink down to C-spring. But the reason for that is if most software, if you ask it to give you a random value, it will give you a random value. But it's based on the time that it gave it to you. And a smart attacker can actually recreate those and know what they're going to be before you generate them because of when they ask for it. So they can actually generate things that are going to come valid without having to actually use it validated or do any of those things because all computers there's no actual random number on a computer. It's just a list of values and where it starts and which one it picks is based on information that it has. And the fastest way to do that is to take the current time turn that into an integer find the number that starts there and that is the number that you pick for your pseudo-random number. Cryptographically secure as additional information, the current temperature of the CPU, whatever information that it has, bus speed, possibly the operating system version, it depends on the operating system and what it's going to do. But there's way more better ways to get something that is pseudo-random. Those sorts of things are random on a computer, it's all pseudo-random. But, so cryptographic libraries or dev-random, those are going to be cryptographically secure and you can count on those. And if you don't want to have to know which one it's going to be that's going to be secure on your version of Linux, you can always kind of cheat using one of the libraries that's going to give it to you. Because they would. And they should expire. So, in order for a nonce to work, it has to be associated with a certain amount of time or that it will be valid. Otherwise, you have to hold on to that value so they can't send it to you again forever. So, in your request or even in generation of your nonce, if you can mix it with some time, that will be super helpful because you don't have to hold on to it forever. If you just want to say, hey, my value that I'm sending is the time, the nonce, it's going to go away and if there's a way that you can validate that it hasn't been modified through a signature, it's a much smaller window that you have to hold on to the nonce. So replay denial of service. If you have a device that is a home security device, you don't want someone being able to load that device with so many requests and so much data that it can't actually send out an alarm when somebody breaks in. Right? If you have something that's monitoring the temperature of a room that has sensitive equipment, you don't want somebody to be able to jam that up and it doesn't tell you when the heat gets raised up to 200 degrees and someone fries out all of your equipment. Right? Bad actors will do bad things and they will find the simplest ways to do it. And a lot of times the simplest way to do it is to send an invalid request that you're going to process anyway. And that's what a denial of service is. And a replay is taking a valid request and playing it over and over and over and over again. So that you have to process it over and over and over again so that they don't have to think. The initial part of hacking is just, I'm going to just try and do something super easy and see if I can do bad things. And then if they can't do it then they have to evaluate whether or not they want to take the time to try and take you down. So the important thing is that you at least make it a little difficult for them. And there's a couple of different ways that you can do that fairly easily and that's either by tracking utilization and identifying things that are weird. If you have a system that has an endpoint that's perfectly valid to go configure something, it probably is not going to happen a hundred times in a second. So if you track that, oh, this same IP hit this same endpoint to update the system a hundred times in a second, you can cut them out because they're not doing normal behavior. The other things you can do which is a fantastic trick is called a honeypot. And so before I told you remove all these services from your system and don't give access to them. But sometimes what you do is you remove the services but you keep the ports open and you listen to those with your application because no one should ever connect to you at port 25. If an IP address connects you at point 25 you never listen to that IP address again. Because they're not connecting to your app they're just trying to do something terrible. So you can set up these nice things like if you listen on a non-standard web port, if you're able to do that then if they come to you on the normal web port then they're trying to attack you. Or if they try to connect over SMTP or they try to connect over SSAs or they try to connect over Telnet. There's scanners out there that will try all these things. They just try every port that's known a couple of the known off ports like 8080, 8000, 9443 they try all these standard things to see if you're attackable. And if you listen on those ports then you can determine they're bad actors. Now mitigating those there's a couple different ways that you can do it. One of the ways that you can do it is you can respond to them as if you had done something but that doesn't necessarily mean they're not going to keep hitting you. Now you're not spending any processing time you're just doing the initial very small check normally you'll make a hash, a SHA or an MD5 a very quick hash of the IP address so you can go look up in some memory table and say yep, this one's bad I'm not going to do anything with it, return 200. Or return whatever is valid for Telnet or whatever your application is doing. The other thing that you can do is you can just not respond to them at all and just lose the connection. So they're going to think that I got no response back from this port there's nothing there. So if you're using something that's doing asynchronous IO and I have to use mostly asynchronous IO you can then just end that. If you don't tell it to do anything with the response you can just end right there it's not going to take up any of your resources it's not going to take up any of your connections but they never get a response back they're going to time out and they're going to say there is nothing on this port so you can do that on your actual port if you see them come in at 443 then they come in on port 80 or they come in on 25 ports they're not supposed to be on then all of a sudden they come here and you say nope they went to port 25 they're a bad actor I'm not going to respond they don't think that you have anything exposed on the web there it's a really really cool way to just shut them out you can control the threads or some sort of asynchronous IO for you to be able to not lock up a connection until they time out by not responding Any questions about that? So this is something you would set up so if you have an application that's going to be accessible externally and so let's say you have a web admin for your device you would also listen on port 25 on your application that does this down there are a lot I know that my applications do that there is actually an entire honeypot there's actually some IoT stuff that's getting used for what's called a honeypot network which is used to just kind of throw them everywhere as a security company we actually put honeypots on every segment of our network to see who's coming in and identify them on the first scale but I don't know of any applications that will have source code that you could view offhand but basically it's just I listen to the port I just don't create a function for the response so it's going to update the IP tables but the issue you have there though is that you have a Python script that has root access you have something that's accepting incoming requests that now has root access so you have root access okay sorry you're right so they have net admin access which most of us that don't know the difference between net admin and root are probably going to set it up as root but again you're giving somebody you're giving a script that's exposed to the internet a very high level of access but that is actually one way to go about it and there are some stuff out there that's configurable that you can do that with I don't really suggest that when you can solve it in your application if you can't figure out how to do that it's definitely better than exposing everything on the planet I do that for IP tables but it's on the honeypot side I would say that you're going to listen in and just drop the response from there and understand who the actors are coming in right but then eventually you've got thousands and thousands of IPs in your IP tables and how does that slow your system down what's that? but then when it comes to port 80 you don't know that that's them again so with the honeypot system I'm going to take your request on 25 I'm not going to respond to it but I'm also not going to respond to you on port 80 because I know you're a bad actor so that's stopping a denial of service on your actual application IP tables you can absolutely just shut everybody off and port 25 or all this type of stuff but what you're on a honeypot what you're trying to do is you're trying to because they're just going to do a port scan anybody who's going to try and invade your system is going to start doing a port scan unless they know where your application is and they're doing a directed attack so the first thing you do is a port scan they try and find out what's available because they're not attacking your device specifically they're attacking a location or just an IP range and they're going in there and trying to find this particular thing to attack and if you expose these things out there they're going to say oh now I've got things I can go out later I've got this one, I've got SSH, so I'm going to go try that one I've got port 80 I'm going to go try these other things I'm going to start hitting with web attacks and trying to do SQL injection and all these crazy terrible things but a honeypot will allow you to kind of mitigate that before it starts happening it's very difficult to IP spoof and get a response back because the network doesn't know how to come back so you can only get so far going that way and that's a pretty sophisticated attack outside the network it's very doable but then again, you have to decide which one is that's a pretty sophisticated attack doing IP spoofing inside a network is a fairly sophisticated attack especially you first have to get in the network but again, it's one of the things where you have to kind of weigh it out is what's the likelihood that I'm going to have someone and do you care more about denial of an individual being able to connect to something or my system not being available so to prevent a DDoS she may not be able to configure me for a little while because somebody has done something but it doesn't mean the system has stopped right so it depends on what's more important to you and what you need to stop more does that make sense it's very very hard to actually stop any kind of DDoS so what you've told like little derivation will be like same attack I can do a handshake I don't have people complete the full going through inside the initialization of action that will shut the system down no matter what so the IP where I go most of them I can do a reflection attack I can keep creating a lot of different type of IPs and they will take you down there is no solution to but sin flooding is something you would stop at a network level you're not going to stop at a device level it's going to be a network level where you're going to stop sin flooding but just a handshake attack a famous thing attack that will go through again you can't stop everything but what you can stop is you can stop people from finding vulnerabilities on your port 80 if you don't allow them to get on the port 80 when they tried port 25 you can't stop everything there's a lot of things that you can stop and you can try and prevent things that people want to get at you and they find a vulnerability that's going to be there which is why you have defensive depth remote code execution don't execute unverified code if you don't know where it came from if you haven't validated that it's good we talked about that in the patch that you can put a signature there to make sure the code is valid don't run it and then if you're storing data in a database there's ways to verify that you're mitigating against SQL injection using a local database whether it's SQLite or you're using one of the variants for key value stores that you're doing for speed make sure that you're valid in your data make sure that you're not doing something terrible like on Node.js and having a local Mongo store that allows you that doesn't validate the difference between an integer and a string and creates a buffer that will just spit out people's data so be careful for those things that depend on the different ones Securements only secures what you stick in there you can't get at it once it's in there and that's going to be dependent on your operating system and how you're going to be able to execute inside of there and whether or not your virtual machine or your code executes in that environment what I've talked about mostly is what most people are doing is you're going to have to patch your code or you're going to get information you're going to be sticking in the database protect yourself against the SQL injection it's particular and different and it's going to be based on the vendor on how they've implemented it I know that there's some standards coming out around that but everybody seems to be just a little bit different so far at least in the stuff that I've seen so really quickly I want to give you some tools that will be some of these will be super helpful for you so we're talking about crypto and signature verification all this kind of stuff they have two standards that do that stuff for you one is called Hose and one is called Kose one is using JSON and one is using Seaworth which is the binary object representation we all care about small packet sizes and not having to move between text and binary so there are libraries written around these standards that will allow you to encrypt and sign data in transit and be super simple same thing with storing data locally the open web application security project this is used by websites everywhere it's also used by IoT they have top 10 vulnerabilities they have all the information you need to determine what your vulnerabilities are and how to actually fix them so if you think there's something I want to do go there bug bounties ok these are super awesome you can pay the best hackers on the planet to come check your site and tell you what's wrong with it the bad guys do in my company we've had one for 4 years we've paid out about $2,000 in serious bugs that were found in the securities in our systems and these are literally people who write the books they do it for a little bit of money a little bit of recognition part of their career path and they get up using ethical hackers is a great way to keep your stuff secure I really really really suggest it and there's some places out there like bug crowd where you can get that information and they will manage it for you or you can do it yourself so if anybody has any additional questions it looks like I'm running a little bit late and I apologize so you can just basically look up bug bounty ethical hacker there's bug crowd there's a few others hacker one, yes, hacker one is one of them like I said we run our own because we have security researchers and experts but if you don't have that you might have to get set up setting up your boundaries and what they're allowed to do what not allowed to do and get you in contact with them you might have to find out what Bitcoin is to be able to pay these individuals because they don't normally live in the United States so and if you have any more questions don't feel free to just grab me but I'm going to let her get set up so she can go because I'm running late thank you