 All right, we're good to go. Thanks. Hi everyone, I'm Greg. I'm giving a talk today on Linux IoT botnet wars and the lack of security hardening. Just a fun slide to get us started. It'll encompass a very important part of our talk. So my name's Greg. I'm currently working on an interesting project called Mender.io. It's an over-the-air updater for Linux-embedded devices, fully open source, and we're using the dual AB root FS layout scheme, currently under active development. So my original talk was supposed to be 45 minutes, and I originally included three botnets. But I'm going to talk about Mariah, which was very popular in the media because it was a very large-scale attack. I'll skip over Hajima. I might mention a bit about it, and then I'll talk about Brickerbot. All three of these botnets had some very common security problems. They were targeting IoT devices with very common and similar security problems. We'll talk about that. And finally, we'll look in solution designs of how we could have mitigated these problems. This is my first FOS stem. I wasn't sure about target audience. I'm pretty sure all of you guys know what this are. But if anyone's unfamiliar, DDoS is essentially distributed down off service, traditionally used to overwhelm a system and prevent it from working correctly. You have IoT. I was buying a heater recently, and it connected to Wi-Fi. Basically, billions of devices now are connecting to the internet. Botnets, I'm sure everyone knows what this is. But just in case, they're used to carry out distributed attacks. I might be using malware, worm, botnet, interchangeably. But this is a definition. So I'm just going to look into the basic anatomy of an attack that you tend to see. So you have the reconnaissance, which is you're going to discover a vulnerability, and you're going to find a way to exploit it. You need an intrusion method to get that initial access into a system, followed by inserting a backdoor. You want to do this in case you get found out and you want to be persistent in reinstalling your worm or bot on the device. Finally, you need to avoid detection somehow. Most worms, botnets, malware figure out the way to prevent the user from understanding that it exists in the system. So I'm going to talk a lot about Mariah because it's the most popular IoT botnet that causes quite a bit of damage. And a lot of interesting research has been done on this botnet. So it was discovered in August 2016 by a group called Malware Must Die that did a pretty nice analysis on it. One of their honeypots detected it. Mariah means future in Japanese, which is very scary. If the IoT industry is going to follow this idea. And it's infected. The numbers are pretty skewed. So there was some estimates around 200, 300 thousands, but some variants of Mariah have targeted around 1 million devices. So you could think these numbers as a grain of salt. It's quite a skewed a range of numbers that were infected, but a lot were infected. The type of devices that were infected is this is a good example. In one of the source code, they actually mention model numbers and models of specific IoT devices that were targeted. This one right here I found on Aliexpress, selling for $62. A lot of the Lua devices were specifically targeted. So the DDOS that were carried out by the Mariah botnets were extremely large scale. We'll look at Krebson security. If you're not familiar about Krebson security, he's a very popular blogger that writes a lot about malware, DDOS attacks. When this bot was discovered, he wrote a really long blog post describing the botnet and also was able to find out with very good confidence who the authors of this botnet was. In response, a couple of days later, they DDOSed them with 600 gigabits per second. What's really interesting is that Akamai was hosting his website already because he was oftenly targeted with DDOS attacks. And it was the largest DDOS attack Akamai ever witnessed. And what they did is they actually pulled a plug on his blog, but the Google Project Shield stepped in and said, we'll host your blog from now on. They have competing technology, a CDN that they promised to prevent DDOS attacks or are able to hold against them. Next is OVH. They were targeted with one terabit per second DDOS attack. What's crazy about OVH was when I was doing the research for this talk is they get DDOSed around 1,200 times a day, which is crazy. And the DDOS attack that stood out by far was this DDOS attack. We'll talk more of why they were targeted. Next was Dyna DNS. They provide DNS hosting solutions for all the major IT companies. I remember specifically one evening when I was trying to browse Reddit and the pages stopped loading, restarted my router, ran dig to see what was going on, and DNS resolution was failing. The next day I found out that one of the largest DDOS attacks ever was performed against Dyna DNS. So the malware has an update functionality where they were able to add new functionality to their botnet. It was only using DDOS attacks. It used around six different methods of carrying out the DDOS attack. And for some reason, the authors of Mariah decided it was a good reason to share their botnet source code online. The authors of Mariah were pretty interesting people. They thought that maybe if they would share the source code online that they would reflect responsibility onto others. So the way the Mariah botnet works is very primitive. You'll see a lot of things here are very primitive. So what it does is it just scans the internet looking for port 23 or port 2323, which is just plain text telnet. It attempts to brute force login with 10 random usernames from a hard list of 62. So if it determines that it could connect to this IoT device, here we're connecting to a video camera. And it determines that you could log in with Admin Admin. It'll talk back to its report server and let it know that this IP address you could log in with Admin Admin. Next, the loader program. I find this quite interesting. Keep in mind that IoT products are running a whole bunch of different architectures. So when you connect a via telnet to one of these devices, you're not sure if it's MIPS. You're not sure if it's ARM and what variant of ARM it is. So via telnet, you have to detect what kind of architecture this device is running. And then one out of the eight downloaders or loaders of the botnet will be installed onto this device and install the Mariah botnet and infect the device. It uses some pretty basic obfuscation, nothing new, randomizing the process name, so it's harder to figure out if something weird is going on in your device. It deletes its own executable, so it doesn't survive reboots. But that's not a problem. In the first 72 hours, I believe 11,000 devices were infected, and these numbers were doubling every hour. So when you have that big of a botnet being reinfected happens very quickly. And most IoT devices don't get rebooted that often to begin with. It deleted or removed competitive services. So telnet was disabled. SSH was disabled, so you wouldn't be able to gain access to the device once it was infected. Then it just continues being a botnet, connects to its command and control server, and listens to who it'll attack next. And of course, it just continues scanning for more victims and growing the botnet. So a quick recap. It targets a whole bunch of IoT Linux embedded devices. Around 30 vendors were targeted, but a lot of vendors have multiple devices, so it's not only limited to 30 different devices. So it was very efficient at spreading via brute force over telnet, and it scanned the whole internet. Something interesting that the authors thought to do was not scan Department of Defense in the US, the US Postal Service, and other American entities. For some reason, they thought if they didn't scan these guys, maybe we won't get the attention of the authorities, but then they carried out the largest DDoS attack ever. So yeah. And of course, it uses an extremely primitive attack vector, username, password, and all these IoT devices. All have very simple combinations. So I'm going to just talk about the motive, which is also very interesting. The motive behind Mariah was money and Minecraft. While reading about Mariah, I learned that if you run a Minecraft server, you can make up to $100,000 a month somehow. And these guys had a pretty cool business idea. So what they did is they set up a company and promised DDoS proof Minecraft hosting. And in order to get people to come to their hosting platform and host Minecraft servers, they DDoS everyone. So that's specifically why they attacked OVH. OVH is one of the largest hosting companies that hosts Minecraft servers. And they thought that if they attacked DynadNS, they would somehow bring a bunch of Minecraft servers down, and everyone would come to their hosting company, and they'll make a lot of money. Of course, that resulted in their arrests because there was a pretty big attack, and they used no OPSEC and wrote all kinds of things on hacker form online, which is crazy. So they caused the largest internet outage in recent history, and the largest DDoS attack ever recorded. One variant of Mariah was capable of infecting 900,000 Dutch telecom modems. And of course, it highlights a new threat. The trivial is or you're used to seeing botnets infect servers or desktop computers, but now you're seeing that these Linux devices that are connected to the internet that are being infected. And if the future is going to have billions of them, something has to be done. So Brickerbot, everything here is in the name. I'm sure you could figure it out what it does. So it was discovered in March 2016. The author said he was really inspired by Mariah. And very recently, in the end of December, he wrote an update to his manifesto saying that his project was over. He claims to have bricked 10 million devices using his botnet and wrote a rant that IoT security is non-existent. And to prove it, he destroyed a whole bunch of devices. So he carried out the per minute denial of service, which essentially bricks your device. I'm not sure if you could see those commands on the right-hand side, but essentially what the bot did was once it connected, it just destroyed your device by running random data to the memory and secondary storage and then deleting the route to the internet and then rebooting your device. So essentially, you're left with a brick. Again, very trivial, a trivial attack vector, scanning the internet for port 23. It also used other techniques, but same idea. Root forcing, telnet, SSH, and HTTP authentication. Something that was interesting is this author actually used a zero-day that was eventually found out that affected some modems. And like I said earlier, basically deletes everything, disable your network, reboot, brick. One thing that's very interesting is so since this botnet destroys your device, you don't really have a botnet. What you have is a central attacking command center and the way this worked is it was around an estimated 10 IPs that were hiding behind Tor exit nodes that were scanning the internet, looking for devices and breaking them. So there was really not a botnet here. So the author wrote the manifesto. He updated it and had more things to say very recently in the end of December of 2017. So you could see what he says here. Large attacks would force the industry to finally get attack together. Again, should have mentioned this earlier, but it's mostly busy box, Linux devices, SSH, telnet, hiding behind 10 IPs cannot spread because it bricks and username password. In India, there were 60,000 modems that were actually bricked because of this bot and one of its zero days, I believe. So a summary of the botnets. I skipped out on Hajima, but I'll give you guys a chance to read this very quickly and I'll give you a quick summary of how Hajima was somewhere like Miriah and somewhere like Brickbot, Brickerbot somewhere in the middle. So Hajima was very much like Miriah, but it didn't brick devices. Like Brickerbot and Miriah, it didn't brick devices. All it did was connect to devices and get rid of telnet and basically get rid of Miriah's oxygen supply. There was no attack code in any of the reverse engineering of Hajima and it wasn't decentralized, it was decentralized in nature, which was very interesting. So the author of this botnet actually created a fully decentralized botnet using BitTorrent protocols. So he used a distributed hash table to talk to all the bots and he used UTP, UDP over TCP for transferring data amongst this botnet. He's never pulled out an attack. He claims to be a vigilante that's trying to just kill Miriah's oxygen supply by killing telnet. So very interesting botnets happening in the IoT world. So why are IoT devices targeted? So the embedded world apparently has the ship it forget it mindset where once you ship your device you basically basically forget about it. Ease of use, coolness of a product is always cooler for some people and lacking security is the side effect of this. So some people don't want to think of what their default credentials are and just try admin, admin and too lazy to look at a manual or a nature device to figure out what the randomly generated credentials are. And finally, a lot of these devices were extremely cheap, unbranded and sold by some random companies. If you're selling a very cheap DVR equipment for $62 and you have no idea who the company is there's no real identity to protect. Finally, they make great targets. They're sold in very large numbers and they're all identical. Find the vulnerability in one, you own them all. So again, you could see that the attack vectors were similar. Another interesting thing is as soon as you log in you get a root login shell. So you could do whatever you want. So how do we get device manufacturers to actually create devices that aren't so easily hackable? We're looking at vulnerabilities that we've seen in 1990s. This is like a Windows machine or Windows 1998. There's no security in any of these products and everything could have been remedied very simply. Device manufacturers should be held accountable. This is not always easy because if your device manufacturer is somewhere in China and you know nothing about who's actually manufacturing device it's not easy to bring them to court and say your IoT botnet causes all this damage. One good idea is that the IoT Cybersecurity Improvement Act is hopefully coming. So we have a major player U.S. government that's trying to pass some legislation. What they wanna do is have a bill that has some basic principles that you have to follow when designing an IoT device like being able to update it, signed artifacts and whatnot. So in case you find a vulnerability in it or there's a design flaw you can easily fix the problem by shipping an update. Or maybe we need more brick-or-bots that just destroy all the devices that are easily hacked. So as we looked at the anatomy of the typical attack you'll see here that reconnaissance was just distributed in fast-port scanning and looking for especially telnet. Some of these botnets leveraged other things like SSH but telnet was the most successful attack. Intrusion, very basic using a password list and some of them used an exploit. One of them was a CWMP exploit. It is some routing protocol I'm not really familiar with that had some remote code execution that they were able to leverage and gain access to the device. Inserting the backdoor just requires detecting the environment, downloading and running the binary. One thing that I didn't mention earlier is the way they detect the environment that you're running on you have to figure out if you're running ARM or MIPS or whatnot. I believe Hajima's method was pretty interesting. What it did is it looked at the first few bytes of the echo command and using that it was able to determine what architecture this system was built for and then download the loader. Finally, process name obfuscation and removing binaries. Very typical things that viruses and malware do. Fixing these problems was again easy. These devices were listening to telnet and SSH on your public interface. You probably don't have to do that. You could probably just close these ports and have some other way of accessing these devices. Next, for the last decade, if you look at a modem or router, some of them usually come with a random generated password. So you don't just guess admin-admin to gain access. And finally, principally is privilege. You don't have to give someone a root shell with default credentials and they could carry out anything they could want to do. I'm gonna quickly talk a little bit about an over the air updater, which is possible to mitigate all these problems. So essentially, if you have an IOT device out in the market and you find out that either your design is bad or a new vulnerability is found and your devices are all hackable, you need some sort of mechanism to update your device. There's quite a few update mechanisms, Mender being one of them, and you would be able to address all these issues by just updating the device and getting rid of this design flaw or vulnerability. And over there, the updater means that essentially, it's not the user that's taking care of the update process. It could be managed remotely. So some of these devices that were attacked were very much like the firmware updating that you do in your router. You log into a panel, go to a website, download a firmware and manually pick and update the firmware. So you're just passing the burden onto the user. And one thing you have to keep in mind when downloading a over-the-year update tool is it's not always easy to make a homebrew one and leave it up to another company so they can take care of design, like what happens if you get power lost during an update process. You don't want to be like a brick or bot and destroy your device. You want some way of being atomic and do some sort of automated rollback if the firmware update fails. Then of course, you don't want to be able to do any sort of man in the middle attack and modify the firmware while you're passing it over and updating your device. So you need, of course, crypto. And finally, you want to make sure that the owner of the device are the ones that are supplying the updates so not anyone could flash your device with any firmware they want. This is the last slide. Not much, just something to think about. I'm open to questions. No questions? We have a few more minutes so we can take a few questions. Yeah, if you have any questions about my slides, please. If you have questions about OTA Updaters like Mender, feel free. I just wonder if any of these buttons are targeting devices from IPv6. Because I noticed that if I put a device in IPv6 network, I don't see any steps. That's a good question. Do you need any questions? Oh yeah, sorry, sorry. So the question was if devices work, if a device is connected via IPv6. I don't recall seeing anything specifically about this, but if I'm going to guess, I mean with IPv6 adoption rate, what is it right now? Like 20% less than your attack vector would be reduced significantly. But it would be of course nice to have as an advantage. But I'm not sure about that. Yeah. Have you mentioned closing, default closing thoughts and reaching it some other way? Do you like to expand a little bit on this? So I know, oh yeah, sorry, sorry. If there was a way to not open a port on your public interface and use some other way of connecting to it. I know some D-Link, I think it's D-Link or some other manufacturer has some URL that you could use to connect to, to connect to your device. Like some sort of reverse HTTP proxy where they allow you to connect to the device by not directly connecting to the interface. That's one technique you use. I'm not sure if that's the best idea, but some routers have this kind of technology. But it is a hard problem to solve as well. There's a question over here. Oh, sorry. Are you aware of any attacks where, I mean not standard, like guessing a password attacks have been used apart from the ones you mentioned. How likely is it that these things happened? So the question was, besides the telnet and brute forcing, what other attacks were used? So the most successful attack was the default telnet. SSH was also pretty good and HTTP. But I mentioned earlier in one of my slides, a routing protocol that some of these routers were using that had a zero day that was being leveraged in Brickerbot. So the author knew of some zero day, used it and checkpoint quickly found out that Brickerbot was using something that hasn't been disclosed before. So it's not only telnet, but telnet was very successful and extremely primitive.