 this is Jacob Baines and he's going to talk to you about ASAs and whether you should trust them or not. All right. Hello everyone. Welcome to my talk. Do not trust the ASA Trojans. That's fast. Now you might immediately be asking yourself what is an ASA? And these are ASA. These are Cisco adaptive security appliance. At the top you see the original ASA released some 15 years ago. That was followed by the release of the ASAX which was followed more recently by the release of the ASAX with fire power services. Now the ASA actually comes in a few other form factors including a virtual appliance known as ASAV. And these are sort of ASA as well despite their names because they can run the operating system known as ASA software. That is of course the same operating system used by the previously mentioned ASA models. Now an ASA typically sits at the edge of your corporate network where it can be a firewall, a VPN, an IPS, and or a router possibly all wrapped into one. The ASA is a critical asset because it acts as the gateway between the internet and your corporate network. And it also implements a variety of access controls and protections. This talk is called Do Not Trust the ASA because we're going to use features and vulnerabilities affecting the ASA in order to get root shells on the ASA itself as well as shells on the administrative systems that connect to it. Now all the ASA models we discussed earlier can be managed by this thick client known as Adaptive Security Device Manager or ASDM. ASDM is installed on an administrator's window system so that they can remotely connect to their ASA or ASAs and perform administrative tasks like updating firewall rules, adding VPN users, or simply monitoring the router's status. So for the next 20 slides or so, it is important to understand how the ASDM client and the ASA communicate. One of the first things that happens is the ASDM client makes an HTTP request for a pdm.sgz file. This file is hosted on the Cisco ASA's web server and the pdm.sgz is downloaded by the client and unpacked. The sgz format is a non-standard one, but regardless, but the client finds inside is a whole bunch of Java classes. The client will load those into memory, execute them, and that establishes the full administrative session with the ASA. Now the fact that the sgz file contains a whole lot of the client's functionality will be really important in a bit. So what's interesting about this is while the communication happens over SSL, the ASDM client never verifies the ASA's server certificate. So which means that a man in the middle like hacker cat here can monitor or even modify the communication between the ASDM client and the ASA itself. Essentially this should allow hacker cat to take full control of the ASA as long as they are able to establish this man in the middle position. Now this isn't theoretical. Pictured here is a screenshot of me using the popular tool man in the middle proxy on the ASDM client. I use the default man in the middle proxy certificate and the ASDM client gave no indication that it was under attack. Like I said, this should give me the attacker full control over the ASA. But I might be able to gain access to the administrator system as well. Recall that this pdm.sgz file is full of Java classes. If the man in the middle can introduce malicious Java to the sgz, then maybe the ASDM client will execute the malicious code. To explore that I wrote this sgz parser called getchu. It can extract all the files from an sgz file and drop them a disk. On the right here you can see a recent release of the sgz file containing more than 13,000 Java classes. Importantly, it also contains a signature file. Now this file contains valid cryptographic signatures for all of the files in the sgz. So if the ASDM client verifies the files in the sgz against the signature file, then an attacker shouldn't be able to introduce malicious Java. So here is some of the decompiled logic from the ASDM client. Highlighted here we can see that the client does actually check the validity of each file. And when it encounters an invalid entry, it will log that failure to a log file. But as we can see here, the client will still load the unsigned class files. The client knows it's loading an unsigned class, so it attempts to use what they're calling the unsigned protection domain. The unsigned protection domain is just like the signed protection domain, except the all permission has been removed. It just so happens that in this piece of software that doesn't have any significant effect for the attacker, a malicious Java class can more or less do anything to the system. Thank you. So this actually has been an ongoing issue with the ASDM. Last summer Cisco issued this advisory in the signs CVE 2021-1585. Essentially, this advisory says a man in the middle can inject arbitrary Java into an SGZ file and gain execution on an administrator system. Now, Cisco didn't release a patch with this advisory. In fact, they didn't try to patch it until June 2022. And unfortunately that patch wasn't effective. I'm told Cisco released a new patch for this vulnerability a couple of days ago. I was at Black Hat. So this issue might be cleaned up now, although I haven't had a chance to verify that. But it did take more than a year after initial disclosure for Cisco to address this. Now, a man in the middle attack is difficult. It's a difficult position for an attacker to achieve. So it's useful to note that 1585 is actually useful for evil endpoints as well. And what I mean by evil endpoint is that if hacker cat can trick an administrator to connect to an ASDM, connect to a endpoint in their control, then the hacker cat can return an SGZ file with malicious content and the ASDM client will load that into memory, execute it, and then the hacker cat will have code execution on the administrator system. Again, that's not theoretical. I've actually written two exploits for 1585, one of which is a Metasploit module. And now this talk emphasizes real exploits, particularly using Metasploit to hammer home that these are viable attacks that could be pulled off by low-skilled attackers. And these attacks should be taken very seriously. Now obviously this man in the middle, the man in the middle aspect of this attack is much more difficult when the administrator doesn't connect to the ASA over the internet. Hacker cat is going to have a hard time establishing a man in middle position within CorpNet, so the administrator is probably safe in this configuration. Now as a researcher, I want to find a way to attack the administrator on CorpNet as well. I figured I'd try to just modify the SGZ file on the ASA itself. Now remember the SGZ is hosted on the router's web server, so I figured I might have some type of right access there. But I needed to find how the SGZ file got on the ASA in the first place. And the answer is that it gets added to the ASA when the ASDM binary package is loaded on the system. Now the package is available on Cisco's website as pictured here, but it's also loaded by default onto some of their ASA, such as ASAV, and the test 5506X that we purchased also had it preloaded. But regardless, this SGZ file is first introduced to the ASA's web server when the ASDM binary blob is loaded on the ASA. And so I went hunting for the SGZ file in the ASDM binary package. Again, this is a non-standard format, so I had to do a little bit of work, but it turned out to be pretty simple. The binary package breaks down into three parts, a general header, a manifest area, and then all the files are catted together at the end. I was also looking for security features in the binary package, and basically I was looking for evidence that Cisco was signing the package. So I did find this hash field in the header, so my question was, is this a security feature or is it just a checksum to ensure the file's integrity? It turned out to just be a checksum, which might not sound like a big deal to a lay person, but it really is. Because there's no Cisco signature on this binary package, anyone is able to craft their own arbitrary ASDM package, which is a big deal because it means we can upload arbitrary SGZ files amongst other files to the ASA, which in theory should allow us to attack the administrator on CorpNet. We reported this to Cisco in February of this year, and Cisco released this advisory without a patch in June. Again, I believe a patch was released a couple of days ago, but I haven't gotten a chance to verify that. But either way, this advisory says exactly what we just discussed. Attackers can upload malicious ASDM packages to the ASA, which can result in code execution on hosts connecting to the router. So we're able to achieve what we set out to do. Hacker Cat has a malicious ASDM package hosted on the ASA. The only question that remains is how exactly does Hacker Cat craft this malicious package? Now, there are actually a bunch of files in this binary package, and there are a variety of attack vectors Hacker Cat can pursue with these files that all result in code execution. So I'm going to highlight some of the files in the package and what they do. The package contains HTML and JavaScript that make up the ASDM web portal on the ASA. Now, these are obviously good vectors for phishing or browser exploitation, but they're not the best. The ASDM binary package contains ASDM client installers. There's an MSI for Windows and a DMG for Mac. And these MSI used to be unsigned until I told Cisco that probably wasn't great, and they started signing the MSI in June of this year. But implanting your own installers is obviously a very good attack vector. The package contains all the files for launching ASDM via Webstart. Cisco just deprecated that functionality in June, so it's not that great of an attack vector anymore, but it's obviously a place that could lead to code execution. But as we've seen, the SGZ file is a really good vector, and it originates in the ASDM binary package, so we might as well use it. So let's discuss a tool that will help Hacker Cat build malicious ASDM binary packages. And this is a tool I wrote to do this, and it's called the WAY. And the WAY has three major functions. It can partially extract ASDM packages to disk, which is really useful for examining and modifying the contents of a package. It can also rebuild extracted ASDM packages. So for example, in this slide, I modify the ASDM web portal's index.html to say hello defcon. Then I use the WAY to rebuild a valid ASDM package. And when the new package is loaded on the ASA and we browse to the ASDM web portal that's hosted on the router, then we'll see hello defcon when we hit that landing page. And finally, the WAY can generate straight up malicious ASDM packages. These packages contain an SGZ file that will generate a reverse shell to an IP import of the attacker's choosing whenever an ASDM client connects to the ASA. So pictured here, you can see both the command line for the WAY, and I diagram how the exploitation actually works. And this is how the attack looks in reality. You can see the ASDM launchers up and running, and a reverse shell has been sent to my Ubuntu box. So by crafting a malicious ASDM package, we can exploit ASDM clients, not only connecting from the internet, but from corp net as well. The only challenge is installing our malicious ASDM package. It does require elevated privileges to install this package on the ASA. So that's certainly a limiting factor. But, of course, attackers can find ways to get required credentials, or they can use an inside attacker or trick an administrator into installing a malicious package. Those are all plausible vectors. But my favorite vector is a supply chain attack. When I bought our test 5506x with firepower services, it arrived with an ASDM package already loaded. I had no way of knowing if that was a valid Cisco ASDM package or not. The reseller could have planted a malicious package on my device, and they could have gained code execution in my home network when I started using ASDM on the ASA. As we've seen, this isn't a theoretical threat. It's a viable and demonstratable attack that I've done correctly would actually leave the victim none the wiser. What's sort of interesting about a supply chain attack is that the ASDM software is actually updated separately from the ASA software, meaning updating the operating system won't update ASDM. And I got to wondering how often people actually update ASDM. Basically if no one is updating the software, our implant lasts forever, and any bugs that we find in this software last forever. So I wrote a little tool to do some scanning of the ASA using ASDM on the internet. And here are the results. Basically no one updates the ASDM package, which is great for attackers, and less good for defenders. At the time of scanning, 7.18.1 was the most recent version of ASDM, and fewer than half of a percent of hosts were using that version. And we can see here the most popular version on the internet was released in 2017. So that's it for ASDM hackery. The rest of this talk will focus on a particular model of ASA called ASAX with firepower services. And in this first section we'll get a root shell on the system over HTTP. Now as a reminder, these are the ASAX with firepower services. They're the latest hardware model to use the ASA name, and they're admittedly getting a bit long in the tooth, but they're still in support and used globally. Now the name firepower services actually describes a special feature on that ASA model. In particular it describes this oval at the bottom labeled ASA firepower module deep packet inspection. And what that firepower module actually is, is a virtual machine running snort. Essentially firepower services is an IPS that's installed directly on the ASA itself. It's a pretty nifty feature. And from this diagram we see that incoming traffic is diverted through this virtual machine for analysis. Now you can access the firepower virtual machine via the Cisco CLI. The command session SFR console will drop you into a telnet session, which prompts for creds. And once authenticated you'll have this apparently limited firepower module shell. Except it isn't all that limited. By executing the expert command you'll drop into a bashell on the VM. And from there you can actually pseudo route using the previously used telnet credentials. And this route shell as a feature isn't limited to terminal connections. You can also use it via SSH. So anyone with SSH access to the ASA can grab a route shell in this virtual machine. Now that's particularly noteworthy because it's a virtual machine when configured has network access. Meaning it can communicate with the inside network that the ASA is supposed to be protecting. And it can communicate with the outside network or the internet. An attacker that grabs a route shell on this VM can install arbitrary software, persists through reboots and upgrades, pivot attacks inwards, exfiltrate data out. And perhaps most interestingly to me, just chill out and sniff the traffic flowing through the virtual machine. And it's unlikely that anyone is actually monitoring this virtual machine for malicious behavior. This type of, it's a really attractive place for an attacker to land and hide. And this type of route shell is actually something most vendors attempt to prevent. But it just so happens to be a feature of this model of ASA. Now Cisco obviously knows that this is dangerous because they have this lock down sensor command that can disable the expert command and thus access to the route shell. But I'm not sure if this is actually used. And our next exploit will bypass it anyways. So I was investigating how I can land in the route shell via HTTP. And it turned out ASDM can talk to the firepower module and generate these pretty graphs. But ASDM can't access the route shell. If I use ASDM to issue the session SFR console command, then the ASA basically reapplies. You can't do interactive shells over HTTP, so get out of here. So I started messing around with some command injection vectors. And here's one that was actually successful. I issued the command session SFR do back tick ID back tick, where ID is a Linux command. And the Linux command will tell us who we executed the command as. So we can see that the ASA responded to the request with invalid do command, you would equal zero or root. Meaning we successfully executed the command as root within the spiritual machine. Which results in this scenario. An attacker over the internet can achieve a route shell on the firepower module virtual machine by sending a rather simple command injection exploit. And the exploit is really simple. It's so simple you can fit it in a tweet. So pictured here you can see a tweetable version of the exploit. You see I use the curl utility bash dev TZP and net cat to throw the exploit and catch a reverse shell from the ASA. Again, this is great for an attacker. They now have their own luscious VM on the ASA to pivot inwards with an actual trait from. Now Cisco did release an advisory for this, but only after it was determined that it was a bypass for the lock down sensor command. Cisco has released patches for most but not all ASAX with firepower services. And their advisory makes a sort of big deal that the attack requires ASDM credentials. Which is true, but those credentials might be easier to come by than you think. So I'm going to list a few ways you might come by ASDM credentials. First recall that the ASDM client is vulnerable to man and middle attacks. So additionally by default ASDM client authenticates the ASA using HTTP basic auth. Which means a man in the middle can trivially extract valid ASDM credentials from any HTTP request originating from the ASDM client. So there's one way. It's also worth noting that the default credentials for the ASDM interface are blank blank. You don't have to take my word from that. This slide is taken directly from Cisco's ASDM book one. And of course we confirm this is true on our own 5506X. And it also turns out the ASDM client was logging credentials to the client log for a long time. So I wrote this metasploit module that will scan through ASDM log files and pull out valid credentials. So that's yet another way. And finally the ASDM web interface doesn't have brute force protection enabled by default. You have to go in and enable this account lockout feature. So the only thing that protects that interface by default is an ASDM specific user agent. One version of which I've highlighted here. Without this user agent the ASA ignores the inbound ASDM request. So that is at least some type of protection. But I ended up writing a metasploit module that will brute force this ASDM interface anyways. So that's another way to get credentials. And I know a lot of people don't like brute force attacks because they aren't very cool. They certainly aren't elegant. We've seen a number of APT like GRU use brute force attacks at scale with great success. And I'd suggest if it's good enough for GRU, it's good enough for me and you. So with credentials in hand I also wrote a metasploit module for the command injection issue over HTTP. Here you can see we provide IPs and credentials, throw the exploit and catch a reverse shell yet again. And this is sort of only semi related but I was messing around with this ASDM interface and figured out how to dump the running configuration over HTTP. And I actually think this shouldn't be possible. I don't think ASDM can do this. It did involve some Cisco CLI hackery. But anyways, if you do come along ASDM credentials, that's just another way to dump more creds. So we're going to switch our focus to abusing the installation of the firepower module. And nothing in this, the remainder of this talk is actually considered a vulnerability by Cisco. But I'll let you be your own judge on that matter. Now interestingly, the firepower module is an add on package kind of like the ASDM package. And it also has to be installed. The ASAX works totally fine without it. And if the user doesn't use the IPS feature, then likely they'll not want to install it in the first place. So a scenario where hacker cat has SSH access from the internet, but the firepower module isn't installed isn't that far fetched. But without the firepower module installed, hacker cat can't access that special root shell. Or can he? So let's see how hacker cat can get that shell by using the firepower boot image. Now the firepower boot image, the firepower module installation process is really involved. It takes three different steps. The first you have to install a boot image, which is labeled number one here. Then you have to install an install package, which is which is labeled number two. And finally, you have to install an upgrade package, which is not even on this page and not numbered. But we're talking about phase one here installing and using the boot image. The boot image needs to be downloaded from Cisco and copied on to the ASA, or an image might already be present on your system, as was the case with our test device when we purchased it. Either way, once the image is on the ASA, you issue two commands to boot it. The first command lets the ASA know this is the boot image that you want to use. And the second command actually boots it. You can then issue the session SFR console command to open up a telnet connection to the boot image and you'll be prompted for creds. Pictured here is the default credentials for the various versions of the boot image. But note that the default user is always admin. So once authenticated, you'll drop down into this shell on the boot image. Now this is actually an extremely limited shell. It's really only useful for installing stage two, the install package. It doesn't have anything like the expert command we saw previously. So let's actually take one step backwards. And instead log in as this undocumented user, root with a password Cisco one two three. And bam, you got a root shell. And what's great about that is the boot image is networks too. So using hard coded creds on the boot image gives hacker cat that special root shell once again. Now Cisco said this is not a vulnerability because there are no security expectations during the firepower installation process. They had the same response when we reported a command injection issue in the boot image. This pictured injection is in the logic for installing part two of the firepower module. I just want to emphasize the boot image contains no vulnerabilities. But vulnerability or not, the hard coded creds are dead useful. And I've written a couple of exploits that automate the uploading and configuration needed to take advantage of this non vulnerability. And both exploits end by catching a reverse shell. The meta-sploit version is a bit better because it catches a reverse interpreter shell. Now Cisco did remove the hard coded credentials from a recent version of the boot image. But since the ASA doesn't prevent you from using older boot images, I can't say it's a particularly useful mitigation. In my mind, it's kind of a forever day. Now the problem with the previous attack is it assumed SSH access, which is obviously not going to typically be available for an attacker. So I want to figure out some attacks that assumed no access to the device. I want to modify firepower install packages to include malicious code and then get unwitting victims to install those malicious packages. So again, let's visit the boot image. So this is the scenario I'm visiting. Hacker Cat is totally on the outside, has no access to the ASA. It's just sitting out there in the cold all alone. So to help Hacker Cat get back to his root shell, I started looking at the format, the boot image. And it turns out it's just a totally generic bootable Linux ISO. Like, it's just a live CD. You can actually execute it with VMware Fusion or whatever virtual machine system you like. There's really nothing Cisco specific about the boot image's format. So I figured, well, why don't I just create my own live CD, my own bootable Linux ISO, give that to an administrator, get them to install it, then the ASA should boot the malicious ISO. Because my operating theory was that Cisco provided firepower boot image was just a normal live CD or whatever. And usually stories like this have a lot more to them. What you tried, what failed, but the funny thing is, this just works. I grabbed an existing script for creating tiny core Linux bootable ISOs. I added in a few features, the ability to play Doom, a reverse shell, you know, the essentials. And this script, which I rebranded as pinch me, generates a malicious ISO that the ASA will happily boot and assign IP addresses to. There's no trial and error because the ASA doesn't attempt to block this behavior. I guess booting arbitrary virtual machines on the ASAX is just a feature. But again, this isn't a vulnerability because there's no security expectations on the boot image. So we can craft our own boot images, distribute them as we'd like, and once installed, have special access to this root shell once again. And we can play Doom ASCII. Now this is sort of cheating because it's running on the virtual machine, but I'm forever going to say that I ran Doom on the ASAX with fire power services. One downside is that I didn't find a way to drop into Doom via the session SFR console because there's some type of kernel module involved. So you have to use this SSH session that I set up. But it's good enough for a hack. But the problem with just hacking, just attacking the system with a boot image is that it doesn't persist through reboots. So I started looking at the install package for more sustained attack. Now remember, the install package is part two of the fire power module installation. The install package is installed by and overwrites the boot image. So I was trying to figure out the format of the boot image. The format the boot image expects the install package to be in. So pictured here is Python taken directly from the boot image. And you can see that expects the install package to be written in the encrypted content signed to check some package wrapper format. It's a pretty secure format as far as I can tell. And somewhat importantly, it's the only format that I think Cisco has ever published the install package in, at least dating back to 2016. And it looks like this. This isn't super important to know. Just note that it starts with this keyword key very early on into the file. But it also turned out that the boot image has this if clause for the fire power module. And that if clause allows for a second install package format. And this format is called checksum package wrapper. And it's actually an utterly insecure format. There's no Cisco signature on this one and it only has checksums like we saw with the ASTM package. Which means in theory we should be able to craft one of these checksum install packages and the administrator should be able to install it. Resulting in root shell access once again. And after some tedious reverse engineering, I did figure out that format. Again, it's not important. Just note that the file no longer starts with a key keyword. It goes straight into data. Now the problem is that we actually need the install package to do its job. Namely install a whole bunch of Cisco stuff and then overwrite the boot image. Otherwise the install process will fail. So it isn't enough to know this insecure format. We need the insecure checksum package to also contain all the Cisco stuff as well. So that installation will actually succeed. So I wrote a tool that does that. This tool is called what's up. It takes in a valid and signed installation package, unpackages it, inserts malicious code, and then repackages it into this insecure format, ready to be installed by an unwitting victim. Now the malicious code is just this init script. Basically it tries to connect to an IP and port the attacker's choosing every five minutes. And this init script will survive reboots and even upgrades. So it's a pretty useful little attack. All you need is to get someone to install it. But remember, this is still not a bone or ability. But the result, again, if installed as an attacker cat is back in the root shell. And this time with persistence. Now actually describe this to Cisco as a potential supply chain attack as well. But they disagreed. They pointed out that the fire power module has a root shell feature. And a vendor could simply insert arbitrary code into the virtual machine using that root shell. So no malicious installation package required. Which I agree is 100% true. But to me that's a separate and also concerning supply chain attack. But I wasn't able to win them over to my point of view. So that's all the hacks I have for you. In this talk we discussed man in the middle problems, credential leaks, code signing issues, package signing issues, root shell as a feature, hard coded credentials for a root shell, command injection for root access and executing arbitrary bootable ISO. Many of these make the ASA a perfect little Trojan horse. And having said all that, let's take a quick look at some indicators and possible mitigations. The number one thing I hope is taken away from this talk is that this can never be done. Never can you use ASDM over the internet without potentially risking your ASA. The man in the middle issue that we talked about, to my knowledge, is actually not slated to be fixed. I would actually encourage you to stop using ASDM altogether and disable that feature on your ASA. I've written a few YARA rules to help identify some of these attacks we've talked about. One rule will detect malicious ASDM packages and another detects unsigned install packages. The other two will look through the ASDM log files for credentials or signs of malicious SGZ usage. Now ideally I'd love to give patching guidance, however there are slew of issues in this talk that are not considered vulnerabilities or unpatched or their patch status isn't clear as of today. When patching isn't an option, we usually apply mitigating controls, which is isolate and limit access. And that's not easy because the ASA is a critical system in a given network. At very least, like I said, I'd suggest disabling ASDM and ensure that you're auditing who's logging in to your system. Now we saw a number of password issues in this talk, so rotating passwords might be a good idea. But finally, when it comes to the ASAX with firepower services, I actually think this root shell feature is far too dangerous, even without all the packaging issues that we discussed. So I'd suggest retiring and replacing those devices. And until you do that, you should spend some time looking at the root shell on the device to ensure you know what's going on in there. So that's actually it. All the code I mentioned, and a lot more actually, is up on GitHub. And if you like this talk, you can find me on social media at the normal places. That's it. Thank you.