 So, we have two more first time speakers coming up here. We're going to give them the only piece of DEF CON merch that each and every one of you out here could earn, but you have to earn it. Not only do we give them shots, they get the shot, the noob shot glass that's our present to them for coming and giving a talk at DEF CON. So, help me welcome Zach and Alex to their first talk. Cheers. And their talk, infecting the embedded supply chain, I was telling them on the walk over here, I haven't read your abstract. But just from the title, I saw that and I'm like, yeah, we need to do better at that because I work on an embedded device and they're poking holes where we've got holes. All right, let me welcome Zach and Alex. Hello. Can you guys hear us? Hello, everybody. This is infecting the embedded supply chain. So, disclaimer, we thought we had a 45 minute slot, but we actually have 20 minutes, so we're going to try our best to condense everything within that 20 minutes. So, we have a couple of videos, which we'll try to do. We'll try to get to everything, but if we don't make it to everything, we'll post everything to our website, SummersetRecon.com, and we'll have proof of concept code on GitHub as well. So, you guys can check that out. I'm Zach. Like I said, I work at Summerset Recon. I do a combination of web application pen testing and reverse engineering. I specialize in vulnerability research and exploit development. I'm Alex. I'm a barista that occasionally does security things at Summerset Recon. I enjoy making cappuccinos, hardware hacking, and reverse engineering. Just a little bit about Summerset Recon. We are a security consultancy based out of San Diego. We specialize in web applications, mobile applications, and embedded device security. And you can find out more about us at SummersetRecon.com. And our Twitter. Cool. So, aside from our day-to-day work, we also like to do some cool side projects, one of which was an electronic safe lock that we looked at that allowed you to either enter in a pin manually to unlock a safe or, you know, over Bluetooth on your favorite mobile device. We found some vulnerabilities there, including being able to decode pins wirelessly over the air and replay them. And also, we reversed the wire protocol and created a brute force device for that. We also did work on the Hello Barbie, which is an IoT doll that kids can talk to and have a full conversation with. We looked at all the web services and reversed the mobile application and pretty much just figured out how that all worked. And found some vulnerabilities and made a white paper, which is online as well. So, these are all embedded devices, and that's primarily what we focus on on our side research. And so, when we decided to do this research, we wanted to do something slightly different, but touched on it still. So, we tried to look at what all these projects had in common. And we thought, why not look at their, you know, the tools used to program and debug them. So, this is our target. We focused on the SEGAR J-Link, which is a pretty popular device on both the hardware and the software used to control it. The SEGAR J-Link debug probe communicates with an embedded device that you're targeting and trying to program, like a mobile device, for instance, over a JTAG, SWD, etc. It's an in-circuit emulator and an in-circuit system programmer, so you could program chips. It primarily supports ARM and ARM Cortex chips, but also supports some others, like Renaissance. It communicates with a host computer over USB and Ethernet, which is kind of interesting, has a cross-platform tool chain and some other cool features. And the SEGAR J-Links are the most widely used line of debug probes available today, which is a quote from them. That's kind of cool. Other than hardware, they create a ton of software, including the software package that's used to control the J-Links themselves. They also have a GDB server, RTOS plug-in source development kit, a couple of real-time analysis visualization tools, and a graphical debugger. We primarily focused on the software suite used to control these J-Link devices, and this package just includes a ton of tools, including J-Link Commander, which is your typical command line tool that you would use with most debuggers, a GDB server, a remote server, which allows a client to connect to a J-Link commander session within the local network, a memory viewer, and a flashing tool to just flash. So a typical setup looks like this. You have a host PC that has a software suite installed, and you're communicating to this J-Link device over USB and ethernet, and then you connect it to your embedded target over JTAG or SWD. So in terms of attack surface, I mean, there's a lot. I mean, this company makes a lot of different products. The hardware debug probes themselves, we're interested in firmware and how that works and also can you infect the debug probe itself, or can the debug probe infect the end device that you're trying to target? And then in addition to that, there's a lot of software attack surface as well. There's a USB driver that's included with this device, as well as a host of different user mode applications that are run, and in addition to that, which is also a pretty interesting thing, they have a custom IDE that's also used to develop applications for the J-Link. So hardware. Yeah, so we're interested in how the hardware works, extracting firmware, how easy is it to do that? How does the firmware work? The feature differences between each of the devices, because Seger makes a ton of different types of J-Links at different price points, of course, and we're wondering like, are there hardware differences or is it just firmware? And with these devices, what are the security mechanisms that are on it? Is the device open? Can I flash an open source version of firmware, or can I modify firmware and flash the debug probe? So this is a typical J-Link EDU. It's kind of like the cheapest consumer end J-Link you can get. If we focus here, and open GIMP, you can see that it is also an ARM device. So the tool that's used to debug and program ARM embedded devices is also an ARM embedded device, which is, yeah, that's pretty nice. Then we started looking for debug ports and found a tag connect, which is kind of like, it's kind of like a J tag header, but a proprietary connector that has Pogo pins. So we had one of those lying around and we were asking, well, do we have anything that can program or debug this device? And we're like, why not try debugging a J-Link with a J-Link? So it turns out that the security flash bits are set, so that's good. The flash bit basically means that you can't pre-program the device unless you erase flash, which you can't do because the security bit is set, which basically says you can't debug this chip. So it refuses to connect and erase. We're looking for other ways around this, so we needed to go deeper. So we took a look at some of the other J-Link debug probes, and the J-Link Mini EDU has an interesting feature in its ARM chip, which is called the backdoor key access, which is an opt-in feature that basically, if you provide the chip with the proper key, it will willingly disable the security bit, and then you can kind of do whatever you want with the device, which is pretty cool. So then we began really looking at some of the desktop software that was distributed with the Seger J-Link for security vulnerabilities. We started out just doing some high-level reverse engineering. We noticed that they distributed their software to both Linux and Windows systems. In reverse engineering this, we determined that they were using cross-compiled code, so that eliminated some redundant work where only one of the operating systems versions really needed to be reverse engineered to determine the functionality and vulnerable portions of the code. We also noticed that they were using some custom string manipulation code, which was somewhat interesting, and we'll talk a little bit about that more later. We also noticed that they used a lot of dangerous functions. They used string copy, string cat, and a lot of other things that don't check the length of the destination buffer. From the point of binary protections, Seger opted into most binary protections. However, they did leave out both PIE and stack canaries in their Linux executables. So once we had that, we began setting up some fuzzers in order to test the different input vectors, see if we could find any vulnerabilities that we could begin to exploit. So we tested all input vectors that these applications seemed to accept. We tested files, both network interfaces and command line arguments that they accepted. Since our reversing revealed that there were some really interesting code paths that were somewhat deep and required magic numbers and different things to reach, we decided to use a generational fuzzing approach, so we used Peach for that. We fuzzed and got tons of crashes and began looking into actually exploiting them. So the first thing is the custom string formatting. We noticed in looking into this some interesting usages, such as shown here, where if it were a standard string formatting function, it would be a format string vulnerability. So we began looking into that a little bit more. So the custom string formatting that they implemented used most of the basic format specifiers, such as percent s, percent n, percent d, et cetera. But it did not accept the percent n that's typically used in format string vulnerabilities. So with that, we were able to do some of the format string exploits. We were able to do arbitrary reads, as shown here. However, due to the fact that it lacked the percent n specifier that's used to write bits, we were not able to turn this into an arbitrary write. We also found in the JLint commander tool, which we talked about earlier, it's a command line tool. There is a feature called command file, which basically you can feed JLint commander a script to kind of set up your environment before you start debugging your device. And this, the file parsing code had a vulnerability, which was a traditional stack buffer overflow. And so in the image, you can see that we kind of just feed a bunch of A's into a payload, and then we pass it into JLint EXE, and it just seg faults. So if you triage that in GDB, you can see that it tries to return to address 41, 41, 41, 41. So we went a little bit further with it, and we were able to create a POC with like five basic steps, which is take over the return address, get the address of libc, use that pointer to get the address of system and call system with arguments, and you got code execution. We won't be able to go over the POC because of time, but ultimately, we got code execution. It works on the 32-bit JLint EXE binary on i386 and AMD64 Linux systems, and it's a Rop with ASLR bypass, return to libc, and reverse shell is very doable, but requires roping in libc a lot. But the general idea is with this attack that an attacker can create a malicious command file and dummy firmware and send an urgent email with this as an attachment to a developer at a targeted company. Developers are typically pressed for time, so they'll probably run this with JLint commander, and it will pop a shell and return back to the attacker. We also found in the same binary, we found another file parsing vulnerability in the settings file flag, which is very similar to command file, but it's in a completely different area. It's in a library called jlinkarm.so, and that's where a lot of the ARM specific code is in for the JLint software suite, and this is in particular as a buffer overrun in the BSS segment, so you can use this to overwrite function pointers in the BSS segment. We also got RCE, which is cool. So after that, we started looking at a really interestingly named executable called JLint Remote Server. So once we saw this, we kind of perked up a little bit and began trying to look at the functionality of this executable. So running a basic net set, we noticed that it was listening on a number of ports, and particularly of interest to us was it looked like it was listening as a telnet server. So we wanted to figure out if this was actually having a telnet server embedded in the executable or what was going on there. So a little bit of reverse engineering revealed that they actually had embedded a telnet server within their executable. So it allows users within the network to connect to this JLint server and do remote debugging. So we began fuzzing this and noticed a really interesting crash where we had overwritten the instruction pointer with AAA, which is always good. We did a little bit of reverse engineering and triage on this and picked up a couple of things. It was a stack buffer overflow that we were able to trigger remotely. We were not able to get the crashes to be consistent due to race conditions within the executable. We had a fairly limited amount of stack space to work with. If we were going to build a Rop chain, we were limited to 48 bytes. And again, we had ASLR and NX enabled on this executable, but PIE was not enabled. So we used typical exploitation techniques. We used Rop to bypass the NX. And then with a Rop chain, we did a dereference of the got in order to leak the address of libc. From there, we could calculate the offset of system and then do a return to libc. The main issue that we ran into here was we wanted to send arbitrary attacker controlled payloads to system. And we had trouble kind of getting the user controlled strings to consistently be passed in the system. So we realized that user controlled data was being stored in one of two locations in the executable. And due to a race condition, we couldn't determine if it was going to be in one location or the other. So we tried to figure out if there was some way we could make it more consistent. And we came up with the following. So we called this space sleds, where we were inspired by the not sled techniques in which nots are propended to a shell code payload. And it improves exploit consistency. So here we prepend spaces to the user controlled command string. And we hope that that would improve the consistency of our exploit. So what we did here is since the two locations where we were seeing the attacker controlled payload string were 72 bytes apart, we prepended 72 spaces to our string. And that created an overlap where we could point our argument to system. So we have a demo where if we have enough time at the end of this, we'll be able to show. And then additionally, in looking into the Jlink remote server, we found some other really interesting functionality. So we termed this the tunnel server backdoor. And Sega has this as a documented feature within their application. And they said it just provides a tunneling mode from a remote computer into that your network allows remote debugging. Even when they're not within the local network. Correct. So our first thought at that point was I wonder what type of authentication they're using, if there's any weaknesses there or anything that we could exploit. So as you can kind of see here, they use magic numbers in your device serial number to authenticate your device. So that was a pretty scary thing. So when you run Jlink remote server with your Jlink device attached, it sends a magic number and registers your Jlink debugger with the server. And then it allows client connections to specify the serial number of the device that they hope to connect to. So we looked at the serial numbers and realized that there were 10 billion possibilities, which would make brute forcing all of the connected serial numbers a bit infeasible. So we wondered is this actually something that would be a potential attack? Is there some way that an attacker could shrink the space of serial numbers that they would attempt to brute force? So we began looking into is there some format that Seger uses for their serial numbers? Is there like some smaller range or number that they start at that we could use to shrink the space? So we googled Seger Jlink and found tons of images that people had posted online that included their device serial number. We called some people that we know that also had Seger J links and asked their device serial number. And in the end we got about 30 Jlink serial numbers that we were able to analyze and started to notice a few patterns that emerge. So the first two digits of the serial numbers were correlated with the model of device, whether you had a Jlink Pro or a Jlink Mini. After that, two digits for the version where if Seger would release a slightly updated version that would increment. And then finally the last five digits are a device unique number that's incremented. So doing a little bit of analysis on the numbers that we had, we were able to shrink the serial number space and still achieve what we think would be very good coverage with only 100,000 serial numbers. And assuming about 10 serial numbers per second could be brute forced which is about the link that we saw the connections taking with the, when it was done properly via the Jlink application, it would take the time from 31 years to brute force the space to about three hours. The impact here really is once you make a connection via this server, you can do all sorts of things. You can flash new firmware, you could read existing, or read the existing firmware, or make any sort of malicious modifications to the connected devices. So we disclosed the Seger and we're really pleased with the response. They responded right away and within a week. And it was very, we even received a thank you from the founder and CTO of the company. So in conclusion, we just really realized that these devices are oftentimes lacking in a lot of the protections and security that we would hope from a device piece of the supply chain. It was lacking the PIE flag which would have made our exploitation much more difficult as well as having lacking the security features that are just architectural and no authentication in the remote server or anything like that. Cool. But wait, there's more. Okay. Before we run out of time, we revisited the Jlink hardware, but instead from the perspective of trying to manipulate the firmware with a hardware debugging device from the OTA process, we realized that if you download a newer version of the Jlink software suite, Jlink commander will ask you if you want to update your Jlink debug probe. And so we started from the network perspective and found that there was no firmware blobs being transferred over the network and we thought that was kind of interesting. So we eventually figured out where firmware was being stored and how the update process worked. And we reversed the USB protocol to be able to flash the Jlink devices. And it turns out that the Jlink device, when it's being flashed over USB, it does do a firmware check, but it's not very good. And it uses dates to check whether our firmware is valid. So this could be a bad thing. Also, firmware is not signed and can be modified. So in the end, we don't have time for the video, but to illustrate it, we created a piece of malware that runs on Windows. It runs silently in the background and just any Jlink device that is connected via USB immediately flashes and completely bricks. By flashing firmware that's set way far in the future and you can't fix your device unless you have hardware debugging privileges.