 Okay, so hello. Good morning, everyone. Welcome. My name is Krzysztof Pasiak. I work for a small Korean company called Samsung. You may have heard about that. It manufactures a couple of different stuff to the market, some consumer electronics, some tanks, some missile launchers, some ships for transferring heavy goods and all other stuff. So today I would like to tell you a few words about some techniques and open source and open hardware tools that could be used to debug USB related issues. So in the beginning I will give some brief introduction to the USB protocol just to ensure that we are all on the same page. Then we will go through USB sniffing and some modification of the USB traffic and then I will talk about some USB security related stuff, fuzzing techniques and things like that. In the end some short summary in Q&A session. So talking about the USB protocol, there is quite a lot of myths in the industry. People believe that there are some dragons living inside the USB because it's not very easy to sniff the USB traffic and today I would like to make some clarifications and ensure you that there is no dragons and nothing bad is going to happen to you if you start sniffing the USB traffic. So what USB is about? We are all familiar with the Internet. So I believe that USB can be easily compared to the Internet. Just like the Internet is about providing and using a service. Just the service is a little bit different and the design and communication form is a little bit different. So a single USB device may provide a one or more functionalities to the USB host. The USB host is a client that use this functionality and this is the machine where usually which is usually used by the end user, so the human, right? A single USB host may have multiple USB devices connected but a single USB device may be connected only to a single host at the time. If you are talking about providing multiple services, but by the same device, it means that we need some kind of abstractions like we have in Internet, which is called ports. In USB, this the same type of abstraction is called endpoints. So a single USB device may have up to 31 endpoints, including endpoint zero and this endpoint zero is the only one which is mandatory and the only one which can transfer the data in both directions. If you have port in the Internet world, usually you have a bidirectional link. In USB world, to realize a bidirectional link, you need two endpoints because all the endpoints are unidirectional apart from endpoint zero. The direction of the endpoint is always defined from the host perspective. So if we are talking about in endpoints, it means that they are capable of transferring the data from device to the host and when we are talking about out endpoints, they are capable of transferring the data from host to the device. In Internet world, you usually just choose between TCP and the UDP depending on your use case. In USB, you have four types of endpoint to choose from. The first one is control. It is reserved for endpoint zero. It is used during enumeration, which is a process of discovering a new USB device. It can be also later used by the application, but due to some bandwidth constraints, it's better to use all other type of the endpoints for the real transfers and to use eventually endpoint zero only for some out-of-bounds data. The next one, which seems to be the most common one, is a bulk. It is used to transfer a large amount of data, which is non-time-sensitive because we have no delay guarantee. It does not reserve any bandwidth on the bus. It just uses whatever is left over. The next one is interrupt. It is used to transfer a small amount of time-sensitive data with the delivery guarantee. And the last one is used to transfer a large amount of time-sensitive data but without the guarantee of the delivery. So, as always, there is no freelance. If you know what kind of endpoints we can use, now we may need to consider how they are composed into USB devices. So, in the Internet world, you just have a server which has a certain number of ports open and if you would like to use some kind of service, you need to use Nmap or some other tool and knock to different ports and see if they are open or not. Because one of the most important design principles for the USB was plug-and-play, we need some method to discover the capabilities of the USB device after it is connected to the USB host. So, that's why every USB device has a logical structure which groups all the entities into some hierarchy which can be then discovered using USB descriptors. So, endpoints are grouped into interfaces. And interface is a group of endpoints that are used to implement some well-defined, well-sunboxed functionality. Interfaces are grouped into configurations and configuration is a group of interfaces that can be used in the same time. Single USB device may have one or more configurations, but only one of them can be active at a particular time. As you can see, endpoint zero is kind of a special one because it is not grouped into any interface nor configuration and it is always available. So, this is the kind of default communication channel with every single USB device. Now, if we know what the USB device is and how it looks, maybe let's try to talk about the USB bus. So, USB bus is a host-controlled bus. It means that nothing of the bus can happen without host first initiating it. So, there is no device-to-device communication and the device cannot initiate any communication even to the host. Everything is based on pulling by the host. So, how does pulling look like at the link layer? As in every protocol, we have a couple of different layers. We've got the physical layer, we've got the link layer and the protocol layer in the USB. So, at the link layer, the in-transfer, so it means that host would like to get some data from the device, starts with sending the in-token to the device and then the device can make a choice. If it has the data ready, it can instantly respond with a data packet. If it doesn't, then it simply responds with a knack. And the host, if it receives a knack, will retry to get this data from the device. If we are talking about the out-transfer, so we are trying to transfer the data from host to the device, then it starts with sending the out-token to the device and sending instantly a data packet to the device. And if the device was ready to receive that data, it will simply reply with a knack. If it wasn't, it will reply with a knack. If host gets a knack for an out-transaction, it will try to repeat the transaction by sending the out-token and the data once again. As you can see, this is kind of inefficient because you have to realize that the out-token is something really short. But the data, so out-token is like 8 bits and the data is like hundreds of bits, like 500 bits or something like that. So it means that it's quite a lot of data, so it takes a lot of bus time to send this to the device. So in order to save the bandwidth, we have a mechanism called pink and not yet. So instead of sending the data packet once again, you just send the pink and depending on the device response, you either send the pink once again or you send the proper data if the device is ready to receive it. So what we have seen on the two previous slides, it was a USB transaction. So it was sending a single portion of data to the device or receiving it from the device. Because using that kind of low-level concept in all USB drivers is not something very comfortable for the developers. We've got a more higher level concept called USB transfer. So USB transfer is a set of consecutive transactions that are ended up with a short transaction. It means the transaction which transfers less data than it could. So all the drivers that you see in the Linux kernel or LeapUSB and things like that, they all operate at the transfer level, so they don't care about any low-level details. The low-level details are either implemented inside the hardware or they are implemented in host controller. The structure which describes the USB transfer in the Linux kernel is called the USB request block. So it's a kind of envelope for the data that drivers just fill in and schedule to be sent by the USB controller. So what you can see inside the URB structure is just a pointer to the device to which we would like to send the data, address of the endpoint that we would like to use, obviously the buffer, size of the buffer and the completion routine. So it means that in general all USB drivers are asynchronous. So you just put the data to the buffer, schedule it to be sent and you are waiting just for a completion of that task. So how to sniff the USB bus? As you say, we've got a pretty nice structure for a pretty nice envelope for the USB transfers. So the first idea that people came up with how to easily sniff the USB traffic was a USB monitor. A USB monitor is a kernel module that is simply a kind of logger for URB-related events. It means that every time when any driver submits the URB, an event is being logged to the USB mode. Also when the USB transfer finished, independently if it finished successfully, if it finished with an error, this event is logged to the USB mode. Then you can try to read all those events from the user space and this way you get some information, some insights. What kind of transfers are happening between a driver on your Linux device and your USB device? An important thing to note here is that you are not going to see any link layer USB tokens because this is only a logger for the URBs, right? So you are seeing only things that are going through generic kernel structures and nothing more. Also another thing which you need to remember is that depending on the direction of the URB, you need to look in a different place for the data. When you have an outs transfer, it means that your driver is sending the data to the device. So you have to look for the URB submit events in the logger to check what kind of data has been sent to the device. If you are looking for intransfers, so you'd like to know what is the response of the device when the driver requested the data, you will need to look for a completion event. Fortunately, you don't need to use either binary interface nor text interface because we have our good old friend, Wireshark. So Wireshark supports USB monitor, so you can simply just load the kernel module, open the Wireshark and see all the USB traffic which is happening on the bus. So now let's try to do some demo. What I have here is just a console. I will just load USB mode module. Okay, after I loaded the USB mode module, a couple of USB mode device instances has been created because kernel creates a USB mode device for every USB bus that you have on your machine. So now if I open Wireshark, you can see that I have a couple of USB mode interfaces to choose from. In order to determine which interface you would like to sniff, it's always good to check your D-message. I will just disconnect and reconnect the device that we will be sniffing. And you see that the new device appeared at USB bus number one. Let me increase the font. So you can see new USB device found USB one dash something. The one is the number of the bus. Now let's start sniffing on the USB bus number one. And you see that there are some, still there are some transfer ongoing, even through I disconnected the device, because we are logging all the events for that particular bus. So if you have any other USB devices connected to that bus, you will get also all the data transfers here. So you will need to later filter those out and just listen for the particular and show the particular USB device that you are interested in. So now let's connect our USB device and you see that we had some events, right? I will also try to click on the presenter and we also have some more events. And now let's check what address did the new device have? Get. So it's a new full speed USB device number 12. So now let's go to here and just write USB dot device address equals 12. So what you can see here are all the transfer that happened between your host and the device that you have connected. You can see that we have some enumeration process in the beginning and then you have communication which is related to pressing the buttons on the presenter, right? And they are of the type interrupt because this is typical for human interface devices. What's interesting here is that for every URB that you have here, here we go. So you see that you have event which is called complete, right? So it's the moment when the device responded with the data and you have the leftover data to be parsed by some next level de-sector. If you would be sending the data from host to the device, you should look for this data inside the URB submit event. Okay, let's get back to the presentation. So this is a very simple method. It's free because it's both Wireshark is free and the USB monies just comes with your Linux kernel, but it has certain limitations. First of all, it's not going to show the link layer packets and link level tokens. And the second one is that you need to be sniffing the data actually on a host to which the USB device is connected. So you cannot be, for example, in between your PlayStation and some device that you are connecting to it. You need to be on the host that actually communicates with that device. So how to solve that? One of the ideas to solve that was to implement a proxy device. A proxy device can be a USB single board computer which has both host controller and the device controller. And that's how the USB proxy has been developed. In theory, it should work on any single board computer which has suitable hardware capabilities. In practice, as I was able to run it only on a big old bone black with a very specific modified Linux kernel for that particular use case. How it works? So basically, it uses LeapUSB to communicate with the USB device that you are sniffing, and then it uses GadgetFS to communicate with the USB host. So in between, you've got a whole user space framework and application for doing money in the middle on the USB. So you can both log the events, you can modify the transfer that are being sent between the host and the device, and you can also inject some additional packets. So for example, what I did here on the diagram, this is the diagram of a tool that I developed which allows you to do a money in the middle on the ADB. You know, ADB, which is used to develop applications on the Android. So using this tool, you can, for example, bypass the ADB feature called secure debugging because you first authorize some other host and then you intercept the traffic and, for example, inject a malware in between. So one of the things to note to notice here is that this is not a ready software. It's not something that you can take out of the box. It certainly needs a little bit of love. One of the ideas that people are thinking about when they are talking about sniffing the USB traffic is to just use a logic analyzer. Definitely you can do that, but only for low or full-speed USB communication. Because high-speed USB communication, the signaling speed is 480 megabits per second. So in order to fulfill the Shannon theory, you would need to probe that at 1 GHz frequency, which is not very common for cheap logic analyzers. But for full-speed and low-speed, definitely yes. You can use CROC and see all the diagrams and then parse that into the USB traffic. So to overcome the difficulty, OpenVisla has been developed. So OpenVisla is a really cheap USB analyzer. It's open hardware, it's available at GitHub. What it really is, is just a USB PHY that pars the physical layer of the USB and then through the ULPI interface pass all the data to the FPGA. This FPGA communicates with the SD RAM to cache the information and then the information is sent through FTDI chip to the host where you are actually grabbing the packets. The important feature of OpenVisla is that it is non-intrusive USB analyzer, which means that you cannot modify the packets, but also you can put that in the middle of any host and any device and still sniff the USB traffic without any of the sites knowing about that. There's a couple of tools for the host side. The firmware, which is used by different people, is mostly the same, but the tools for the host side vary from use case and from people who are using that. The default tool that is inside the OpenVisla repository is just a Python script that allows you to get the data as a console output. There is a viewSB, which is a very initial state of some open-source USB analyzer. I have put it here, but actually I don't believe that anyone would be willing to maintain a separate program just to show the USB traffic. And that's why, together with one of the students from my university, we decided to add support for OpenVisla to the Wireshark. And now I have developed also the XCAP interface for the Wireshark, so you can run the Wireshark and start capturing USB traffic from any device and also reuse the disectors that has been working for the USB MoM and sniff and parse all the traffic. So how it works. So basically, if you run the Wireshark, you will see that I have some additional interface, which is the XCAP, so it's a separate binary. And now if I start listening on that interface and reconnect the USB device, you will see that I get all the communication. I get both startup frames, which are low-level USB packets. I have also intokens, which are sent by the host, and I have also the response from the device. So I can sniff a very low-level USB-related issues, even if you are developing, for example, USB stack for some microcontroller, you can sniff all that without having the expensive hardware that is sold by many manufacturers. One of the interesting things is that you can also see only the USB packets. They're here as malformed due to some issues in the code. But you can see that the dissecting of the USB protocol is being passed to the upper-layer desectors. So if you connect some standard device, which has a disector, like ADB or MTP or something like that, then you will get all the traffic dissected to the right fields. Okay. So tools for USB security testing. Now, one of the issues for the security researchers is that they are using some very popular operating system whose name doesn't start with L, but it starts with some W. I don't know what kind of operating system it is. But they think that they have some issues and they don't know how to, for example, use UDCs. And the other thing is that Linux kernel stack does some, let's say, sanity checks for all the transfers and on the descriptors that are being provided by the USB stack in order to ensure that it is compliant with the standard. And if you would like to do some fuzzing of the USB stacks, then you need to generate invalid descriptors, right? So that's why they developed a simple board which consists of FTDI chip, microcontroller, and the Max USB chip. So what it really is is just a custom USB device which can act as, with custom USB stack, which allows you to malform any packet that is sent over the bus. One of the interesting things to notice is that the chip from the Max chip that is used here can be used to both act as a device and act as a host. So if you connect two of those devices to your computer, then you can do the same as the USB proxy and have the man in the middle without actually using a single board computer, custom kernel modules and stuff like that. Another board that has been developed initially for, as a platform, the base platform for hacking some radio stuff is the great fit. I have this board here. I have also the OpenVisla here, if someone would like to see it after the talk. And basically what it does is it has a microcontroller, right? And this microcontroller is equipped with two UDC controllers. One of them is a high-speed USB controller which can be connected to your host with any operating system. And the other one is a full-speed controller which can be connected to your target host. So basically this board is backward compatible with Face Dancer. So the whole Python framework that has been developed for fuzzing USB stacks with Face Dancer can be reused here. So you can make this board to act as any particular USB device, any particular vendor ID, product ID and stuff like that and do things like enumerating the drivers that are accessible on the target host, trying to exploit those drivers and things like that. Interesting thing related to that board is that there is a project called Great Fit Rododendron, which is a simple add-on board for the Great Fit, which contains the same USB fi as OpenVisla and allows you to capture the USB traffic in a non-intrusive way. But it has a certain limitation, so basically as you see this board, it doesn't have any SD RAM memory inside. So the thing that OpenVisla does pretty well is caching all the data before it is picked up by the host and on this board it is not possible. It has only a few kilobytes of RAM and that's all. A very interesting tool which has been developed for Face Dancer and the Great Fit because it's kind of backward compatible is UMAP. What UMAP allows you to do is to scan what kind of drivers are available on the host. It allows you to emulate any particular USB device also from Python. It allows you to sniff the traffic in the man in the middle setup or if you connect some USB device to your computer and then just pass through it to another computer. And it allows you to run some fuzzing using KITI framework to try to destroy the drivers on the target. One of the issues or limitations that are here is it is quite hard to generate a valid USB traffic that could give you some nice results. So that's why all the fuzzing of the USB stack is typically done by just catching the correct transmission of the data between host and the device and then just modifying it from that point. Another approach for fuzzing USB drivers is to use virtual machines. So there is a framework called VUSBF and some clones or variations around that by the idea is quite the same. So basically most of the hypervisors allow you to pass through the USB device from your host machine to the virtual machine. So we are trying to kind of abuse that functionality and instead of providing a real USB device we just provide a daemon which generates a USB traffic. So this way instead of having a physical device that you need to connect, that you need to maintain and stuff like that, you just start the virtual machines and then you have a better coverage and you faster get the results because you can execute those in parallel. Another approach to fuzzing USB drivers is the one which is used by Andrei Konovalov. He's kind of famous in Linux USB mailing lists because his CIScaller bot sends a lot of emails with bug reports so he get pretty nice results. The idea here is to use CIScaller which is a fuzzer for fuzzing kernel CIScaller interfaces and use dummy HCD and GadgetFS or custom USB fuzzer module to avoid those checks on the descriptors and USB traffic that I mentioned earlier. And just have a kind of... The dummy HCD is a kind of loopback for the USB so it is both UDC and host controller connected to the same device. So on the same system you can both emulate a Gadget and then have it connected as a USB device. So what we do here is we get some description files, we pass them to the CIScaller and CIScaller generates and fuzz the CIScalls and this way it generates the USB traffic which comes to the same machine. Okay, so that's all from the tools. Definitely there are some more tools in the white. If you'd like to chat about that, if I didn't mention something, feel free to catch me here. From my point of view, I believe that you don't need to spend a lot of money on some very expensive USB sniffers. You can just grab one of the open source project or open hardware projects depending on your use case. Obviously, there is no perfect architecture for testing the security of the USB stack. The importance of that is definitely increasing in recent years so I believe that in a couple of years we will see a lot of efforts happening or at least I hope that we will see a lot of efforts happening to harden our USB stacks because it's generally everywhere, right? In cars, in mobile phones, in laptops, so the attack surface is definitely huge. So that's all from my side. If you have any questions, I will be happy to answer. So from your experience, which solution would you recommend for sniffing USB traffic? And the other question is whether any of the presented solutions support sniffing USB 3.0? So no. None of the solutions support, none of the physical solutions support sniffing USB 3.0, but the USB mode can sniff any speed that you have because it's just a logger for URBs. So it will work perfectly fine. But all the hardware solutions that I presented, they can be used only to sniff the traffic. How expensive are they? So OpenVisla is around $100 and the grade feed is around $90. Fair enough. Yeah. So they're kind of similar in terms of price, but for grade feed you will need to buy also the Rododendron board. But I expect this one to be quite cheap because it's just a two-layer board with a single USB 3.3, 4.3 cheap, which costs, I know, $2 or $1 or something like that. Sniffing 2.0 can still be very useful. Yeah, definitely. What would you recommend? Which one? So as a starting point, as a starting point, I would definitely recommend using USB Mon because it's for free and it will show you most of the data, especially if you are dealing with some USB drivers. That's definitely a way to go. But if you are implementing USB stack for some microcontroller or things like that, I would recommend to go with the USB Visla because it is kind of ready and we are now in process of mainlining changes made by the student to the wire shark mainline. So the support should be available soon, just out of the box in the US version of the wire shark. Yeah. So with the USB Mon kernel module and the boards, is there a way to capture the data like remotely and then view it in wire shark later, kind of like TCP dump? Yeah, yeah, definitely. So you can use either text or binary interface to catch the data on some external system and then read it on your developer machine. One of the things to note is that for other operating system, there are similar interfaces. So if you are working on some ugly system, which name starts with W, there is also a dedicated solution for that. Okay, any more questions? Okay, if there is no more questions, then thank you for your attention and have a good lunch. Thank you.