 I think we can start with a warm applause for Peter. So thank you for all joining us here in the CDC Congress. It's the first time for me being here. My name is Peter, our PT. Today I'm going to speak about how auditing proprietary protocols met us today more than it meant probably a few years before. Because basically we had a walled garden approaches there, where you had serial lines you had to physically connect and talk to devices to. And nowadays everything's on the internet. And yeah, basically, it's IP-based. You can do your exploit from anywhere you want. So at first, I want to start with a saying from Claude Chen. And he said, the enemy knows the system. This is basically a phrase that symbolizes exactly what's happening here. You should not expect the others not to know what you're doing on your own protocol or on your own serial line. You've got something else. As an introduction, I will start with why I happened to hack a PBX and actually didn't want to. And furthermore, I'll explain the context in which this all happened. Then the next thing will be about reverse engineering protocols in general. This is just for those who have not seen it yet. And I'll show some actual results from the reverse engineering of this particular protocol for the PBX. Finally, I will make a strong point for responsible disclosure and working with the vendors fixing the problems we found. So first, a few words in beforehand. Don't laugh too loud about any bugs we will see here, because basically, if you're a developer, it can happen to you because you need to rush some product out of the door because of a trade show or something. And it's a real world example. Unfortunately, I'm not able to demo it here because of a NDA and still pending release of a fix. So basically, it's a little obfuscated, but in fact, it would work. So how many developers are here today working in telco or something? OK, two, three. OK, how many people of you do security auditing in some kind of way? Five, 10, 12, yeah, that's better. So OK. And so basically, why did I hack the PBX? Actually, I didn't want to. We were a contractor and did a job for a customer who had PBX integrated phones with a lot of functionality, lots of blinky lights on them, and all this stuff. And they can be customized to your demands in the enterprise. The client had more than 50 of those phones. And unfortunately, it took five minutes per phone each to read, modify, and write it. And it was interactive. You had no chance of automating it. So basically, they restructured. And actually, all phones had to be reconfigured. So their admin wasn't happy, you can guess. So we were asked to do a program that could batch, automate, all this updating of the phones. So it started reverse engineer protocol. But what do you need for that? You need the original software, for sure. You need some PBX hardware that's not the actual hardware used. So it's just a photo. And you need Wireshark. For those who do not Wireshark, go on the internet. It's the tool for reverse engineering protocols and looking on TCP or any kind of data stream and helping you out there. Then you need your brain. And actually, too much time of your customer at your hand, if the customer is willing to pay that just to batch automate updating phones, you're pretty lucky with that. So you need understanding of how the original software is supposed to work. So you poke around the interface. You click some buttons. You try to download, upload things, and get just a grip of how the people who made the software used to think. But you might find some gems like that one. It's actually a debug menu. It says debug, check properties, check V24 remote, and all this stuff. It's pretty neat to have something like this. This can help you reverse. This can help you reverse engineer a protocol quite a lot because you see most of the time what the packets are supposed to mean. That's a great thing. And try to think beyond the GUI, not only this GUI stuff, but really go into the binary, look at all kinds of data you see, and try to figure out how it works. You need to dump the communication. But what do you want to dump? You can just dump everything that is going over the ethernet. And hope you'll find out what's happening there. But it will be not so efficient in a way. So basically, the trick is that you define a test case, which you can repeat over and over again and have a clean way of reproducing it. So my test case was pretty simple. It was launch the software, click on load, click on from phone, select the phone you want to know what the data, enter your PBX password, and watch it download. Great stuff, simple. Then you enable the debug output and wire everything up. That's what actually I did. So I had the phone hooked up to the PBX, hooked up Wireshark on the computer running the PBX tool. The PBX tool was dumping its debug output in a file. Wireshark was dumping its own data into a file. It was me running the test case on the PBX tool and repeatedly sniffing the data on the network. And next thing is to file it cleanly or leave it. It's up to you. Next thing is to analyze what you grabbed from the data. So you basically have the advantage that you got a test case and you can correlate what the dump and the debug output say and try to figure out what happens when which packet is transferred at which point in time. So by the time you get a grip of the data flow, so basically you hit connect, you push the end of connect button, there is a hello with a reply from the PBX to the PBX. There's a query to the PBX. It's a reply from the PBX, all this stuff. Just try to make sense of what all those lots and lots of packets mean on the line. Then next thing is to look at the actual hex dumps of those packets. It's a real-world example. This is from the real thing I reversed. It was like, OK, let's have a look. The four here is similar here and there. It's four bytes here. It's, OK, it could be the length. And there is a 7-9 and 7-a. This is plus 1. The 2-1 and the 2-2 is incremented by 1. It's funny, because these are patterns you tend to look for and try to make sense of it. And also the 0-9 matches the length of the remaining bytes in that packet. So basically, yeah, you try to get a grip on how the protocol is supposed to work. Then you look for known data, because you control the program. You can inject any data you want, for example, enter wrong passwords as long as you want and have a look how the program tries to communicate them to the PBX or leaves it. So now I'm going to show you the results of what I found there. It was pretty interesting. As I told you before, the packet has a packet length field. It's the first byte in those packets shown here. And each packet from the client to the PBX triggers a response from the PBX to the client. That's great because you know when you set something right to the PBX, it will respond to you. Each packet type is the second byte in each package. It is incremented by 1 if it's positive for knowledge. So basically, when you set the right thing, the PBX will even respond to you in a way that you can figure out, yeah, I set the right thing to the PBX. It has a kind of virtual channels where you can communicate to subsystems of the PBX. You can open and close them at will. And you can open channels to up to 250 about something, device on the PBX at the same time. It is sometimes quite handy to have that. Unfortunately, it has an idle timeout, so you cannot cobble up your packets in your hex editor and wait five minutes and post data in a wait for the PBX to answer positively. No chance you got about half a second per packet. So you need to do a C or a Python or whatever code to do the communication with your PBX. Here are some packet types I found in the communication stream. It's a hello packet that basically introduces the client to the PBX. They exchange about their protocol versions, a read and VRAM and a write and VRAM, which enable you to read and write arbitrary data in the VRAM of the respective device you are talking to. There's a channel open and a channel close command for opening channels and closing channels to the subsystems. And there is an inquire hardware which enables you to look in the device tree what devices are below the device you're just talking to. And there is a ping just answered with the same data you send it. Note, every single packet type in the system applies to all devices. Not only the phone itself, but also the PBX. So basically you can read and write any part of the NVRAM of the PBX itself. You can explore the communication sequence as a next point. So you go there and have a look how many times what kind of packet is transferred on which channel. So I find out there's a hello, there's a read and VRAM which does something probably detecting what kind of hardware we're talking about. There's a channel open command and so on and so on. I got to explain this structure in the next slide a little deeper. So unfortunately there's one thing missing here and what do you think? Where is the authentication sequence? Wait, there is no? Actually there is a display asking you for entering a password. So you might ask yourself how come? I didn't enter, there is no kind of login packet transmitted each time you enter a right or wrong password because when you enter the wrong password nothing happens on the line. This is strange because you're supposed to see some kind of packets flowing back and forth between the PBX because the PBX should verify what you told it but now it's a trick. Where is the authentication gone? I'm gonna go through the sequence of communication compared to what my test case was and there will be a point where you could figure out where it happens, where the magic happens. So when you launch the software itself nothing happens on the Ethernet towards the PBX. When you click on load still nothing happens. When you click on from phone there is a hello packet to the PBX, there is a read and be around packet, there is a channel open, there is a lot of inquire hardware packets that explore which devices are present in the PBX. Afterwards the software will display a menu where you can select the phone from. This when you click on the phone you want to edit there is an inquire hardware packet and there is a read and VRAM packet and an inquire hardware packet then when you click on the phone afterwards you're prompted for a password. In that case I set it to zero, one, two, three, four, five in order to have something to compare with and something to find easily. Normally the passwords are a little more complex. So and finally you watch the download and there is a channel open command, it opens the channel to the phone and inquires its hardware, it reads its VRAM and close the channels and you're done. So basically still where the hell is the authentication gone? Yeah, there was a short read and VRAM, I go back to one slide, you see a short read and VRAM at step four when you selected the phone. I actually looked at that one and thought maybe it could be that there is some magic going on here. There is a short read and VRAM and it seems to read some by nature British. The software then shows you an authentication window. Wait, no, no, there is an authentication window asking you for a password. It as we remember was zero, one, two, three, four, five basically six characters long. There is a packet of a length of six going through the line, strange. So I thought, well, let's apply some evil hardcore crypto like SOAR and SOAR it the password with the data I saw here labeled as surprise. Basically it came up with B6 all the time. So basically use it as a key and don't, yeah. Basically the password was there but in fact the PBX told the software what the password was supposed to be and the software had to ask the user. So change the software, modify the software it doesn't ask anymore. In fact, no more authentication, a few seconds skipped. Well, not that good. Yeah, now we've got a problem because you can basically connect to the PBX, read and write anything you would like to do without even knowing a password or even worse you can say the PBX write your password and you've got a new one. So the story so far, what have we learned here? This kind of authentication is neither useful nor necessary. No privileged system has been implemented so basically you can go there and read and write everything on the PBX, even the firmware. There are read and write firmware commands. That's bad. And most of those commands looked to be quite useful when I would be debugging a device. These commands would be very helpful to me. So I think that this was probably a debug interface which was used as a interface for, yeah. Yeah, more than just debugging when they had to go to a trade show or something and needed to present something new they used that port and this protocol and just forgot about it. So probably it was a developers interface they just forgot in their own code. Everything else in this PBX is secured like something. They got a zip as they've got their own certificate authority running on the PBX exchanging crypto, it's very wonderfully done except for this one thing and it's a barn door wide open. So what could happen? You can read and write any phone and any data on the phone. You can reset the PBX password or even worse you can basically rewrite the firmware of the PBX. So for those who do not speak German this word, Stöhrung means error or failure. So you can bring the whole phone system down or even worse you can back it. You can snoop on data, everything you want to. So what happened? Now I contacted the vendor. So I rang them up said guys there seems to be a problem with your authentication. At first they said no way it can't be reuse SSL we use everything I said yes most of the time. Well next thing was being nice to them not saying yeah you're idiots you don't understand it I release it to the public and you'll see what you get from it. But in fact I was like okay I gonna send you an example run it on your PBX and yeah wonder what happened. And they called back and said okay thank you you helped us improve and they were quite nice. So except for saying I'm not supposed to name the manufacturer or the model of the PBX. That's yeah the problem but in fact it's all right because they are about to release a patch and everything will be good. For us as a community it's quite important to carry on and find more such bugs. Everyone looks at zip and HTTP and HTTPS and all this stuff which is standardized which is supposed to be on every PBX that's modern but in fact there are lots of legacy interfaces and proprietary interfaces that are even more buck prone and can lead to really nearly fatal things. What are lessons learned for us as a developer? You need to anchor authentication and encryption in your protocol basically this is what I tell you at a university or if you're doing a formal training for that you'll be just told use crypto use off but in fact sometimes it doesn't happen because they use the debugging interface and debugging interface supposed to be simple and virtually fail save they should work in every condition you can imagine in that device you need to get on there and have a look what's really happening but do not use those in production environments or even ship them wide open. You should audit your code base once in a while so you would have come across that your authentication routine in the administration tool was basically bogus but in fact it was 10 years in the repository and got yeah they forgot about it could happen but audit your code base once in a while you would it would turn up. I guess Shannon was right because bad guys I think knew that bug but actually it's not good because yeah it was in the world garden anymore where you can do your own roll your own protocol and nothing can happen and yeah vendors are happy to be informed about problems in their PB accent at least the good ones the bad ones are going to sue you that's not that funny. So yeah so now I've come to the end of my quite short talk today if there are any questions I'm happy to answer it. Was it too fast? There is one question over there. How are you going to ensure compliance because it's not exactly unheard of that the vendor will be super nice and say yeah we're patching this immediately and nothing happens for six years. Well that's a good point you know it's I think the most important point of responsible disclosure it's sometimes trusting them but in fact they cannot afford to have something come up in the bigger way so I don't think they'll need pressure but yeah you're right with a bigger vendors it can always happen so we got a good contact with them so basically I don't think it would happen because we're working closely with them. So pardon me. So why do you have an NDA? So you found the buckets your information so you would never sign an NDA if I say look I have a bug for you then it's a nice service if they say my bugs I found them but I wouldn't sign an NDA for what reason? You would get nothing from it. Well I needed the NDA because I still wanted to be able to use the software I created for batch automating the deployment when their new password system would be implemented. Well yeah that's a good question why I signed an NDA for software I wrote but in fact I want to get the information how the password exchange is done in the future from them and not have to reverse engineer it it's just a basic cost effectiveness thing. So there is another question over there. Oh and while he's running over there's one question from the internet a vision and L asks on IRC what consumers could do to do this auditing whether we should all use debuggers or have wire shark running all the time or if you would recommend to firewall every hardware device. Yeah firewalling every hardware device in the consumer environment at home isn't really a practical approach there but in fact in a corporate setting this is what I recommended the customer VLAN every device you've got on your network there will be a talk is there's a talk coming up today and it's about printer hacking and that a printer could even infiltrate your system so basically it's always a danger with all those embedded devices today in your networks you have no control over them. There was this other questions? Yeah so I was wondering did you try this kind of audit on other vendors or other machine or only on one PBX. I tried it on several types and they even sent me some others to test it on this was quite nice of them so yeah actually it worked on all of them without any flaw even on the news flexure product which this was kind of awkward because they said they used a new system on it and I was like I come on let's try it and I said oops. So it was only one vendor that was concerned but this by this problem so only one vendor. Pardon me? Yeah did you test only one vendor or several other brands of PBX? I used only one vendor this is vendor specific I think. Yeah so maybe some other vendors have the same kind of problems and then don't even know. Yeah in fact it could happen so basically I'm about to talk to them when they release the patch if I can release all the buckets details but until that happens there's unfortunately no chance of giving it out. One more question? Is that patched yet? No it's not patched yet they're about to ship it in mid January I think so and the good thing is they have a good system of deploying firmware updates to every device in the field so basically when they will release it it will be fixed quite soon. All right it is not clear to me yet what the attack vector is so what privileges do you need in order to be able to carry out the attacks in first place? Okay so basically you just need a way to TCP connect to your PBX no authentication no authorization nothing just it's an open TCP port you need it on your network and you're done. Just one more question from the internet are you planning to disclose the vendor as soon as the NDA runs out so he won't have the chance to just hide that kind of mistake in some footnote and release notes? Okay I didn't quite get the question. Are we going to know which vendor it was at some point in time in the future? I think they will make it they make their own point in their own release note I think I hope so. So any more questions or? Who wants to have the last question? Okay one last question. Well but still when would you be able to issue TCP packets to the PBX I mean you need to be an admin of that system in first place. Pardon me? Do you need to be an admin in order to to be able to issue TCP packets to the PBX? No you need you need to be on the same subnet basically or even when they expose it to the internet it's even worse than you can hop on there it's just an open port with no restrictions at all. Is that commonly done that PBX is all exposed to the internet? I don't know I'm just asking. Okay no it's I don't think so it's a it's a configuration port but it's open to the network the PBX is connected to in the default settings and I think you cannot restrict it at least not in their app interface. All right thanks. So thank you Peter for the session and