 I didn't come here to fuck spiders. Let's get started. This is Switches, Get Stitches, and it's the third episode. You might want to know what the first two episodes were if you missed them. One was a workshop at 44Con. 44Con believed in the research very early and said why don't you bring it to a small number of people and do it as a workshop. Then I took it to Chaos Communication Congress and had a lot of fun there. And I wanted to get these guys to join me to bring in some different elements to the talk. And it's a talk focused on industrial ethernet switches and breaking the management plane of those switches. So we've got the Siemens Galance, the GE Multilink, and over there, the Open Gear. So that's what this is about and that's the previous episodes. So, who are we? I'll let you guys go first. So I'm Colin Cassidy. I've been a senior software engineer for 15 years and then gave that up because it was a horrific job and decided to break stuff for a living. Rob Lee, so I was one of the guys you probably hate. I was in the government recently, but I'm no longer. I got out two weeks ago, so I'm alive and a civilian again. And I'm a researcher in the industrial control system community as well. And I used to be a senior risk researcher at the Cambridge Center for Risk Studies until I started telling jokes at DEF CON. All right, so we expect you to be a DEF CON audience. We expect you to have a lot of fun. So there's two things I want you to make noise for in this presentation. One is private keys when you see them on the screen or we talk about them. And the other thing is when you hear a vendor patch time story that you like, I expect you to make some noise. So switches get stitches. Let's get started. The scalance, X family, so this is X200 switches, X400 switches, basically this is the one I worked on, right? They have session IDs in their web interface. And we all know that session IDs are a good place to get started, see if you can do some session hijacking. As we said before, all of the vulnerabilities that we're talking about today are in the management plane of the device. So when you go to the website of the switch to manage it, you get these session IDs. Now any of you who do a bit of reverse engineering, does anyone in here reverse engineer? Good. I'm glad that there's a few people in here who can teach me something. These vulnerabilities should be pretty obvious to you, right? You see the CO and the AA and you probably think to yourself that looks like a local IP address, right? But you'll also notice that they're strictly increasing. So these session IDs are increasing only. They're not as random as they could be. So if you don't mind flipping to the next, you can see that the ones on the right-hand side, if you pull the device every second, it will actually give you incremental session IDs. And when I compared that with an SNMP request for the uptime of the device, I discovered that it was indeed uptime in hex. And then I got confused because I checked the IP address at the front of the session ID and it wasn't the IP address of the switch, right? So I kind of thought about it for a little while and then went no, surely they didn't. And of course it was my machine that was connecting to the device. So client uptime and sequential session IDs based on time was one of the first vulnerabilities that we found here. Weak session IDs. And you can sort of see in the mind of the developer here some sort of madness, right? Like, I know I'll create all of the IP space and all of time and that will be impossible to ever estimate, right? But clearly it's a very weak session ID. I actually go back one and I'll say one other thing. The bit at the top is essentially how the passwords were hashed on the wire. So the admin at the beginning is obviously the user. Password is the clear text of the password. And then you take that session ID that I just talked about and you use that as a nonce. Now the nonce, the number used once, is passed back and forth. And the idea there is that you can't replay a hash from previously, right? So that also probably affected why they used the uptime. And there it is on the end. But you'll realize that by putting the password in the middle in between these two colons and then MD5ing it, it's not a very large space to have to brute force. So you can snatch one of these IDs off the wire, one of these hashes off the wire and brute force it fairly quickly. Like 16 characters I think took maybe 15 minutes. Something like eight characters would take a minute, right? And then on top of everything else, it was MD5. So continuing the theme, the open gearbox here also has a weak session ID. Essentially it's the number in red. Eight characters long. It's basically a hex decimal value. It is semi-random, it doesn't increment. But beyond doing any more analysis above that, we didn't really look into much further. However, there was a nice configuration item built straight into the switch that allows you to extend it. So this one's already fixed and it was fixed from the start. It's just not on by default. And this highlights some of the problems with these switches is that they're not always deployed in the most default, hardened state. Some of these reasons are to do with backwards compatibility. They've been deployed in the field with this previous state. And in the ICS world they tend to be very risk averse. If there's one less thing they can test and just do a quick swap, that makes their life so much easier. All right. So after finding these issues with the session IDs, you can do some side-jacking on a switch, both on the open gear or the Siemens scalance. But if you work regularly doing pen testing, you'll realize from an operational point of view this is not the greatest attack. Like, how often do you guys log into your switches? So I have to wait for that to occur, right? That's probably going to happen every couple months at best, maybe once a year. So we didn't like those attacks. We wanted to go for something a little bit more robust in terms of being able to use it any time. So we were focused on authentication bypass and particularly on firmware. If you go over to my GitHub, you can pull down this script. And basically this is a CSRF, right? But the CSRF is, it makes it possible for you to download a configuration file or download the log file of the device. That would just be a get. Or if you do a put, then you can put the firmware or you can put the configuration or the log file. And I find this kind of amazing, right? Because you can change the log file on a device before you've even broken into it. And this creates an authentication bypass in the sense that I can have a known, good configuration with a password that I know. And the passwords are sort of hashed in those files, by the way. And I take that known, good one. Well, before I use the known, good one, I download the configuration of the device. Right? So I have the passwords that used to be there. I upload my own configuration. I log in with my password, which might be finally waxed mustaches or something. And then do whatever I want with the device. And then re-upload the old config. And no one's the wiser that the password has ever been changed, right? So this forms an authentication bypass on the device. And it's also brilliant that you can just post a firmware image to the device. So Siemens have fixed this in newer versions. But if you do pen test in this environment or you have access to these switches, feel free to download the script and prove to yourself that those versions were vulnerable. I think this one's you. So I essentially got involved in this project about a month after joining my firm. I'm active. And Aaron said to me, I've been breaking these switches. Do you want to take a look? I said, okay, fair enough. He said, we've got the latest version of the firmware. Download it and take a peek and see what you can find. So the tool of choice more often than not is bin walk. It's a nice wee tool. It basically looks through an entire binary, looks for sort of headers of particular file types. In this case, we can see there's a large compressed file. And so you can use bin walk to essentially pull it apart. So taking the large compressed file, we can see the first four items there, closer. Is that better? Sorry. So we can see the first four items there relate to the certificate and private key of the Siemens Skellant. And indeed, there it is. So we reported this to the vendor as you do because, you know, we're nice like that. The vendor came back and says, yeah, but you can change it in the web interface. Okay, that's fine. We're in the documentation. Do you say you have this key that needs changing? Otherwise, that's just like an undocumented login password back door. Okay, we'll change the documentation. So self-sign certificate, it can be changed by the web interface. Nowhere was it in the documentation. So guess what? Vulnerabilities in these devices tend to be pretty common and pretty in the same style, in the same pattern, right? Default credentials, default keys. All of the OWASP top 10 are in there. And that's important actually because we like to do a bit of embedded and we like to do binary analysis and we're teaching ourselves to be better reverse engineers. I'm just looking at ECOS at the moment, trying to build flirt signatures and stuff. But we didn't want this talk to be like that. We wanted this talk to say, look, if you do web app pen testing, every embedded device now has an embedded web server. And some of those classic OWASP top 10 are enough to get you control over the device. So we could use your help if you want to come and look at embedded. Yeah, you need to learn some stuff about control systems and hang out with some engineers. But frankly, that's really good for you. They tell some of the best jokes, as you heard earlier. And in this particular device, I had to figure out how it was compressed. I carved out a firmware upgrade. Actually, it's worth backing up a little bit. Someone sent me a PCAP, right? There are various engineers who know that I work on this stuff and they'll send me sometimes anonymous letters, sometimes not anonymous letters. And one of them sent me a PCAP of the firmware upgrade of this device, right? And said, you know, here's this HTTBS traffic. But then once we do the firmware upgrade, it goes over FTP. And it uses the same credentials, right? And he was like, this is pretty bad, right? And I was like, yeah, actually, let me look at the rest, right? So I basically used TCP trace and I carved out all the files. And one of the files was much bigger than the other and it was over FTP, right? So now I have a copy of the firmware image just from the PCAP. And I looked through it and BinWalk didn't recognize a lot of it because I hadn't recompiled it and upgraded it and some other things. So I had to figure out that a section of it was compressed, which you can do using entropy and other stuff. And then I ran a little script over it to figure out where the compression lied and managed to decompress it and found a ROM image. And inside the ROM image, there are these private keys, right? And I know you guys like private keys, right? Everyone has a private key collection? Yeah. So these are, these private keys, the first one is for the HTTBS interface, which has just been ruined anyway because you've seen the credential go over in HT, sorry, in FTP, at least if you've ever witnessed a firmware upgrade. But if you don't witness a firmware upgrade, you can take the key from the ROM image that I just described and install it in Wireshark and you can read all the traffic. And the passwords underneath are unhashed. So you get those as well. And I think this is a really key, sorry for the pun there, that was entirely accidental. But I think this is a real key issue in the sense that once, if you only have one encrypted interface on a device and someone breaks that, you now don't have a secure channel in which to up-no-load new keys if attackers are persistent on your network. And I think that's an issue. The second key belongs to SSH, but couldn't be enabled on the device. And another researcher has since brute forced the key because I couldn't be bothered. So the password on that is Magnum 6K, which is a little clue to something else we'll talk about in a minute. Oh, I guess we're talking about it now. So it turns out that this switch is, you know, sold by GE, but it's manufactured by Garakom. They make a line of switches called Magnum 6K. So this key belongs to the Garakom switches in the same way as the GE key belongs to the GE switch, right? So you basically just take a different firmware, you analyze it again, the key has changed, but it's the same. And I should clarify that these particular keys affect seven out of nine of this switch family. So there's nine of them that are managed switches. Well, one of them is not managed, so therefore this stuff doesn't apply. Then seven are managed and the biggest one has a different firmware image. So, you know, just what, an hour or two of bin walk and fooling around a little bit, you pull these keys out and you get an authentication bypass for, you know, a couple thousand switches of a switch family. Just continuing a thread on that. Whilst we found these keys in a GE switch and we reported it as such, essentially it is a rebadged Garakom. And so when you see the reports coming out saying there's issues with GE switches, we don't know who else has rebadged these Garakom switches, so who else is affected? Is there a Siemens switch out there that's effectively a rebadged Garakom? And how does that essentially get out to the wider public? There's a sort of web of network vendors and rebadgers or resellers that it makes it hard to do proper sort of incident response and incident control in that regard. So flush with our success pulling keys off Siemens switches and GE switches, we moved on to the GE MDS-wise. We don't actually have this switch, instead we just pulled down the firmware. This sort of investigation sort of highlights the problem of not having this switch. What we found was not really problems. We went to GE about it and they said, no, they're not really problems but we'll go through the process of what we found anyway because it's interesting if nothing else. So binwalk being our friend, pulled it down is essentially a squash FS. Binwalk knows how to extract this stuff and it happens to be a nice essentially Linux system, file system at the end of it. There's a search directory so you can applaud that later. But there happens to be a .password file which we thought was very interesting and indeed it contains the passwords and hashes. So, you know, it's got the admin one, it's got the guest, it's got auth code which is used, if you forget the password you need to reset it, you ring up GE and say blah, blah, blah. It's got a factory password which was not documented and it's got a root password that I got bored of waiting for Jack the Ripper to break. So if anyone has a faster password cracking rig than me, take a note of it or find the firmware and go knock yourself out. We did report this to GE, the guys rang me back, I think I had one of their head product cert people and about their entire development staff and they said, yeah we looked at it, we don't believe it, doesn't I think. Now I don't have this device to test that so if anyone's got one I could borrow or can lay their hands on one and wants to find out, that would be ideal. But the other slightly bizarre thing is this is an industrial network switch, why is there a games user? Oh and they've got private keys too, yay! So yeah, key management in network equipment, you are going to find default keys all over the place. If they're undocumented that is bad, if they're unchangeable that is worse. We have not found unchangeable keys however other people looking at switches have. Self signed keys are more confusing issue, yes you have self signed keys but you lack the ability to revoke them but if you're in an isolated network environment that's fine because you shouldn't necessarily be connecting your industrial system to the rest of the world just so VeriSign can verify that you have the right certificate authority and key installed. No, so but if you've got an internal one you can probably get away with doing that sort of thing. These switches tend to lack the sort of processing power and any sort of entropy to create their own. But if you're going to set these things up you will plug them into your laptop and your laptop tends to have this sort of capability. So to the vendors out there maybe they need to consider a sort of initial step process that helps lock these things down from the start rather having one key that we can now talk to every switch on. That's not helpful. Essentially the problem with key management is you have to manage your keys. Can't just leave it to the vendors or whatever crazy mechanism of just leaving it there. All right, so you know let's get back to some web app phones instead of some you know scraping, gripping keys out. Actually this is a good story about that. There's a bad ass pentester called Ilya Vansprundel and I've spent a little bit of time with him and I once asked him you know he does code review. How do you find so many vulnerabilities? And he said I just gripped for them. And I thought you know what he's right like if I'm going to write one script for industrial control system devices that does something what would it be and it'd be like gripped for private keys and hex. And it works. So this switch also amusingly has a flash interface if anyone's really into flash. I didn't explore that. I just went looking for cross-site scripting and I hope that GE will forgive me for using that phrase in my proof of concept but I was listening to DJ Kubert and that was a sample at the moment that I found the vulnerability. So it just seemed appropriate. So there are eight types of cross-site scripting on the management plane of this device. They also apply to Garrettcom and you don't have to put them in specific parameters. I don't have to tell you which parameters to put them in because you can make up parameters for the web server to put your cross-site scripting in. Hello gentlemen. Hello. How are you? Very good. So we have a thing we do for people who have never spoken at DEFCON before. It is called shoot the noob. We won't actually shoot them. Don't worry. Jeffrey, I think they're familiar. How many of you are not familiar? One guy. So I'm talking directly to you. So sir, were you born yesterday? Have you been to DEFCON before? No. Okay. Is that your real name? Yes it is. Wait, what's your name? Ryan. Is that your real name? Yes it is. We don't use real names at DEFCON. Your hacker name is Arapastoli. So we're going to explain to Ryan if that is your real name that it's very hard to become a speaker at DEFCON. To get on this stage, you either have to be very smart or very stupid. That was hard. I'm sorry. I'm sorry man. I really am. That was uncalled for. Ryan, I wasn't talking about you. You're the smart guy. Anyway, so just a quick toast for our new speakers. Are they doing a good job? And Ryan is up here representing all the new attendees. So thank you to DEFCON. Thank you. Now you stuck your fucking thumb in it. The bear didn't get a shot. By the way, you guys got two minutes. Okay. So then we'll have to make that pretty quick. So we promised to come up here and drop private keys like DJ Jackalope drops breaks. I hope you've appreciated that. I think she's awesome. I come here to drink and dance. And this is an excuse to do that. But I do have to do a lot of hard work for it. So let's go to the next slide. So GE firmware integrity. Remember earlier, I didn't want to brute force that password that Ashish Campbell of QALIS brute forced. The reason I didn't want to do that is because I can patch in my own keys and firmware and then run a CRC Adler at the end and then try and reverse the algorithm at the bottom. So there's basically two checksums. One's code signing more than checksum. And if you patch in your own key with your own password, it turns out that just works and you don't have to do any brute forcing. So top tip there. So this is one of my favorite vulnerabilities. Okay. Colin once asked me why do you hack switches? And I said because that's where the packets are. You know paraphrasing, Dillinger, right? But also this is important because industrial system networks have a little bit of authentication and a little bit of crypto. But for the most part they don't, right? Most of these protocols, if you can put packets on the wire, you can tell it to do something. Occasionally you see something like a, they'll prevent a replay. But essentially if you understand the protocol well enough, you can just tell it to do what you want it to do. And you can even perform things like recording traffic and replaying it, right? So once you have control over the switch, you can exfiltrate traffic. Now, that'll work in many environments but on some real time systems, let's say electrical, you have like 500 milliseconds of time in which to perform some of your attacks. So if you route your traffic halfway around the world to another country, you're going to fail to reach your timing constraint in that system. So you need to be able to alter the firmware to perform some of the process control in the style of Jason Larson or something. So then I'm playing around with the switch and before authentication to the actual web server, I find this DOS, right? And essentially there's a config file before you log in that you fetch. And I created a no-cache parameter and just, you know, incremented or randomized the number and requested this particular config file 2,000 times very, very slowly. And the web server of the switch rebooted, right? It was just initial fuzzing when I first started. And I thought, oh, it's a DOS. So what? DOSes don't matter. But they do really matter in industrial systems, particularly when the web server going down causes the switch to reboot for two minutes and all the traffic to be dropped in that time period. So this is the fix, the current fix from GE. They had to go to Garakom and ask Garakom to fix it. And I think originally Garakom said, sorry, this is end of life and now they've changed their mind and now they're patching it. But I want you to read that very carefully. Their mitigation is basically turn off the web server. I'm not too happy about that. I think we can do better as a community. But how do you configure it now? Well, you would configure it through Telnet, which as we all know is an awesomely secure protocol. So why does DOS matter? Why does DOS matter? In the ICS community we try and be really good to each other and reference great work where we see it. You may have already seen the talk of Marina Crotaful and Jason Larson. Amazing stuff. They dig really deep into the physics of damage and exploitation in industrial control systems. I stay a little bit higher up and then try and focus on vulnerabilities and cost of attack. But basically they showed in this paper that having a DOS in certain types of chemical process control is enough to give you almost complete control over the process. So go and look that paper up and you'll understand why we're trying to raise DOS as a much more serious vulnerability in industrial control systems. All right. So I'll do the first one. There was an old day essentially, the SSH username enumeration if any of you remember it. I found it when I first started looking at the open gear. It's worth saying open gear gave me this switch. I met them at a conference and they were like, oh, you research switches. We'll give you one and you can test it and we'll get free testing and you'll get publicity and that's pretty cool, right? And I thought that was remarkable, especially in the ICS space. So they fixed it in one week. They patched this vulnerability in one week. Now I grant you, it's just a PAM configuration change. But still, GE took eight months to tell us to turn off the web server. Siemens took three months to change some of their documentation and fix the CSRF. So one week, I think particularly deserves a round of applause in this case. Yeah, so I spent most of my time looking at the open gear switch and it's out of the few switches we've looked at, it's probably the most secure default state. Everything we found essentially was post authentication. You had to be logged on to the switch to be able to do anything. So it makes a lot of the, whilst the impact may be high for some of these issues, the actual likelihood and realistic sort of impact is much, much lower. And in our conversation with open gear, they were upfront and told us when these fixes were planned, they had scheduled them all in, get them all worked. So they really did help us out here. But we'll carry on with the issues we did find. So that is the open gear switch. It looks like that. It's that on the website. You can download the firmware. They also have a open development kit. So you can basically build your own firmware for these switches. So if open gear ever go out of business, you don't have to replace your hardware and you can keep things running over. So that's pretty cool. First issue, they have on their web page this nice support report, which gives you all sorts of useful information. It's probably really tiny, even though it's blown up there. But they have a link right at the top that allows you to download the support details for essentially offline viewing. But in this page is normally only accessible by the root or admin user. However, you can directly navigate to the page as any user and pull down all the information. This includes things like the Chrome tab, the init tab, the entire syslog, a support text file, and that contains the IP tables configuration, the location of all the SSH keys, which becomes useful later on. The IP tables config and the config XML file which includes contains all the user names but no passwords. So immediate user name enumeration. So you can pull down the theater of everybody else. The next issue they have, we found, was get file. And essentially this allows you to get any of the files on the device. Useful if you don't have SSH or telnet access, but you can still essentially pull down any of the Linux files you've got there. So you can pull down their password file, which looks slightly more sensible than the GE password file in that it has no passwords in it and only contains sensible users. However, you can also download their private key. Again, it's their default key. You can change it. It is documented. And then we tried the traditional sort of cross-site scripting. Can we get cross-site scripting in place against this switch? No. Impa validation denied. You know, they've actually thought about this stuff. Yeah, I'm as shocked as you are. However, we decided what about outbound validation. So they have this config XML file. You can log on to the device and you can change a config XML file and basically just input your own XML that looks like HTML. And it executes. Oops. But again, you know, cross-site scripting on these devices, but you have to already have sort of permission to do this sort of thing. So yes, it's nice and quirky, but realistically, it's not the most brilliant of attacks. However, this leads us to the last pack, which is essentially cross-site request podries. So this is a burp suite dump of the sort of creating a new user. And so this is all the bits. It's got all the parameters. It's a post request. So we have mocked up a small web page that contains just that code that gives us essentially all these users. We deliberately asked ourselves to give ourselves admin user and if you can have somebody logged into the device and get them to navigate to your evil web page, you too can create your own login backdoor onto the account. And we can probably demo that if the demo gods are good to us. Can you make that look easy? Can I make that look easy? We'll find out. We'll find out. So at this point in the presentation, you're probably wondering what the hell this guy's on stage for. So why don't we give him a little chance to explain that while Colin sets up the demo? Cool. Thanks. So mostly I stand up here with good looking guys and I don't know if you're familiar with this but what I really like focusing on is the defensive aspect of it. So these two found a bunch of vulnerabilities actually 11, right? And five different switch families. And at that point, if you're an industrial control system person, you're probably kind of depressed. So I'm the kumbaya defenses doable kind of person to talk about how attackers can take advantage of these things, but more importantly how defenders can look for those things. So the unique thing about industrial control system networks, are you ready for the demo? Ah! All right, so this is the login page. Well, it's not the login page, but this is the user groups page. And we can see down the bottom we've got a root, a testing and admin user. The root is the default one. The testing and admin I've just created. So our test script looks shockingly like that, which is exactly what we had in the first place. And so we're logged in and in another tab, if we have another tab, we can navigate to our evil page. Our evil page does stuff. It thanks for a while. Thanks for a while. Please look. Song and dance, anybody? And we can go back to our user groups. And there's a new user. Yeah! And it's admin. Yeah! I know it was frighteningly so. But again, the ability to get that sort of timing attack is probably really low. Someone logged in as admin and you can get them to navigate to a page you control. You either have to have local control, local network page somewhere or they've configured their own device to connect to the internet and that's probably wrong. So yeah, it's reasonably high impact but likely had way down there. So yeah. So in summarizing, it is a strong default state compared to their peers. The issues have been thought about and they have remediated them. The session ID is a default thing. You can just switch on. The configuration's there. They've got excess input validation. They do have potentially high impacts but realistically low and it's all post-authentication. None of this was pre-auth and these people have essentially fixed these issues in what I like to develop a sprint time, so two to three weeks compared to the vendor patch time of eight to nine months and I think that deserves a big round of applause because these guys are on it. All right, back to Defenses Do-able. Build it back up. All right, so there's a couple themes that go throughout the research. The first of which is as I was working with these gentlemen looking from the defensive standpoint of trying to analyze the post signatures and things like that. There was an interesting amount of vulnerabilities they found and only a level of them made it into the presentation because that's how many PowerPoint slides we could get through without killing ourselves. So when you look at it there are huge issues in the community around these vulnerable infrastructure. So if we see GE or Siemens or Open Gear, that's not the only ones that are bad, right? It's across the industry and we're trying to make it better. But the second theme to come out of the vendors, right? So we want the vendors to do better. That'd be great. But unfortunately if it takes eight or nine months to make a patch it usually takes another year or so to actually implement the patch in these facilities. So we don't want upwards of two to three years of vulnerable infrastructure where adversaries are trying to target these facilities. So we as a community especially for those of you in the ICS community can do a good job of monitoring for and defending these pieces of infrastructure without the vendor. So that's what I want to talk about a little bit. An interesting aspect of industrial control systems networks that I really like is that they're fairly static. If you think of your enterprise IT kind of network you have your users going to Facebook and LinkedIn and all sorts of weird places, right? And in ICS network that shouldn't exist they should be relatively segmented networks. You should have multiple places to capture data on the environment, right? So this screen up here an ideal network is you have your processes separated out from each other to capture data but they're static. If they're using Modbus TCP, DMP3, OPC whatever ICS protocols that's the protocols they should be using on set ports. There should be relatively static bandwidth usage and these type of environments. Now unfortunately we don't really usually do a good job with the networks. This is how a network should look like and this is the typical layout when you walk into an ICS facility. It's all flat properties. So the downside with that is adversaries can get away with doing a whole heck of a lot without ever being noticed because on the network it's really easy to spot. When we're talking about industrial network vulnerabilities adversaries would use that kind of thing like what does it matter? Well adversaries use that type of thing to learn the system, learn the process. There is no getting root on the PLC or the control system itself that doesn't matter. You can change control information do whatever you want just by having access to the network. So the ideal set for any sort of national adversary or anybody in the government that would want to get into these facilities across the world is to get the network. That's it. But we as defenders can see that and I always find it interesting that there's these narratives that build up about how amazing adversaries are, how amazing the quote-unquote APT or whoever is and it's funny because those are usually high. I think it's kind of interesting that folks think there's less power point and less bureaucracy in the government. So we actually aren't necessarily as good as people think we are. And realistically we can do a good job as defenders. So when you look into the ICS environment there are problems. I want to note yes, the vendors have good excuses sometimes about legacy equipment or who owns the infrastructures at the union as the actual vendors themselves who can go ahead and mitigate those. So the first one I want to talk about is network security monitoring. Now what I've done is I've thrown HabEx up here as a quick example and some of you are probably thinking well that's wire shark data, what is wire shark data doing in this talk and realistically after dropping 11 vulnerabilities on stage I feel we've earned the right to talk about basic security shit. So when you look at an ICS network it's really easy to map it out. So pre HabEx this second piece of malware ever to specifically target it. When you look at what it does on a network it's very easy to see it looking for ports and protocols and things that don't normally exist on that network. If you're inside of safety critical systems or inside of an ICS you should know your network traffic and it's very easy to spot these things. So from a safely capturing environment especially for the recorded aspect when there's my ICS sisters and brothers watching in about what do we do in our facility what can we do today without waiting on the vendor. And go safely capture data out of your environment. One of the things that I dislike is there's an atmosphere in the culture of don't touch it. If it's not broken don't touch it. If the lights are on and the oil is pumping please don't touch the network. And unfortunately you can safely go acquire data you can safely go acquire information and see the things that are going on. So as we mentioned during the press conferences leading up to this there's high confidence that there are folks there's adversaries that are interested in that kind of thing and we need to go collect the data. So again especially for the ICS folks tuning in we want you to make the infrastructure better. This isn't about shaming vendors this is about saying we have a problem in the community and we can fix it. This is about raising that awareness. So throw up some tools in the slides you can easily go through and look at it there's YouTube and defcon and black hat it doesn't look that simple right and in terms of malware on the network yes and the pre have actually have that nice little heart beat but post have actually don't but in a static environment it is really easy to see things like firmware modification. It's a giant spike in network data inside of an ICS environment. You do not have to be a trained security professional to do this. The interesting part is it's very very similar to a control system engineer system architect process guy. When Aaron and Colin made this comment before and black hat and that is that the same skill sets that these folks have looking for abnormalities anyways around alarming conditions and things like that inside the environment is the same skill set it takes to look for abnormalities of the network. It's just different sets of data that you have to look at. So today your ICS engineers and architects go back to the previous one just for a second any engineers in the room excellent right so the point here being we're talking about doing change control using wire shark it doesn't have to be security stuff but you should know if anyone tried to change the firmware not just sign a piece of paper that everyone in the facility agrees to not change the the firmware unless they go and speak to Ted or Scott Adiva or whoever it is to use the wire to see if these upgrades are happening at times they shouldn't be and I think there's something to be said for merging the idea of some security monitoring and your change control processes that already exist in the plants all right thanks and so we're not going to go through and say like here's how to defend every aspect I know that would probably be kind of boring for the audience and the environment went through this up here kind of as a joke when we're looking at security inside of ICS it takes a whole lot of resources for an adversary to actually break in and do something neat inside of an ICS when you think of that power plant or that nuclear facility they're all unique there is no just one vendor takes control of one area or there's no one network configuration it evolves and it grows to do to even break or steal data or do anything interesting inside of an ICS so if we just do our jobs inside those networks I truly believe that ICS isn't this super vulnerable thing compared to IT I think an ICS network is one of the most defensible networks that we have and we can actually do that security well and to add to the kumbaya flavor yes IT and OT or operations technology need to start working together better with OT folks and break down those barriers that are required to go out and identify these things because all of us want to have secure infrastructure all of us want to bring that legacy and I think that's really where we want to focus the end point of the talk is saying I'm ashamed really when we look at these infrastructures that we're leaving behind that we are not doing a better job of security that we're not actually developing things out for critical assets human life trains power plants we are ashamed we understand the cost of failure in society of banks we know how much money to invest in banking security but we don't necessarily study the cost of failure of industrial systems infrastructure and we don't necessarily think about the long-term cost of buying things that can be OEM and not patched when we need them to be some time in the future so we are ashamed and we would like you to be ashamed as well we can all help contribute to this whether you come from a web pen testing background we've shown the web vulnerabilities whether you come from a strong reverse engineering background this is firmware you can start pulling apart the binaries and all the rest of it this is something we can all contribute to help improve essentially the infrastructure that's keeping the lights on in here that allows we don't mean this in like a def con you should be ashamed of yourselves kind of way we mean think about the infrastructure that was left to you right you were left a legacy of road infrastructure of train infrastructure of telephone infrastructure that enabled all of us to come into this room and talk about hacking and security and culture and technology we were left a legacy now when we're screwed so we want you to reclaim the word legacy we want you to treat industrial infrastructure as infrastructure and dream of a world where you leave your children a legacy of secure functional infrastructure