 Thanks. Thank you very much for the invite. I've never been to DEF CON or Black Hat before, so this is kind of an interesting phenomena for me. Thanks. I'm an engineer that got into security rather than the other way around. So what you're going to hear is the engineering view of the world and what's going on and what's needed and what's missing. I've deliberately, this thing moves, put this slide up. Everything we're talking about is this guy right here. If you saw that in the village, that's a sensor. In this case, it's a 4 to 20 milliamp sensor. It's probably measuring flow. They've put it in line. It could be measuring pressure, flow, temperature, picket. Well, we're talking about our fingers. This is what measures things, and it's going to the brain, which would be the controllers. If you cut your fingers off, the brain can't do anything. And that's what all of this is about. I'm going to take it now back to the beginning. OK. So like I say, I wanted to thank Larry for inviting me because this is a really important area. And one of the biggest things you're going to get out of this is we've got to, got to, got to have the engineers and the IT security people working together. And that is not happening in most cases. Because what we're talking about here is pure engineering. It is not IT. You've probably heard all of this. The big thing is that last bullet in what's in bold. Control systems monitor and control physical processes. It doesn't matter what it is. It doesn't matter if it's a power plant, a pipe, a human body. If you've got physical processes and you've got a monitor, for example, temperature, pressure, or flow, or radiation, and then you want to control it, that's a control system. Industrial was a poor term. We didn't know what else to call it. Because all of the building controls here are essentially ICS, because you're monitoring temperature, you're monitoring steam flow, water flow, humidity. All of the things you think about, there's nothing different. So everything we're talking about here crosses every border. But the big thing is most of you, if you're coming from the IT world, you have the CIA paradigm, confidentiality, integrity, availability. It isn't just that it's backwards for us. It's actually missing the most important letter. And that's the letter S. And the letter S is for safety. Because in our world, we can kill people. That's the most important. Everything we do is reliability and safety. Confidentiality, really one of the only places it's important, is if you have a meter, like a smart meter in a house or something like that. For almost everything else, confidentiality is very, very trivial. When you put it in a database and store it somewhere, then it becomes IT and confidentiality is a big thing. We're talking about data in motion. So one of the big things is a problem in Europe. Europe is totally confidentiality based or privacy based. In the control system world, that's not us. This is just a Venn diagram I've had for years. Part of our problem is you've got two worlds coming together, the IT world and the control system world. And you'll notice the number of arrows. Most of the people that are working this are coming from IT. Very few are coming from the domain. And part of it is it's unfortunate, but control system people generally do not like security. It's an impediment to what they do. So you don't have near enough people who actually understand the domain even wanting to be in this. And we need to figure out what to do. There is a really, really, really small dot where you have actual ICS security experts. And by the way, one of the things I always tell everybody, if somebody tells you they're a SCADA security expert, run the other way. It generally means they learned how to spell SCADA within a year or so. Because there's one other thing. How many people would go to Macworld and only want to talk about PCs? Sounds kind of silly, doesn't it? When you talk about SCADA, SCADA is not every control system. SCADA is only one type of control system. And it isn't used in a lot of places. So when you use the term SCADA and you don't use it right, you will lose all credibility really fast. We are here in, you could either say the cyber-physical world, but really we're in the physical cyber world. Everything we're talking about is physical. Cyber happens to play into it. And you need to address it from that perspective. People hear about advanced persistent threats. When you look at things like Stuxnet, those were hard-coded default passwords that were part of the design of that controller. That wasn't an APT. In a sense, it was a persistent design vulnerability. You can't change it. And by the way, this isn't just Siemens. We have a lot of PLC manufacturers that have, if you will, used the passwords as part of their actual software computing platform. So if you change the hard-coded default password, that PLC falls on the floor dead. This is not like what you would normally think. Well, geez, how stupid can you be? This was never, remember, we didn't start out life worrying about security. We worry about reliability and safety. And a lot of this came because we were trying to capture mean time between failure data. The only way to do that was to have a backdoor in to be able to continue to capture and get that data. Totally different reason. In IT, you normally want to stop data. You want to do it denial of service. When you're in the control system world, and you want to do something bad, it's not a denial of service. You want to cause boom in the night. So what you want to do is you want to take control of the process. Or you want to take control of the operator displays and have the operator do the wrong thing. It is not a denial of service. What you're looking at, level 012, this is all from the Purdue reference model. This has nothing to do with the seven layer OSI stack. So what you're talking about, this is the process. So this is like a boiler. Or it's like the electrical wires. Or this is like the refinery process. Here, level 1 is basically the real time measurement. This is normally, now again, years ago you could have subdivided this easily. What happened with Stuxnet, that was level 2. Now, many years ago, there was a distinct difference. Now with networking and with ethernet, et cetera, it's blurring. Level 2, which used to be totally isolated from the internet, is no longer isolated from the internet. So it starts blurring when you start talking about levels 2, 3, 4 on up. And it's even starting to blur even when you get down to here, because you start talking about sensors on chips and things like that. But what we're really worried about is when you can compromise a system by physics, you can't stop that. One of the things Aurora does is use physics to cause essentially very, very, very large torques and current spikes in milliseconds. There's nothing you're going to do about that. That's how, by the way, you bring the grid down for about 9 to 18 months. That's what we're talking about. This is not a day or two or anything else. This is how you break very large equipment, like generators, motors, transformers. These are really long term equipment. And you don't make very many of these, and these are almost always very specific to the specific process. And there aren't very many vendors who even make this anymore, and a lot of them are offshore. So when you start breaking things, there's a lot of questions that come in. Now, again, our communications are not native IP. You're talking about, say, Modbus over IP or DNP over IP. Pick something, but it's not native IP. What we do is we use IP as a wrapper, OK? And there's work going on. But these protocols are not, like I say, native IP. And the real protocols we're worried about here aren't Modbus or DNP. There are things like heart or wireless heart or Profibus, Fieldbus. These are your level 0, level 1 sensor protocols. There's not only no security in them, they were designed almost to be insecure. You'll find a lot of these have things like XML, OK? So one of the other questions is, so what? Well, you can get to them. The IO, the input-output cards, you'll see them. I mean, when you go to the board, you'll see things like Phoenix Contact or whatever, OK? It allows the instruments to communicate bi-directionally. You'd think about things like data diodes. Well, that doesn't do anything when you're talking about sensors, especially smart sensors. Smart sensors have to communicate bi-directionally. Smart means it's digital and it's done all of the conversions at the sensor layer. And you'll notice, again, engineers can no longer simply measure the output analog signal. They need to communicate with a transmitter and read the digital signal. Well, there's no air gap. It's impossible to have an air gap. Everybody thinks, hey, I did this, therefore I'm fine. You can't do that. And the whole point about the presentation, there is almost no cybersecurity in them. There's no authentication. And from a cyber perspective, there's no forensics. One of the things that's really important to get across as an engineer, I don't care if the process fails because it was a system problem or because it was hacked. The process still failed. We need to understand when the process is starting to fail. And what you'll find in our world is often the only difference between malicious versus unintentional is the motivation of somebody. Why did they do what they did? It's different. And we need to think different. We need to think about how do you keep the boiler within a prescribed temperature and pressure range, regardless of why? And like I say, the protocols are vulnerable. I put this up. This was an ICS cert notification. Last revised, July 25, 2017. We're talking Thursday. That's profanet. When people think, hey, this isn't vulnerable. Yes it is. We've had any number of demonstrations of hacking heart, wireless heart. A friend of mine just got his doctorate from the Air Force Institute of Technology at Wright-Patterson Air Force Base. He's now at Oak Ridge. His doctoral thesis was on hacking heart-enabled process sensors. What they're going to be doing next year, even though he's already proven it, is the next one going through the PhD program is going to go directly after wireless heart, even though we already know that's vulnerable. His thesis happened to have been on wired heart. So just so people understand, not only are we talking about a problem, it's a very exploitable problem. Now, control system cyber incidents are real. This is my database. It's been well over 950 to date. These are not vulnerabilities. These are real incidents. And you can see the impacts ranging all over the place. And you can see this is complete destruction of a turbine that happens to be Aurora. That's the DC Metro train crash, San Bruno, Bellingham. That was Marucci. That was an offshore oil platform that they hacked the buoyancy and tilted an offshore oil rig. This is really different than things you're talking about. Gee, somebody stole my data. This is the dam in Russia. I don't even want to try to pronounce it. That lifted a 1,000 ton turbine off its pedestal. There were 75 people killed there. In fact, the Russians originally thought we did that. That happens to be a brand new Navy ship because of control system cyber problems. On its maiden voyage had to be towed in the last 40 miles into Norfolk. So when you start looking, you can also see there's a lot of different industries here. This is just kind of my summary. There have been over 1,000 deaths to date and well over $50 billion in direct damage. In fact, I'll give you an example of what we're talking about. And it also flows into what this presentation is about. I think most people know what Stuxnet is. It was changing the logic in a controller to change the process and then changing the logic back so nobody would know. You know what I just described? Volkswagen. And by the way, Fiat, Harley-Davidson, all of them. They all did exactly that because they couldn't meet the environmental controls. And when I had first was looking at all of this about quote unquote ICS sensors not being secure or authenticated, I called one of the professors at UC San Diego who was doing the analysis on Volkswagen and Fiat. And I asked them, are the sensors in cars, do they have authentication or security? You're right, no. There's nothing specific about this. But think about it. If the sensor data can't be trusted, what does it say about every single thing you're doing with network monitoring? If you're a doctor and you cannot trust your temperature or blood pressure readings, how can you make a diagnosis? That's where we are here. And you go think about Microsoft with the Azure Cloud or any of the others. What's their assumption on everything? The sensor data is authenticated and trusted. That is the assumption for the entire Azure Cloud. So think about what I just said with respect to the ICS world and cloud computing. It's really that simple. Here's the techie that explains most everything. What you have over here is what most people realize. Here's your normal net, your operator displays. This is all windows. Everything here is generally IT. Prior to Stuxnet, don't even look there. Since Stuxnet, people are really worried about the PLC, only this is a valve. Here's a motor. That's a sensor. This thing here, that's what makes that 1970s or 80s valve a 2015 valve. This is the smart electronics. That's the smart electronics for the motor. Same thing with the sensor. And that's all remotely accessible. So what you've got here, there is no security. Now, here's our process. That's the most important thing in the world to us. This stuff starts as serial. It ends up going through the serial ethernet converter. And by the way, what did the Russians hack, both in the US and in the Ukraine? It was the serial ethernet converters. Only they used it to go up into the network, not down to the sensors. Here is where all of your network monitoring starts. They assume whatever happens here is correct. That's the flaw in every single network anomaly detection program today. They assume, when you start here, that that sensor value is correct. And they have no way of knowing that it isn't. And I showed you that picture there. Now, this is the holy grail. The holy grail is to know. In fact, I'll give it to you this way. About four or five weeks ago, Honda shut down an assembly plant in Japan. Cost them 1,000 cars. Why they found want to cry on the networks. There wasn't one single alarm on the factory floor. They did a self-denial of service without having a clue whether it affected them or not. This is where we are today. You cannot, from here, make any real view of what's going on in the real process. Because all you've got is the data that's going from that serial ethernet converter. The holy grail is to be able to look at the process and cross-correlated to the network anomaly. To be able to say, hey, I found malware. Is my process changing? Assuming it shouldn't, and you're watching the sensors and you know precisely what's happening, why do anything? This is probably the most important thing in the world when you talk about the industrial world or the commercial world. When do I care? Today they cannot make that statement. This is just the sensor. And again, here's the point. What you're doing is you're coming back in, and if you change things like zero and span, you can prevent a sensor from ever reaching a set point or to reach a set point long before it should. And you will never know that from network monitoring. This happens to be a smart sensor, the same thing. You get into here, you make your changes. Nobody will ever know. Now, what happens? This is what I was doing 20 some odd years ago when I was still at the Electric Power Research Institute before I ever heard of security. We were worried about doing root cause analysis on failures. But what we were doing was a snapshot in time, doing post mortem. Well there's technology now that allows you to do this in real time. This is where the Holy Grail is coming. But what's happening is this is what's filtered out as you go through the serial ethernet converter. That's why nobody here on the network side is ever going to be able to really understand what's happening. What you're getting essentially is the absolute value of the signal, not what's actually happening in the process. This is what is completely and totally different. Here are real things. And two weeks ago I was at a power plant. It had tripped. They had tripped it. In other words, shut it down. Very large plant. A pressure sensor reached a set point. Why a sensing line clogged? It is impossible to see that from network monitoring. But if you're looking at the process, it becomes very obvious something is going on with that sensor. This is something I worked on a long time ago. This is a nuclear plant where a safety relief valve didn't lift. This is safety. This is as much safety as you can get. Why didn't it lift? Because the sensor could never reach its set point. What happened? That's Tomsock, the dam in Missouri, basically failed about five or six years ago. What happened? It had a couple of level sensors attached to the wall of the dam. The tie-downs broke, so the sensors came out from the wall. When they came out from the wall, the level readings changed because you measure level by elevation head. You measure level by height. Well, the height changed. So the SCADA system took the data, took it as real, had no reason not to. And when it said the level was low, it turned the pumps on until the reservoir overfilled and the earthen dam collapsed. There's no hypothetical here. This is a stuff that every bit of this has actually happened. This was one of the biggest fires in the UK, which was a refinery fire because the sensors didn't work to tell them that the tank was being overfilled. These are huge catastrophic problems. The controller can't do anything, or it's different. The controller will do exactly what the sensor is telling them that the process seems to be. This is what is reality, and this is what network monitoring can't do today because it doesn't have a view into the process. Part of what's going on is this. Sensors, actuators, and drives to this day are still being made with no security and no authentication. They're engineering systems. What our vendors do is they have the security done over here where you're worried about the network. Over here where they do the engineering for the engineering systems, there are no security requirements. That's to this day. This is a huge problem. In IT, it may all be all about packets. For ICS, it is not. IoT, IoT is a funny animal. What is IoT? If it's Fitbits and refrigerators, who cares? But in a sense, we're part of IoT. We were IoT 20 years before there was IoT. And they're certainly not looking at anything like this. And then here's even ICS cert. They put out their 2016 statistics. There is no mention whatsoever of anything at level zero on one. The only thing they talk about are network issues. And it's the same thing from 2015 and 14, and you can keep on going. There's nothing that's ever changed. And what needs to be done, you've got to get the engineers involved. And reliability and safety have to be number one. Confidentiality is fine, maybe. But if you don't take reliability and safety there, why bother? And the other thing to realize, everything we're talking about in the sensor world is all reliability and safety. It's really not cyber. But it's the most important input to cyber. Because how can you monitor anything if you don't trust the input? And that's what this is all about. We have, what do you do about supply chain when you talk about this layer? What do you do about risk assessment methodology? Risk assessment, certainly vulnerability assessment, assumes you're secure. And it's a gap analysis away from what you think is secure. When these have zero security, what do you do about a vulnerability assessment with zero? It's effectively infinite. Like I say, the ultimate holy grail is to correlate, process anomaly detection with network anomaly detection. Short of that, we're going to continue to have the Honda issue. And this occurred, by the way, 10, 15 years ago with Code Red and Slammer. We had utilities shut down, not utilities. Paper and pulp plants, you name it, they shut them down. Because somebody said there's malware out there. Not because there was anything within the process that changed. Nobody knew. You want a funny form of ransomware, go do this. But malware, nobody has any idea what it means. So with that, any questions? Martina, I was going to put this back on. Here are some of those, because I know you were late. These are some of the real sensor related incidents that have already occurred. I just went through my database quickly. This is not all of them by any stretch. But like I say, you're talking about hydro, water, waste water, fossil, nuclear. This was essentially mining, refineries. This is all over the world. This is not a utility problem. It's not a North American problem. It's not anyone vendor's problem. What's a mindset so far you've done in the sense that there are multiple sensor same type, multiple different types out of man? Well, one of the things that I'm going to have to get going, because I've got to go to that thing, too, where Chris is, part of the reason I brought up risk assessments. We've not done very good risk assessments. One of the things, how could a single pressure sensor shut down a large power plant? From an engineer's perspective, something is wrong here. Now, part of what it's saying is you better understand what is really, truly critical. The other is, what happens if you put redundant sensors but they're on the same, if you will, ethernet land? That's exactly right. I'm working with some technology to try to really understand what's going on at the sensor layer. But first and foremost, you've got to have a good risk assessment. If you don't know what's really critical, you're dead on arrival. I mean, it's a really good question, and this is such a great answer to your question. One pressure sensor isn't supposed to shut down a multi-unit power plant. And it did. A security from a sensor perspective, what do you think is the solution to that? Because it's like you're trying to be aware of process issues and you have to use other sensors. How do you use sensors to protect your sensors? This is the only way I know. I'm going back to what I did years ago. Well, you need monitoring to detect what's going on. I'm working with one or two small companies that are actually looking at this. And they've actually done some really neat, interesting work. And that's why I'm saying we have a shot at the holy grail of being able to correlate the actual process to the malware. And without that, it's not possible. Yeah, don't forget, where is Pi getting its data? That's what I was trying to explain. This is the official catch 22. Everything you're doing starts at the sensor. If you compromise the sensor, everything you look at downstream, I just had this discussion with somebody about, well, hey, I've redundancy because I have this historian. Yeah, but where did the historian data come from? That's what I heard. Because I got in yesterday and I didn't have a chance. Yes, yes, absolutely. And that's why I wanted to put this back up, because you came in late. But this is where we are. And by the way, the serial D ethernet converters were how the Russians put malware into our US grids in October 2014 and is still there. It's how the Russians put the malware into the Ukraine. I will say the attackers. There you go. You weren't telling you're right. It's how the attackers implement it. And I apologize. You're right. And I have to go too. Yes. And the Arco's units and one of my men who was out in the facility was climbing up and he tripped. Hit his foot, hit the sensor, and it sent the entire station into flair. Yes. All the processing went straight up and the whole ground started shaking while there was an earthquake. And because we knew what had happened, we were able to tell the control center when they were able to shut it down that they would have had to troubleshoot that thing for hours to try to figure out what was going on. And that's for what it's worth. The city of San Francisco was brought to its knees April 21st by one relay and one substation. That should have never, ever, ever, ever happened. Yeah, I just want to go to that console.