 Okay, so this is industrial cyber security from the perspective of the power sector today Quickly introductions, I'm Wade Polk. We got Jay Novak here and Paul Pamukowitz over there We're all control systems engineers. We design the brains of power plants mining facilities, etc Our company's worldly Parsons. We got 29,000 employees in 40 different countries We do everything from infrastructure to environment to power mining minerals hydrocarbons and even some manufacturing Okay, so what kind of stuff can you expect today? You learn a little bit about what has happened what can happen You'll learn a little bit about traditional process design as well as malicious process design something fairly new You'll learn the basics of protecting a power plant and then we'll give a case study of a PLC The security flaws and how to mitigate those security flaws You'll see examples of worst-case scenarios and after the Talk we'll give an audience free for all where you'll all have the opportunity to attempt to hack our network And doing all this will get an indirect overview of process design and controls What not to expect? We won't be discussing industrial cyber security in-depth. We wrote a pretty large 70-page paper on that You won't learn how to be an INC engineer that takes about six years We won't talk about network hacking in-depth either Okay, so what is an INC engineer we design we specify and we purchase instrumentation controls controllers and network equipment We develop test and evaluate control logic and programs operations maintenance procedures and System and network designs. We also are required to meet various regulatory requirements such as NERC NRC, which we'll talk about later Thank you So what is a control systems? I mean, I think my first introduction to control system was through a cartoon with Homer Simpson where he's sitting at his control table and twisting levers and pushing buttons and turning on the parts of sitting turn them off But in reality control system is quite a bit different and complicated Control system is a set of instruments controls and controllers used to manage and control the behavior of process machinery and That's the process First thing on the list. We have these instruments. These are variable Devices that collect and transmit data. They can be for flaw temperature Pressure vibration, etc. What they do is usually be a either hardwired IOs or analog IOs Or maybe some kind of proprietary communication that transfer data back to individual controllers As far as controllers goes to modify the operation of machinery or the process Some examples of this are actually buttons and levers and switches. Other ones are more involved like HMIs human machine interfaces and etc Now we're going to move down to controllers Skated you guys have like six or seven speeches of skated this past few days. So we're not going to get far into that DCS distributed control system basically set of modules and controllers spread out throughout the plant In the different locations that vary the process of the plant Then you have things called PLC's program allows controllers and these devices are actually Earlier they kept they were fairly simple and they were used to present You know to work a simple process, but now they get a more They're getting more robust. They get it more developed. They get to get a bigger They're trying to catch up to DCS a portion, but they're not there yet Then we got smaller controllers such as a single loop controllers Socs for small occasions. So basically all this IO and all these controllers do one function They energize turn on actuate devices like motors valves pumps and actuators In terms of magnitude a typical 2 gigawatt power plant Has up with 150 PCs around 50 PLC's 1 to 10 DCS loops And hundreds of HMI's again the complexity in the side the size depends on complexity and size of the plant the more units the more Environmental controls you have the more things you have running a plant the more complex the plans can be So what are some roles? Responsibilities we basically have five groups within a plant first. We have engineering and that's that's us We design a built and by the equipment. That's what engineers do next We have operations and operations basically run the plant their maintenance They're the plant 24 hours a day that make sure they make sure everything works And next we have the IT and IT's a handle enterprise management hardening Running of the computers etc And last group is management. I've really don't know what management does I'm kidding. I'm kidding that they give the money they give the budget in the schedule that we need those guys We really need those guys But if you can if you can always have one more place and also we have the otters, which are people that eventually Will come in and well Do very bad things to us But if you can look at a network right here, you can notice there's a connect between engineering and operations Basically, that's a kind of a marriage of convenience. They have to talk to each other They have to communicate with each other because if they don't the power's gonna the power's gonna go off That's as simple as that IT is a little bit separate on the side They know much about control systems They know a lot about network diagrams, but they don't know what to do at control They don't know how to control process. So if we go into ideal roles and responsibilities We can see that all the three factions or actually all the four factions are kind of merged together But this would mean that Engineering will provide the design the specifications IT will run the update the patches and the operations will keep the plant running But all three factions have to work after cooperate with each other in order to have a safe robust plant I'm gonna let go ahead and also forgot to say management has to give the money and give us long enough of schedules to Do what we need to do So it's where's that happened I'm gonna let Paul talk about that All right, so what is the worst that can happen and there have been a lot of talks today How many people just out of curiosity went to James Arland's talk this afternoon about? Cyber cyber dupery something like that. Yeah He talked a lot about this about What the worst case scenarios are essentially there's two two different viewpoints on this and everyone falls within Somewhere between these two you have the really optimistic viewpoint that says well Nothing's happened so far. There's nothing broken. So why spend a lot of time and money trying to fix it The the industry for especially in the power the power sector head is sort of missing a Catastrophic event that's happened that you can point to and say well if you don't secure your network This is what's gonna happen to you. It's gonna cause this much money and this much damage So this is that's why it's important to do this stuff So It's like you can't see the problem. So there's there's no problem there It reminds me of a story. It might be a joke. I can't remember when you talk about this stuff They kind of get blurred together About a I think it was a military installation that they were running a network They wanted to install there and an intrusion detection system So they put it into place they turned it on and immediately they saw all kinds of unauthorized activity on their network And so they go to the the person running the installation the general or something and they tell him what happened and Instantly the general gets angry at them and starts yelling him and says why'd you guys put this thing in before? We didn't have any problem. There was no one trying to attack our network And now there's all kinds of people trying to attack our network That's the kind of the kind of problem We have is that you can't see the problem because nothing has really happened serious. So you can't point to that One of the other things that that these people will tell you is that these plants are inherently They're intrinsically safe because when you build a power plant you're putting in all kinds of redundant safeties mechanical interlocks Things that even if someone was able to get complete control of your network and start turning things off and turning things on and screwing things up These safeties are gonna gonna kick in and prevent pressures from building up and prevent things from really bad from happening prevent the most catastrophic types of damages on the other side of the scale you have the pessimistic view which is that The IT person who is preaching fire and brimstone up on the stage telling you that If you don't secure your your control system that work PLC is gonna come to life and drive to your house and punch your kids and drink all your beer at night Which you also don't want And there's some truth to that too because power plants are typically they're extremely complex They've a lot of times just have giant piles of explosive materials lying around in the yard specifically at coal plants And they're things that need to be protected So you've got to realize that these there's a lot of dangers inherent in running one of these things In addition to that they're they're often really really old Especially with power plants some of these are have been running for 30 40 50 years Maybe which means that a lot of the original equipment is in there That's 30 40 50 years old a lot of the original control systems in there are also that old and then things get upgrade upgraded you get a patchwork of Different types of controls from different ages, and that's what causes a lot of the vulnerabilities that you see out there now in addition to that Because you don't have the original design in there a lot of the safeties that you've built into the plant Maybe they're not going to function like you expect them to when you need them to You know you have a pressure building up in a tank somewhere And you expect when your pressure relief valves to open up and it's hasn't been maintained in 20 years Or it's been taken out because somebody didn't think you needed and you no longer have that safety So I talked just a few minutes ago about What the worst case what what's happened in the industry so far? And like I said there there haven't been a lot of things that you can point to and say That's that's a really bad thing that's happened, and that's why we need to fix our system now There have been a few things that have happened I guess you can call them near misses because nothing terrible has come out of them, but For the most part There's nothing to point to I'm just going to go over We picked out a small cross-section of things that have happened There's things in the news even even recently that are happening and with control systems and SCADA, but Just a few up here one of them in 1999 a petroleum a gasoline pipeline up in Washington State Was building there was some pressure building up it was all due to non-control system related things on unrelated incidents the pipeline was damaged and so you had some structural weakness there and As a result it exploded and spilled a bunch of gas into Nearby River and that lit on fire and it killed a few people Like I said, that's not a cyber there wasn't a cyber-tech involved there There was there's no control system problem involved there Unfortunately, what happened at the same time completely independently is there was a contractor or someone working on that system as this was happening Whatever they were doing for whatever reason caused the control system to freeze up become unresponsive so where there'd normally be procedures the Operator of the pipeline would have seen that pressure bill They would have been able to do something take some type of action to relieve the pressure. They weren't able to do that So, I mean that's not a cyber attack But that's the type of thing that can happen when you don't have when your your control system isn't operating the way it needs to be In addition to that just another example of an untargeted attack something that can happen without someone specifically going after you in 2003 in a nuclear plant in Ohio the slammer virus was able to make its way under a control network through I think an unsecured contractor connection As a result the network the plant wasn't operating at any time, which is important if it wasn't operating plant This would have been much much worse. It wasn't operating and as Since there was so much additional network traffic due to that virus The safety monitoring systems the modern the computers that monitor all the safety equipment weren't able to communicate with the rest of the plant which is a terrible situation to have in a New plant especially Like I said, there's a lot of others, but it's hard to point to one in particular that that has a devastating effect Alright next we want to talk about some of the some root causes the first thing that I want to say is there's a lot of root causes All right there there've been five or six talks about SCADA and cybersecurity and all that stuff at Industrial plants and I'm sure every one of them is going to point to a different list of causes and a different list of problems And they're all right to some extent The ones we want to talk to you guys about are the ones that are important to us as controls engineers And that's the design of the plant. So the stuff that happens before the plant is even running right now a lot of the focus on security is on the IT side the side about of Hardening your your computers and which software you install and where do you put your firewalls and that type of stuff Which is all really important stuff and that's where the focus should be but you miss the boat a little if that's all you focus on The thing that's important is that you need to start the process earlier when you're purchasing equipment when you're specifying equipment You need to specify stuff that's that's intrinsically safe on to put on a network if you're buying stuff That's not safe and is just automatically is dangerous off the bat and has all kinds of flaws You're creating a whole lot of work for the the people that need to make that secure and harden later And you're you're essentially wasting a lot of time and money the reason that the Focus is where it is right now on the IT side is because these plants like I was saying before are all really really really old plants So you've got all these systems that are on the network that weren't even designed to talk on a network Because they were built so long ago, and that's causing all kinds of headaches and flaws and vulnerabilities that that really shouldn't be there So we just sort of take for granted when we go out and buy new equipment that it's going to be better than the stuff That's out there because how can it be any worse? It can't really Unfortunately, that's not always the case what what we see happening a lot of times is that When you buy this new equipment and put it in you're replacing an insecure network with a really another piece of equipment That's a really insecure network, but in a user-friendly way in a way that my grandma could probably exploit and the reason this is happening is because manufacturers are putting features into their equipment like for example relays or MCC's that you can reprogram or reconfigure using your iPhone wirelessly or over a Bluetooth connection or a PLC that you can access over the web And it's just got no password and no security whatsoever and you can make changes to ladder logic and stay the PLC and all that stuff if you think about it is really good from a maintenance and operations point of view because It makes it easy to operate You don't have to lug a laptop around from device device every time you want to program it You just bring your iPhone or whatever around and do it right from there, but From a security point of view the trade-off just doesn't make sense So there's the kind of things the low-hanging fruit that from our point of view from a control systems point of view We can take advantage of now prevent ourselves the a huge headache later and Five ten years down the road when we we've replaced a lot of these systems We don't want to run into that same problem again Wade's going to talk to us a little bit about securing your plant and complying with some of the regulations that are out there, okay? How to meet compliance and protect your plan since this is all going to be fairly common sense to you folks But to control system engineers. It's not so common sense. It's fairly it's pretty new to us Okay, so rules and regulations out there right now NERC is a big one They can impose fines of up to one million dollar per day per violation some of these plants have hundreds of violations It's geared largely toward protecting the grid as a whole We also have NIST, which is the National Institute of Standards. These are all voluntary Standards it covers everything from cell phone use to implementing domains, etc Finally, you have the NRC, which is the Nuclear Regulatory Commission This is mandatory for all nuclear plants Compliance is required to maintain your operating license And they did just recently released new standards up until then it was all physical security There are many others we talk about them in our paper. I'm not going to talk about them today Okay, so real quickly policies list and procedures We've seen a lot of organizations spend upwards of a year trying to determine how to arrange their new policies to meet these standards Here it is. This is what we recommend You need at least three lists. The first would be a sites list Things like generating stations backup control centers control centers distribution facilities Then you have systems lists, which are internal to each site So for a generating facility, you might have coal crushers burner management control room, etc And then internal to each system you'd have devices such as PLC's DCS's Recorders PC servers, etc. You don't want things like valves or Solenoid valves Or any of those components on there because you really can't protect those devices effectively at all right now You will need to set a master drawings your network diagrams a lot of organizations right now We're just trying to create logical network diagrams to give a general idea of where they think a connection might be going It's not a good idea from our perspective. You can't protect a plant unless you know where every connection and every port is Okay, so the first procedure you need is policies. It gives a general overview of what compliance standards you have to meet sets your roles and responsibilities And gives a general overview of all the other procedures and whatnot The last five are pretty self-explanatory information protection physical security plan electronic security plan And the fifth one is a pretty big one. So we do feel that one deserves its own procedure change control and configuration management You'll also need design guides. You shouldn't plan on these. You should just develop them as the need arises But ultimately the goal of this is to design a set of policies that will be compliant with any standards that come out Even overlapping standards Okay, so the first step of meeting compliance is to identify all your cyber assets to classify them Classifications are used to prioritize the devices Ideally they should be based on likeliness of attack ease of attack and importance to operation You need to design a comprehensive automated and open-ended system the only way to do this is to track your classifications in a database to automate your classifications and To do it in such a way that any new regulations you don't have to modify your process very much Categorization is largely focused on categorizing information as top-secret sensitive, etc Okay, so electronic security controls network electronic Hardening, this is segmentation. This is protecting your network as a whole First you got to define a demilitarized zone Any device that actually touches the outside world needs to be in the demilitarized zone even if it's a relay or something very simple After that you have the primary electronic security perimeters, which is the entire network as a whole secondary electronic security perimeters would be a system or device grouping and tertiary ESPs would be something like a DCS cabinet DCS cabinet one All ESPs need to be protected all access points need to be protected effectively Device electronic hardening. This is our definition. It took a long time to come up with it's illegal. It's a lot of legal jargon I'll just read it real quick to ensure that only those ports programs and services required for normal emergency operations are enabled to ensure all Security policies are met and to add or strengthen security mechanisms to result in a more secure system than initial examination reveal That's compliance for you. Okay, so device electronic hardening generally involves these things You've probably all heard about them. A lot of these keywords are new to control systems engineers Anyway, you have to do surface area reduction configuration and security settings. You have to install protection software of some sort You need to do communications hardening limit the protocol use you use encryption authentication, etc Data hardening as well as hardware redundancy hardware redundancy is actually implemented fairly well right now That's just the way we've been doing it for many years. You don't want an entire plant to be shut down because one computer crashed Also, you have to count for maintenance Okay, plant physical hardening. It's pretty much the same thing. You got to find a demilitarized zone It's generally the area between the plant and the fence that surrounds the plant You'll generally have three or four access points there one for Employees where they get in and out of the plant another one for coal another one for things like lime or ammonia You've got to define a primary PSP, which is generally the building of the plant and second areas PSP's or rooms. You also got to protect these using something Preferably two-factor authentication Here's another legal definition for you. I'm not going to bore you with it. I'm not going to read it, but This is the kind of stuff you got to deal with in compliance You've got to define these things because the standards really don't tell you how to do it Anyway, whatever definition you come up with for these things. You got to do it in a systematic way Area physical hardening generally involves security devices locks keys cameras, etc Target hardening is focusing on a specific hardening To deter or delay an attack on a specific area materials hardening like Paul said there's a lot of dangerous chemicals ammonia Coal etc Damage mitigation is things like installing blast walls Access point management. We already talked about Environmental hardening is inherent deterrence such as lighting or shrubbery or installing roads in a certain way Security personnel policies patrols how often you have to go on patrols how often you got to store Your tapes and how long you got to keep them? And finally you got to do something about social engineering mitigation There's not much you can do about it. You can train on it Which they're kind of doing somewhat effectively right now and you got to control communications, which is kind of limited Okay, the last thing you got to do to meet compliance is incidents response as well as security reviews We're not going to talk about that today, but here's a couple standards to get you started Okay, so how do you protect your plant the idea is pretty simple To truly protect a plant you've got to if you do it right it doesn't matter what compliance standards you're Required to meet you'll be fine Just do it right Okay, our case study is a Security flaws and mitigation of a programmable logic controller Okay, before we start talking about security flaws and mitigation of PLC Let's talk a little bit about some of the assumptions that were made first of all all controls are associated with the PLC That means all the safeties all the interlocks everything is PLC control all the electronic safety as well Since the systems are simplified. Well, we have to carry this thing over here. So we have to make it as small as possible Only bottle I always are presented on our on our demonstration Again process and the logic knowledge Has already been obtained. This means is that if you already hacked it you have to know what the process is I mean just varying some variables really won't do anything you have to know what the process is You have to know how to affect at the right times Again these scenarios are conceivable for a percentage of plans, but you're not going to find this in every plan But be sure there are as many problems as there are processes are the potential problems Talk a little about the hardware We have a PLC of a undisclosed manufacturer and Model basically PLCs came out at the end of the 1960s begin of 1970s American automotive industry wanted to reduce the massive amounts of relay logic they used for the controlling of their installations So they built the PLC The net the network interface module that we have basically comes to flavors standard and web enabled The one we have basically takes the proprietary serial communication and converts it to proprietary ether communication Other devices that we have here in front of us. We have a variable frequency in drive I think so this side right here and we have underneath the board a three-phase motor with a the spins of the indicator the wheel Some other devices are the lights and the switches basically lights present the outputs on the body PLC and switches present the inputs All right, so the security flaws in this device The PLC that we have here is something that is really common in the industry right now We're not going to tell you which one it is, but it's something being used all over the world We we set it up and started playing around with it playing around with it And it took us only about two or three hours to find three horribly horribly wrong security flaws with this device The first one is that the device has a module in there an internet interface module that lets you communicate between the PLC and a computer or an operator station using an ethernet The module is it comes in two flavors all right There's a embedded web server on that module that lets you just type in the web at the IP address of the PLC and Access the the PLC over the web essentially the two flavors that the device comes in are a Non-write enable device so a device that doesn't doesn't let you write back to the PLC You can only read data from it sort of see what's going on. That's the version that we have There's the other version which is a write enable device that lets you not only read what's going on in the PLC, but change registers and inputs and outputs so that you can affect the ladder logic and anything that's going on in the device essentially Probably a little bit more less secure device you would think What it turns out is that both devices that have the same functionality apparently When you go to the web page on the non-write enable device the one that we have it redirects you to a page That doesn't let you write stuff if you open up the source code of that page You can just you can look at it and you can tell it's what is redirected me this way But if I go to this other page, I am I have accessed all the functionality to write to that device So that's really dangerous because you have the device out there somebody thinks okay Why I can't write nobody can write to this device except through the proprietary Software so I'm safe, so I don't need to set passwords. I don't need to protect it. I'm fine But that's not the case. They both have the exact same functionality you can Use that to exploit the system The second flaw that we found is that by default this thing is not password protected when I say not password protected I don't mean the default password is admin admin or admin password I mean they they literally thought that password protection was not a Feature that people would want to use by default, so it's just not on there you can password protect it so if a Security-minded person is setting this thing up. They say even though you can't write to the device apparently I'm going to still put a password in there the password is Enforced by a JavaScript a short JavaScript, which is run on the client's computer Meaning that anyone with a debugger JavaScript debugger, so something like Google Chrome comes with it automatically you can go in there Look at the code See what very the variables are see the password that the host is sending you and you have the password right there That's that's all it takes But it gets even easier than that if you can believe it you can also just skip the password checking completely all right the the device where the web page works by sending commands back to the PLC to a CGI script all right so if you know the URL you know the command you want to send all you've got to do is send That command to the URL it skips the password checking completely making it completely pointless It also doesn't do any all the formatting checking and things like that are done on the web page too So you have potential to do a buffer overflow or send any kind of garbage you want in there There's nothing checking that stuff So that's the the third one is the one we're going to use right now We're going to demonstrate a few common functions of a PLC and show you what happens if I'm over here With a script sending commands that I'm not supposed to be sending to the PLC and what with the outcome of that's gonna be That is if the the demo gods are in our favor. We have a lot of equipment up here So there's a lot of things to go wrong All right. Well Jay setting that up. I'll tell you about this first demonstration. This is demonstration of a single component A VFD, which is a variable frequency drive It controls It controls both the speed and direction of a pump or fan Okay, so this is the graphics screen an operator would typically see he has a direct control over manually Slowing down or speeding up this VFD So Jay go ahead and slow speed it up for us. You guys also see it turn here. Should be turning there. Hopefully Speed it up one more time All right. We're now we're just going to show that this exploit works Paul Can you slow it down for us and speed it up one more time for us? Now this is pretty dangerous I mean when you get direct control over a component as powerful as a VFD You're directly controlling how much material is forwarded to some other area of the process whether it be steam or coal or slurry or whatever So both upstream and downstream components can be affected by this. This is a very dangerous situation The second system we're going to simulate today is enunciators Jay's going to get that set up for a second My network. I'm going to disconnect first. Good. Okay. Enunciators tell the plant operators of problems They're kind of like alarms plants usually have few alarms flashing all the time just due to maintenance activities or Faulty equipment they know about these. They know which ones are false which ones are real Physical enunciators usually accompany the software enunciators you see up here today We're not going to Simulate those because we don't have the hardware but anyway We have to dump the program in and restart the application every time we do this so that's why it's taking a long Yeah, but we're almost there variable frequency drive What it does is it varies the frequency that's sent to a three-phase motor It also varies the phase which changes the direction you vary the frequency to change the speed They're very common in the power industry. They're very powerful very useful stuff They can be the model we have here does have a network connection. We're not using it today though Okay, so this is what she typically see on an operator screen You see a few alarms flashing due to maintenance activities or whatnot. No big problems And now we're going to show Paul has full control over the enunciator screens All right, so Paul just said hi to the operators Now you could do this you could say hi or f you or whatever you want to the operators But ultimately if this was a real attack that wouldn't be the goal The real goal would be to suppress the alarm so the operator can't see what you're doing in some other area of the system This would cause more damage with your attack No, no, no, he's just suppressing the operator from knowing about the problem Yeah It can because sometimes these enunciators are controlled the enunciators that are on the walls They're controlled by lights in the back. They're often controlled by PLC just like we have to hear today So if you get control of one you get control of another It's probably just the projector. I'm sure they have to see the controls. Otherwise they have trouble Well, if you get They see something like that they may just shut down they may trip the whole plant They may just stop the plant from operating because they have no idea what's going on They no longer have control over over the plant and that would freak out an operator right there Sure, okay, so the third system we're going to simulate real quickly is the bottom-ash system So what happens is you have a boiler a burner? FD and ID fans which are forced draft which blow air into the furnace and induce draft which kind of suck out The smoke in the ash You also have water tubes running down the side of the boiler and tubes hanging inside the boiler that all feet steam to the turbines There's two types of ash. There's a fly-ash and bottom-ash. We'll exploit the bottom-ash system today So I'll just give you a quick overview of how the system is supposed to work The hopper's filling right now. That's what the green means the fill valve up top is open Now the drain valve opens the hopper fill valve closes the crushers crush the slag That's melting down the sides of the walls now the spray valve opens and sprays the little bit of extra stuff That's stuck to the walls and the process starts over it fills again Etc. Okay, so now you know you're gonna see Paul exploit this system and cause a real mess Just a moment Okay, so now Paul is setting the state back to one the fill valve is open the drain valve is closed You saw what's supposed to happen the drain valve open for a second and closed again the fill valve open and You're just filling that hopper all the way up Eventually what's gonna happen is water is gonna get inside the boiler You could put out the flame possibly cause a boiler explosion at least hypothetically If you also have control over the annunciators the operator might not know about this If you have control over other systems, this might go completely undetected for you know many many minutes And you could have a real mess that could take weeks or months to clean up possibly fatalities It depends on the facility. It really does You know if it's a two megawatt or excuse me a two gigawatt plant it could affect the grid as a whole That's specifically what NERC compliance is focusing on is protecting the grid as a whole So if you generate, you know two gigawatts or more You got to be concerned about that It also depends on how how quickly it happens If it happens real quickly and you lose loot units completely and people are unaware of it for a little while It could take out a lot a lot of area Yes Right a lot of a lot of this information is publicly available a lot to be obtained by a social engineering You do need to know a little bit about the process I mean, this is one thing we're trying to get across to you all today. It's kind of a new concept called malicious process design Process design is something we do all the time, but designing it to be a dangerous process. It's sometimes it happens inadvertently We're kind of afraid that it might happen intentionally in the future the other problem with that Address your point is if you if you were at the talk earlier He talked a little bit about that how you have to know both sides Not just how to get into the system But what to do once you're in there the problem is that first of all if you're in if you're already on the network a lot Of this stuff you can probably find on there and there's all kinds of there's vulnerabilities There was one release recently About Siemens about how you get access to the essentially the process map So in addition to that it could be there were there have been instances where x employees I think there was one in Australia in a waste management plant who already knew the entire system It was a contractor knew the system. It's not impossible to know the the process and or if you don't know what to get that information right Yeah way back there Yeah, you unlikely But possible Different systems are designed differently after understand. There's like 5,000 power plants in the United States built between you know 1940 and a year ago Whatever you do it's gonna have to be plant specific. It will always be plant specific. They're all just designed completely differently There are quality assurance engineers that do that nuclear is much better than fossil right now, but It's starting to get better Sorry, I didn't hear you Yes, that is true That's true. We should have put a QA in there It's I guess in my experience QA isn't always required for things like CT sites combustion turbine combined cycles or even fossil plants Yeah We're trying to Well, typically designers are separate we design them or we lay out the basics they drop the the drawings What's supposed to happen is supposed to go to a QA? But in a lot of organizations QA is just like a mechanical engineer Who's reviewing control system drawings electrical drawings civil drawings, etc? So I mean nuclear is much better at this than than every everybody else You're I Think at least one or two of them already known about and their response was to you know Make sure it's running on a secure network to unload it on the client Yes, sir. Hmm I've heard of connections similar to that. I haven't heard of that one specifically. That's completely true I mean a lot of times they require that you have that connection there for To maintain your warranty so that they can monitor the operation of the turbine and make sure you're not doing something That's gonna gonna void the warranty and that's definitely a vulnerability that could potentially be be exploited It's something that a lot of times doesn't even go through a DMZ. It's just a connection straight to a vendor Yes, sir Agreed, you know nuclear already does that they're really strict about this stuff with fossil all of that stuff is privately owned so to be able to get agreement on that kind of stuff is Well, that's what it's an undertaking That's what these standards are trying to do the problem is is there's so many different standards spread out different organizations Sometimes they overlap and you don't know which one you have to comply with the most It's a real mess right now in terms of the standards that people are trying to comply with and also there's a lot of resistance Some people don't want to change You know, it's also very tough for the plant that was built 50 years ago that was modified slowly You know decade after decade. I agree Problem is is, you know, you know, one of the points Paul made is it's bad design I mean these PLC's just simply were not designed with security in mind They were designed to get get out quickly as possible to make the most profit and to give functionality good functionality and they do that well, but in terms of security there is none Yeah Well, there is there is somewhat a need You know, you've got so many power plants generating so much electricity each with a different amount then you've got renewable sources coming in now and When you look at the graph of how much energy we consume in a day, it's it's like this It's up and down so that all needs to be coordinated. How many megawatts is each plant generating which units go up? Which ones get priority? I Agree if there's a way we can do it without having a connection to the outside world, we should I Don't think we've determined how to do that effectively yet. I mean even in the olden days I think they use telephones or something, you know But yes, sir Right now my impression is that most of them are just straight TCP connections A lot of them are trying to start to put in things like that a diodes and that is a good effective thing to put in It's not enough It's a start I guess Yes, sir Right No, that's true. There are always mechanical interlocks problem is modern plants Or excuse me older plants weren't built to modern standards Also, if you have a 50 year old plant sometimes these mechanical interlocks go on inspected for years or decades You know for example pressure safety valves which are set to explode when the pressure gets too high You know if they get rusted they're very sensitive components if they get rusted that they're gonna fail Yeah, yeah Even if you have mechanical interlocks, I mean there's there's still a danger here if you put Hello up on the operator screen and his alarms, they're gonna shut down the plant and that thing takes a day at least to start back up again So I mean that's that's wasted revenue wasted time and then there's you're probably gonna get fine You probably got to go back and re-examine your network. So it's it's a huge waste of time Just do it right the first time and You don't even have to worry about your mechanical interlocks being safe. They're doing what they're supposed to do Any other questions? Yes, ma'am Mm-hmm It was in there a little bit further back Hmm That's That's true. That's another point. We're trying to make today is you don't need a cyber attack to have a real incidence on your hands I mean look at the deep water of horizon. I mean, it's it's terrible That's true an insider could use the knowledge disgruntled employees or just oblivious employees. You don't know what they're doing training is key But you're right I mean there's there needs to be more safety systems inherently designed in these process and they need to be maintained and I Don't think we're doing an effective job at the moment On the what? Hey Wade Do you want to wrap up and we can have people come up? Okay? Yeah, we're gonna wrap up real quick and have a QA session I think it's room right over there somewhere We can do it in here. Okay. That's what he said There's no one else in here. So yeah, we're staying here Okay, okay, we'll stay in here if you've got a laptop and you want to try connecting to the PLC And and do whatever you want to it. You're welcome to try Yeah, yeah, so we do have a Solution of these security flaws. We found it's called the tofino security compliance. It's like a two-point Managed Ethernet switch. It's got firewall protection. It's got a little bit of intrusion detection prevention as well as alarming It's designed by control system engineers for power plants industrial facilities If you want to put this device to the test and prove it works for us, we'd appreciate it I don't think I don't know if you guys can actually hack the network through it. We'll see Welcome to try