 Hi, everybody. Welcome to the edition today of Think Tech Hawaii, Hawaii State of Clean Energy. I will be your host today. I'm Derek Sonora. I'm substituting for Jay Fidel. And my co-host on this side of me is Les Taniyama. Aloha. And we have our special guest. Thank you, Jason Forrester, for coming on the show. No, thanks for having me. You know, if you've been watching this month's series, we've been talking a lot about cybersecurity. And are we prepared to move forward in time with our energy grid and all its interconnections and the vulnerability issues or the non-vulnerability... I'm nervous. You're nervous. Yeah, I'm nervous. Just a little bit. This is my first time I'm doing this. I got a request to do this from somebody who's sick late yesterday. So bear with me, everybody. Hope you feel better, Jay. So we want to talk about more about cybersecurity and where the state of Hawaii wants to go with with our renewable plans, doing this all by 2045. You know, we can get all the energy ready, but always cybersecurity ready is our topic. And Jason, I want to thank you for being here. I'm looking at your long list of credentials. It's quite amazing. Right now I noticed that you're here at Island Controls. That's great. And tell us a little bit about Island Controls and what you do there in your role. Okay, sure. Island Controls is an organization which does building automation primarily as well as AC service and supply. But on the automation side, what we do is we take the systems in a building, the air conditioning, the lighting, whatever it is in the building that traditionally would be controlled by a switch or would be controlled through a time clock. And we put these on automated systems which can be scheduled, which can address failures on their own, which can control devices to their peak efficiency and save a lot of energy in your building. And, you know, for example, even just turning the lights off automatically is an obvious one. But our system can adjust for the outside lights based on the time of year. So we don't have to go reset the time clock or worse yet leave the lights on when the sun's up. So we can just, you know, like 30 minutes before the sun goes down, we'll turn the lights on and it'll change throughout the year. So that's a simple example of the automation and how it can, you know, be used to help. It doesn't sound simple at all. If you think about everything you just said, that is a very complex system that a lot of people who work and live in a building are really dependent on it. So, you know, if I could be so bold, you're one of those unsung heroes that are out there that's doing a lot of good for the state of Hawaii and making a facility. And a lot of people who watch this show know that facilities everywhere throughout the state uses the most resources today as a one shop area. So what Jason and I just understood you saying is that you manage all of the expectations of how the building uses these natural resources. If I heard you right. Yeah, that's correct. So you have to use some resources to cool a building. You have to use some resources to light a building. Our goal obviously is to use as few resources as possible and still provide the same level of service. So we don't want to over cool a space. We don't want to provide lights when they're not needed. We don't want to run when a space is unoccupied. And so by using the controls, you can run only the amount you need to run. You can supply only the resources that are necessary. And in addition, sometimes just tracking your resources can tell you a lot. I put in water meters for a large local customer and it turned out they were using hundreds of thousands of gallons of water, which they knew about. What they didn't know is that one of the systems was broken and was using millions of gallons of water. Nice. And so the addition of a low cost meter informed that my customer that they were using a ton of water they didn't mean to be using. So sometimes it can be relatively simple, just providing some data. Sometimes it can be quite complicated. Les is an energy engineer that designs systems to use every bit of their capability. And so, you know, initially this system would turn on and then turn off. And then it would turn on but only run, let's say, as much as we need to. So we basically put a knob on it to run it like 50% when you only need it that much. And then Les came along and said, but if you ran it at 20% and then 60% and then did that on only when it's the right temperature, we could save even more energy. And so I translate Les's ideas and engineering into a set of controls that then run the building. So given the huge complexity and the task, and I'm assuming you have a lot of clients throughout the state of Hawaii. Yeah, if a malicious person wanted to come and attack these systems, how prepared are the systems that you touch to prevent this kind of malicious activity? Well, okay, so the thing to understand about the automation software is that it is primarily based around the traditional web server. You're, as an end user, you're going to be accessing this the way you access Google, the way you access Yahoo. Whenever you log in, you log in with the username and password. So these are traditional systems for the most part. So when you say user, who is that person? Well, it really varies lately. Traditionally, when we would install it, we'd have one user and that would be the person in charge of making the air conditioning work in the building. This is usually going to be a facilities manager or a person with a mechanical background. But lately in the last decade or so, the energy consortiums are coming and saying, look, how can we save energy? Well, and then there are also people who are saying, well, we want this to fulfill some other requirements too. We're not just going to be on or off. We're not just going to control for energy. Maybe we're going to also put this into some sort of a display. We're going to promote this. We're going to have a partnership with the energy company. We're going to have a partnership with this other vendor. And so now your customers for this data have broadened dramatically in the number of people who need access to the data, come from a number of different walks of life. They might be billing their customers. If it's a large organization, like a building or a mall or something, they might take the data we provide and then cut that up and pass that on to their customers. So there might be accounting customers. There might be management customers. There might be the guy who's using it to control and do the work itself, the facilities folks. But these are all users of the system and they all have a different view into that data and a different use for it. So let's take this in different levels or tiers. Let's say one building is one level. How important is it for the building management and the building owners to take cybersecurity of concern or issue? How do you feel what damage can be done if they don't? Give us the level of the degree of impact. Let's say money is tight, security is way down there in the owners and the property management line, but they're not paying attention to it and shouldn't there be what could happen worst case scenario with just one building keeping in mind? Unless you're in this world as well, if you have thoughts and comments about this, please kick on in on this one. Okay, so the automation systems shouldn't be thought of as really different from other systems. Your corporate IT systems have many of the same hardware, software, interaction, security demands. All of these systems should be treated relatively equivalently, but we find that in facilities in particular, they're very late to the game, not only in using the information technology, but in joining the corporate IT system that already exists. Most of these organizations have built themselves like an email infrastructure, a web-faced and web-facing infrastructure. They supply resources to their workers and so forth through the network, but these systems are often kept completely separate or perhaps not even, IT isn't even aware they exist, and that really should change. The possibilities are very bad if any system, whether it's the automation system or their email system, if any of these systems are not cared for, patched, backed up, secured, these systems all are vulnerable because just through the nature of being a system and being connected to the internet. Now you get a lot of benefits from being on the internet. Email without the internet, not particularly useful, but an automation system can be quite useful without connecting it to the internet. So there is a start right there. There is one way to look at it is, if you don't make it available remotely, you have far fewer vulnerabilities. But there's a lot to be said for getting a call and being able to fix it while you're traveling, being able to stop the problem without having to drive to the office. If you have a remote site, I have customers on different islands. If they had to wait for me to fly to their facility to fix a problem, that could be days. But I can often fix their problem in a matter of minutes logging in remotely. So the power of it, it makes it worthwhile. But you want to treat it like you would treat any public-facing service. Yes. And that's what's really not happening I think a lot of times in facilities. Well, what you just described, if you don't mind me just kind of reiterating is, the facilities persons today has to be an incredible person because this person is taking care of build-outs for putting up walls. This person is taking care of the plumbing, is taking care of personnel down there. They're doing lanterns outside with the landscaping. They're doing all of these things. And now what I'm hearing from you is this person has to have a level of sophistication in the IT realm as well. This person must make a tremendous amount of money more than the president or the CEO. And be very handsome and drive a very nice car. That's what I wonder because you're all around the state. How many of these individuals, these properties are equipped in their facilities to take on this kind of responsibility? Almost none of them. Well, I think that's where one of the major disconnects is with people like Jason having to put in a $50,000, $100,000 system that fully automates certain systems and gathering a lot of data. But it's the user, the operators that may not be up to the level of that technology in regards to understanding how to keep it online. Because how do you determine what is bad data? You want that data, but how do you determine if it's good or bad? Can that reflect you on your business decisions? The answer is obviously yes. So we come down to the people. Part of why we're here is to understand we need training, I think, as one of the forefront issues regarding cybersecurity. Because I think everybody in the building from the secretary to the janitor needs to have some knowledge about cybersecurity. Because everybody that picks up that computer in the building is susceptible to causing an issue. And it can happen and has happened. Look at the news, right? So because our show topic for the whole month has been linked to our grid and making that grid a smarter grid. Now, I'm going to think very negatively. I'm going to pretend I'm going to take some words from Washington, DC. Bad, bad people out there. What could bad, bad people out there do if they were able to break in, take control of many of our large facilities which use a lot of energy? Could they actually disrupt our whole grid? I'm just thinking like a bad, bad person. You know, that's always going to be the case with any system that is integrated closely into technology. The vulnerabilities in the technology are going to cause issues. In air conditioning, especially here in Hawaii, we don't have freezing. No one's probably going to freeze to death if I don't get the heaters on. And no one's probably going to boil if I don't get the AC on. So there's a tendency in facilities we tend to think, well, it's probably going to be all right. No one's really going to do any significant damage even if they got in. But the truth of the matter is that a lot of the hacking that goes on is not targeted hacking. It's more of a shotgun approach. There's a lot of scan every IP on the internet, see what's vulnerable, attack that, take it over. Now use that computer to scan every computer on the internet and see what's vulnerable, attack that. So non-targeted vulnerabilities are just as big of an issue. There's ransomware where they'll encrypt your files. There's just any sort of traditional hacking, spamming, any of these types of things are vulnerable on your facilities computer just like they are on all of your other computers. There's nothing special about that facilities computer. The vulnerabilities to the system for people being in it are going to depend on what you're controlling and to what level. Could I, for example, damage a piece of equipment? I could turn it off and on several times a second until I guarantee it won't turn off and on anymore. So I could probably do that. Is that going to cause the 24th scenario where the terrorists know it's not going to? But is it going to be damaging? Yes, it will be. Could I lose control if somebody takes over my computer? Yes, I might not be able to turn on and off my lights or my AC until I deal with that infection. But that's for your standard customers. Now, when you talk about the grid level, you're talking about a larger organization. On the mainland you're talking about multiple organizations. Here we really only have the main electric company. There are sponsors too, by the way, so be gentle. I have no problem being gentle. I have a job, especially with the amount of renewables on the grid right now, which is why they need to do the management that they're doing and they need to do that adjustment. They need to do that, you need the controls. Could the lights be taken over and shut down? Yes, they could. Any computer is a vulnerable computer. So you have to take steps to minimize your vulnerability. And for a business, some of those steps are relatively straightforward. Don't give everybody access. Don't give anyone access that doesn't need it. Don't make it accessible if you don't have to from just anywhere on the internet. I like to advocate putting a VPN in front. They're very affordable now. A VPN is a... It's called a virtual private network. And essentially what you're doing is familiar with the little router that you get at home when you plug the cable modem in. And that's how you get from your computer out to the internet. We're going to start to twist down as we kind of take a break here. We want to do a little kind of exercise that they always do. So I'm going to just kind of let this one go. Do I do this? Hi, I'm Jay Fidel. That's Ted Ralston. You know, Ted is the host of Where the Road Leads. It shows every Friday from 4 to 5 p.m. It's about technology. It's about how people collaborate and solve problems with modern technology. It's where the road leads. We all know that. We should all be listening. Join us there. 4 to 5 p.m. every Friday. Now what about that you agree with? All of it. I knew we'd say that. Aloha. Say aloha. Aloha. Good. Hi, I'm Stan Energyman. And I want you to be here every Friday. Noon! ThinkTekHawaii.com. Watch the show. Be there. I pity the full way. Thanks for watching ThinkTekHawaii. And look forward to seeing you at Education Matters on Tuesdays with me, Carol Mon Lee. Hey, has your signal just been taken over or am I supposed to be here? This is Andrew, the security guy, your co-host on Hibachi Talk. Please join us every Friday on ThinkTekHawaii. We're back at ThinkTekHawaii. Before we continue on, I do want to take a moment to thank our sponsors. Our sponsors for the show are Hawaiian Electric, Hawaii Energy, D-bed, and of course Hawaii Clean Energy Policy Forum. So we have Jason Forester here. My co-host is Les Taniyama. And I really want to thank you. We've been having a great conversation. And right before we took a break, you know, I was mentioning some of the things about taking control of multiple buildings. And I wanted to just share a really quick story. When I was a youngster and we all stayed at my grandparents' house and there was a lot of grandchildren, a lot of cousins. And my grandparents were fortunate enough to have like three showers. So what we would do after we all went swimming in the next-door neighbor's house, we all take a shower at the same time. And the boys used to gang up against the girls. And what we would do is we turn on all the hot water in our shower and the girls would scream because now they only got cold water. And then we play with this kind of thing with the pipes. And I kind of equate the pipes with the electricity. So if a malicious person wanted to start up all the chillers, for example, on a Saturday night, when Hawaiian Electric isn't expecting this, and then they turn them off 20 minutes later, what would happen to the stability of the grid? I'm just being like the bad, bad mind. Just something simple. Yeah, you know, that one probably wouldn't be, and the grid is pretty large. It would be very difficult to affect enough people at the same time to affect the grid. Okay. You know, I think that if you were interested in, if you're trying to damage something, you know, the, I don't know, it seems to me that the damaging, the straight damaging, would be sort of a limited win for anybody who's doing it. It would be, you would do it once. You would cause damage one time. Someone would find out that you would cause the damage. Yeah. And fix the hole. It would be a one-time shot. It would be a one-time shot. It wouldn't really seem to be a very useful thing. And that's why I think we see more of these cyber attacks, if you will, taking over, collecting the data, collecting information, using it as a stepping stone to other systems. And that's where I think the facilities, you need to pay attention in facilities, because when you install these systems, they are, depending on how they're set up, they could be connected to your primary systems. And now there's more and more demand for the primary systems, like accounting, to get the data from your automation systems. So they might even be somewhat integrated. I see. And that user who has the login might use the same login and password on the automation that they do on the accounting system. So you can, you see where those sorts of vulnerabilities to me are at a larger risk than somebody got in and flipped something on and off, caused damage and so forth. It seems like that would be a more rational motivation. And we've saw that, the most famous one would be Target. Target had a very highly publicized vulnerability because their automation system was on the same network as their point of sale. Now had they segmented their networks, there wouldn't have been a vulnerability when the automation system was hacked. It wouldn't have been a vulnerability to the point of sale system. So I like to promote segmentation, preferably physical segmentation, separate networks entirely. If you can't, then a virtual segmentation of the networks where the networks don't communicate even though they might share the same hardware. And we'll see that in a large organization which already has networks in all of their buildings, that they'll provide a network that is only used for the automation. Even though it's all on the same device, I can't talk to the other devices. That sounds a little beyond the facility, guys. Which you're right. You're absolutely right. And that's where, again and again, I would encourage the customers to work this through with their IT folks no longer to treat your automation system as being part of facilities. A lot of the times the customer will come to me and say, can I buy a computer with this system? And I'm like, you could, but would you normally go to your air conditioning company and ask them to sell you computers? And I would suggest that you work with your IT organization. Now you're going to have a supported computer that matches your corporate standards. It's going to have your patches. It's going to have your property tag. It's going to be on the inventory. Most importantly, it's going to get backed up with the other important IT systems. That's very important. If you don't have a current backup that's been tested, you probably don't have backups. And at some point, hackers aside, you're going to have a hardware failure. You're going to have, you're in the facilities down there. If you have this computer under a desk and it's in the Chiller plant and the water pipe breaks because there's lots of water pipes. Now your computer is floating away. All your data is floating away. You don't have any backups. These would all be taken care of as part of the procedure if it was just part of IT in the first place. I want to give less of a chance over here to say something. One of the big issues that we always talk about is the disconnects. Uniquely, the decisions that are made at the facility level as to how the system operates many times, in fact, most often than not, it's done by mechanical engineers that design the system. They're dictating what they want the system to do and they're giving you the rules of which to follow, of which you're trying to program for. Now, that's one of the things. Rarely do I see in today's specifications coming out of the engineers is IT concerns or cyber concerns and what to do. And so, you know, that's one of the areas. The other thing I like to touch on is when he talked about target, that's a very, very famous one because there was a building automation system that was the root cause of the issue. But here's another issue. If I was the building engineer and I used a laptop for my work and I also go into the building automation and also communicate with administration, with that same computer. Is that a problem? Because isn't communicating with all these different sectors within the building as well as surfing the net going to be an issue to the security of the building, especially the assets. And I'm talking about the financial assets. Yeah. Well, and we obviously we don't recommend personal use computers and work use computers. And you know, you see that more and more in the office now as a computer is a more affordable thing. It's no longer like this is the only computer I'll ever have access to. People have a device at home. They've got phones and pads. They've got network at home and so forth. So the part of the problem used to be, well, you know, I've only got one computer. I've got to do my banking and I've got to order something from eBay and I do my word processing all on this one device. But now we like to encourage everyone to segment their usage, to do purely work on work devices. And this isn't just for automation either. This is all around. We have, everyone has seen the problem. You know, you're using it for Facebook. You go on Facebook, there's an infected game and infected ad or something. Now your computer is infected. Now it's looking for other computers on the network to infect. But you're at work now. You've taken that into work. And now that's another virus propagating vector, right? Here's another big picture thing. Since we're talking about energy. A minute or so. Sure. We have data. We're attaching meters to the building to understand how much we're using, when we're using it. Can those numbers be changed and mimicked through spoofing of sorts and to misinterpret? You know, that's pretty deep. But obviously those numbers could be changed. Could they be changed in a malicious, useful way? Like reducing your bill. Could I log in and change my bill by a decimal place? You know, those systems have a lot of security on them and they've been worked on for many years for that reason to make sure that they get every penny, right? But we need to treat our systems as though they are as critical as that. And we need to do that with some basic hygiene, making sure your usernames and passwords are different. We've got a little bit time left. Is there something that you want to come across and say, hey facility guys, do this one thing to help you guys right away? Definitely get your IT folks in there and plan this going forward. Don't wait for it afterwards. Don't let somebody come to you and say, OK, look, we're all done. Now we're going to put this system in on top of it. And then after we're all done with that, we're going to come talk to you a little bit about security at the end. Make it from the start, design it in from the start. Jason, you've been a great guest. This is a great topic. We could talk all day about this. We probably could. And I want to thank my co-host, Les Tanayama. It sounds like we need more discussions in the future about the facilities. Les has been harping on this for a year now about facilities and the criticalness. So we need more discussions about this. Ashway on Friday. Ashway on Friday. Ashway is doing lesson seminars. Some talks about energy, energy efficiency, and things related to everything automation.