 Welcome everyone to my talk today, ICS security operations, Active Defense Concept with Effective Instant Response in Industrial Control Systems. This is John Krunas. My name is pronounced as John, which is pretty close in English and John. And I am working as an OT Security Consultant and I have GICSBO, ICP, OSWP, and Certified Ethical Hacker certifications. My bachelor's degree in engineering and my master's in cybersecurity and you can reach me on Twitter and LinkedIn and GitHub as you can see here. I am of course a bit nervous about the presentation and let's take a deep breath and have some relaxation, then we can continue. Okay, what's today's agenda? So I will discuss a bit on Active Defense Concept and I will explain briefly threat intelligence, set identification and network security monitoring, instant response for ICS, and then taking the advantage of found threats. So what is Active Defense Concept? It is something like this, yes, improvising, adapting and overcoming as we agree. But yeah, it is also related to this picture. But in general, what I can explain is somewhere in the middle in here in the picture, it is the process of analysts monitoring for responding to and learning from adversaries internal to the network. So basically that means it is a combination of analyzing the adversaries as steps and monitoring constantly to the internal network. And then when you see any animal is responding to that. And basically when you see any action, basically create lessons learned and learn from the adversaries. It is not deployable by itself, of course, it is built on the top of the good practices such as good architecture and then the passive defense. And it's not definitely a hackback. So you need to stay on your internal network to protect and you don't need to, you don't have to attack back to the adversaries, of course. So why Active Defense is needed? As we can see, at least for a couple of years, traditional methods of protection without constant human interaction such as firewalls, IPS and antiviruses only provides a certain level of security. And adversaries are getting stronger and stronger every day and the passive defense usually don't stop them, unfortunately. So let me take a look to the Active Cyber Defense cycle. It consists of four elements, threat intelligent conception and asset identification and network security monitoring. Then incident response when it's necessary and using threat and environment manipulation to get over the adversaries. Basically, this diagram is coming from the suns and references down there. So basically the idea is monitoring your area of responsibility which is quite necessary to at least have the, let's say, the idea of getting the baselines then checking the anomalies. And that said, implementing monitoring will bring you the chance for quick response when it's necessary when you see something odd. And responding to incidents and attacks on time is of course crucial because especially on the industrial control systems, if it affects the industrial control network, it's going to cost you a lot. Constant changes are necessary since you need to beat the adversaries and kick them out of your industrial control network. And after that, you need to share and you need to consume the lessons learned. At least you need to share with the community and you need to observe and collect the lessons learned from the community to develop better defenses on your systems, on your ICS. What is threat intelligence? I'm going to, I'm trying to explain this pretty quick. Let's start with the intelligence. So intelligence basically means the process of collecting data, turning into information and producing an assessment that satisfies a previously identified knowledge gap. So you need to obtain from the raw data to important information. So normally you can do, everybody can say, you can do this with the tools, but unfortunately tools can only help and your analysts can create intelligence, not the tools. An intelligence life cycle, when we take a look to the intelligence life cycle, it starts with planning and direction, then collection, then processing and exploitation. Exploitation here means doesn't look like the exploitation in the computer science. It is more like clarifying and filtering the information that you need that make it useful. And then analysis and production and dissemination and integration. So basically the intelligence life cycle consists of these elements. And then if we, if we discuss about the open source intelligence, it is a type of intelligence, which is quite useful for adversaries, as well as also for you. It is low cost and low impact, but it gives essential information about you. So most of the time adversaries are googling on the internet checking related documents and data about you. You can also use this as your own beneficial by checking these documents and trying to reduce your threat landscape. Normally these sources can be public relation documents or partnership announcements that you share. Most of the time these documents are not considered dangerous, but they are dangerous because they contain valuable information. And company information, of course, the, if the company is a public company and if the company shares monthly or quarterly or I don't know any reports. These reports also have valuable information about the company. Most of the time job descriptions. Basically the leak the technologies, of course. And advisories and alerts sometimes also might be related to your systems and give some hints about your systems. And of course, your internet connected devices. These are the most most dangerous things. So threat intelligence if you continue with the threat intelligence in simple terms again threat intelligence is a combination related to adversaries that have intent capability and opportunity to harm you. So that specific type of intelligence give defenders knowledge of the adversary their actions within the defender's environment and the capabilities as well as their tactics techniques and procedures. And the threat intelligence is often shared in different methods. For example, indicators of compromise IOC as I mentioned tactics techniques and procedures TTPs and oftentimes complete report about an incident or an attack. And what do we need to do to identify our threat landscape. The threat landscape is the combination of information attack space, threat groups, industry and non technical influences. So basically to identify your threat landscape you need to ask yourself who is interested to attack your organization and why. What would be what would be the reason of causing damage. And how can they get into to your organization. To identify the threat landscape you need to reduce the threat landscape. And most of the times a set and network identification is necessary to succeed. First of all, you need to clarify your external and internal landscapes because you need to focus on your responsibility areas. So as you need you would have some connections from third party vendors or any other third party service providers, etc. Then using known information. You need to try to reduce your threat landscape, basically by checking these information and trying to prevent any leaking information like that. And performing assessments and measurements also with the third party companies or external companies because sometimes you need to have a second opinion or second eye to check that and after that you need to continue doing these again and again to reduce your landscape. And external threat threat intelligence is the threat intelligence comes from the specialized teams which can be a professional services companies, and such as regals. And, or it could be from ICS search and other search, basically provides indicator of compromises and TTPs. And most of the time it is for large audiences. It might not be relevant for you, but still, it is important to follow these threat intelligence reports. But sometimes of course, the industry is not relevant for you and you don't need to take an action. Oftentimes ICS threat intelligence are limited. It's really hard to find, especially purely ICS threat intelligence but when you find it is really priceless. And ICS search multiple Isaacs and all the Isaacs can provide some internet storm center also providing threat intelligence threat feeds and reports from vendors and professional service companies, as I mentioned like from Monday and from from Drey goes, for example, and what is internal threat intelligence and internal threat intelligence comes from your data. It is usable against the current problems in your environment because the data is just coming from your environment and after you analyze, you basically realized there are some problems and it's tightly relevant to your issues. For example, it requires personnel, in order to do the analyze analysis, and it requires specialized skills such as malware and threat analysis. If you summarize threat intelligence, tactical threat intelligence contains TTPs and IOCs which are quite useful to add the change and implement your defense, or at least search for intrusions in your environment. And strategic threat intelligence can be used to look overall patterns, suspected attribution, trends and teams. It is also useful to identify where your defense might fail, or already failing. So if you go for asset identification and network security monitoring. The purpose and importance of the asset identification is, it is really hard to defend if you don't know what you have in your plant, in your facility or in your environment, in general, because if you don't know what's there, you cannot protect it, basically. And advanced security solutions that you have also not effective if you can't provide the whole map to defend. So you need to at least provide the fundamental security to to have this and network security monitoring threat intelligence and instant response usually works better if you have the network knowledge if you have internal network knowledge. And how do we identify the assets in the ICS network then we can say, first we need to start determining the area of responsibility because as I mentioned, most of the times there are some remote connections that you need, you shouldn't include. Because, yeah, of course, VPN endpoints on your side that can be count on yours on your responsibility but still, there are some ways that goes to outbound and not in your responsibility anymore. And then finding and utilizing the existing information. For example, if you already have some network diagrams that you have you need to start with this then you can validate no non information. And if it's not sufficient and if you think you need to, let's say, create a new one you can try to collect by doing physical inspection which is normally a bit harder and takes much more time. You can do traffic analysis in your ICS network. You can do traffic analysis that we can also say, we can do configuration file analysis. And we can do active scanning which is not recommended. Except your, let's say, you are not in the production, not in production or if you are on the, if you're in the turnaround, you can do that but otherwise it's a bit dangerous. Collecting the data in all the assets you need to document it again. In order to have nicely done diagram and have the old assets. And unfortunately, most of the ICS networks are flat networks, you will notice that once you once you've done the asset identification. If you find the assets, it's often required to have a physical or logical separation or both. It is quite necessary since it makes makes more difficult adversaries to pilot in your network, otherwise, if it's flat network. Normally, game would be over within let's say, half an hour or one hour for the active defense concept it is really crucial, because you need to build on top of the good architecture and passive defense. For me the aim would be separating network as explained in purge model. So it could be a good reference to start with. And then if you want to have more more separation then you can micro you can use micro segmentation, etc. But it is the fundamental for the ICS networks and network security monitoring is a continuous process. You can collect, detect, analyze indication of threats in order to respond faster to incidents and attacks in your network. It is threat centering approach instead of traditional vulnerability centering approach. Network security monitoring requires dedicated personnel as analysts. And it also requires preparation for infrastructure and also tools ahead of time. And it brings proactive approach to security and detection of the threats so you can you can take some remediations. Once you identify the issues in your network and network security monitoring provides visibility and helps you to identify the changes and anomalies. This might not be relevant to only adversaries but also some misconfigurations for troubleshooting misconfigurations. And it helps you to detect intrusion attempts and movements between skater ICS skater networks. And it can evaluate separation and segregation for levels that you have and zones that you have. You can also find tuning and evaluating the settings of passive defense elements, for example, checking the firewall rules and to see if it's correctly implemented or missing some points, etc. And it also can help reduce the threat landscape by hardening non-required ports and services since you are seeing this when you are doing the network security monitoring and you can instantly take remediation. And network security monitoring is more useful in ICS networks since all the assets are, most of the assets are critical. And most of the times all the assets or any assets cannot be immediately patched. Because, yeah, we know ICS networks are quite, let's say untouchable, and then you need to have any other layer to have the protection for your endpoints, etc. And ICS networks have many dependencies and connections such as enterprise network connection, vendor connections, business applications, contractor VPNs. And it is, it is really valuable to monitor these external parties for anomalies because once your vendor or once your contractor, let's say compromised, it is just a moment to jump into your network by using these VPNs and normally you don't have any connection on these connections without the active defense. And detection approaches. Most of the time, what we check is identifying the most use IP addresses and ports used within the network to create basically a baseline. The biggest bandwidth users identifying encrypted communication because most of the times command and conquer centers are using encrypted communications. And identifying the critical assets and usual traffic is also important to continue on the on the baseline to have a stability on that. And identifying network anomalies of course, and identifying lowest bandwidth and communication, because sometimes you can see in the network unusual traffic, which could be pretty less than your usual levels. And of course you need to identify ex filtration. You, you need to check that. And asset identification and network security monitor takeaways, asset identification and network security monitoring is the key for active defense concept. And network security monitoring is a great approach for ICS because most of the ICS networks are quite stable. And basically you can once you create a baseline, it is easy to to to to, let's say, monitor. And these two elements supports implementation of better architecture and passive defense. Once you implement that. Yeah, you can take a look back and you can, let's say, improve your, your basic fundamental security also detection often relies on the sensors and ICS network. You need to be sure that you need to detect the baseline changes anomalies and ideas rules. Because these can basically warn you when you are doing the detection and logs and visibility are really important. But yeah, you need to have an analyst to, let's say, contribute value on that because by itself logs and visibility doesn't give you the chance that that you can prevent. An analyst need to follow it constantly and verify it if there is any animal. And network security monitoring will eventually lead to instant response, you need to be ready for instant response by saying that instant response for ICS. It is a bit different comparing to usual it instant response because you can simply, you cannot simply bring down the systems during the instant response when you're doing when you're working on ICS networks. It's a bit different by saying that maintaining safe and reliable operations is the most important team, acquiring meaningful forensic data within limited time, performing timely analysis and containing and educating the threats. Basically, these are the four elements that you need to follow when you're doing the ICS incident response. And before that preparation a there are some some steps that you need to follow preparation, same as traditional IT but some limitations such as testing tools, testing methods, but in a lab. Integrated detection and identification working with network security monitoring team to implement rules and detection capabilities, which is tailored to your threads in order to identify impacted systems and evidence in acquisition acquisition. Normally you, you don't have that much time like on it site, you don't have that much time to deep forensic analysis, because you cannot stop the operations. Here the focus is maintaining operations while acquiring enough evidence to perform later. And time critical analysis using fast and well tested techniques to quickly determine the overall impact to the operations and support activities. The other teams mostly share information and evidence, all the evidence should be passed to other teams to begin deep analysis and continue on the active defense cycle, and containment present preserving the operations by collaborating with other teams operators and also engineers on the field. In the recovery, you need to neutralize the threat by, for example reimagining the system reinstalling known good software implementing patches. And then you need to provide lessons learned, which is documents, finding for all past information to network security monitoring team, in order to identify and see if there is any reinfection etc. So you need to prepare an instant response team. You need to determine the requirements and dependencies within your facility. For example, up time availability and specific systems you need to take care. You need to decide if it's going to be in house or outsourced. There are some advantages on both sides. But you need to decide at least have a couple of people in your facility to do it more faster. A team size 34 well trained people on site would be enough to cover but you also need to think about the shifts etc. So in total it needs to be at least 8 to 10 people. The chain of command is really important because there will be chaos and someone needs to communicate with the management team etc. And it should be instant response director. Then all the evidence and instant handlers need to report to lead responder, and the chain of commands needs to go up and down like this. And to build your ICS instant response team you need to find the right personnel. Of course it's the hardest part. Because instant response is really really tough subject to focus and develop yourself. And then also you need to take care of your jump kits because often jump kits, when you need it during the incident. Sometimes you are losing some parts from the jump it's it's easy to borrow from the jump kit. And then when you go to when you go to the incident, you are missing couple of hard disk etc when you need it on over there. So, it's, it's important. The evidence acquisition, sorry. You need to take care of the order of loyalty. And you need to decide if it's going to be local or remote acquisition acquisition. I would prefer local because it's less risk and more, more fast faster tools should be test before on on on the systems because we are not talking about regular IT systems. We coordinated with all involved personnel and you need to discuss beforehand when you are touching in the field device with engineers and operators. You need to gather all the evidence, if the time permits, you need to start from registry and any, any memory and then taking the disk images for example, and you need to take necessary photos. You need to see on the on the let's say devices. If you see any command prompt you need to have an evidence because once you turn it off, you will lose it or you can lose it in any second, and all the data should be analyzed in an approved facility. That's for the forensic data in ICS networks, highly volatile data system memory network information and system processes and VPN connections and these logs and register hypes, HMI mostly Windows computers and you can have system logs, etc. In engineering workstations, controllers such as PLC and R to use, and sometimes virtual resources, or such as VMs and cloud environments, if it's connected. And you need to do the high quality instant response in a timely manner. It's, it's really important ICS when you are doing this timely analysis is important to keep operation safe and reliable. You should focus on understanding the scope and then the type of incident and the baseline information comparison will be really helpful when you do that. And focusing on the new connections increase increase in the bandwidth new new routes anomalies in the VPN connections, change register keys, spawn processes, you need to check these first. Identification and network security monitoring help to respond quickly, and utilize good tools to reduce the analysis period this is also important you need to do the practice before going to the real incident. And how do you use threat intelligence in the instant response indicator of compromise to scope the infected systems. Then, after that, you need to identify the network data on host and TTPs to identify adversary effort efforts, and you need to utilize to ensure that the threats are gone after you done the incident response and it's a response takeaways. You need to focus on providing actionable information about the scope of the threat, and it's potential impacts while you're acquiring the evidence without breaking the security safety. ICS instance response should be tailor made it's a bit different than the it, as I mentioned, efforts must align with the goals and requirements of the operations, this is the first priority preparations should be done ahead of time. You really need to practice before it really happens, and acquired evidence and lessons learned should be shared with other personnel. So once you get evidence you need to share it to complete to move forward the, the active defense cycle. Let's summarize this taking the advantage of the phone threats. Once you complete the response. If you can safely interact and understand with the threat, we can have the best source of defense information from the adversary's best capability. To defeat the active cyber defense center that with the found threat intelligence to build better defense. It is important to identify and use indicator of compromises from the threats to help incident response. And understanding the malware tactics to identify weaknesses in the current ICS architecture is valuable because after that you need to improve your architecture. So these these steps are really valuable. And my last presentation is about lessons learned sessions for long term success for the success of active defense, active cyber defense cycle lessons learned should be shared within the teams internally, and also necessary actions needs to be taken to build better And after that, either during the during the incident and after the incident. And after that, once you shared internally. It's also recommended to share your lessons learned in appropriate way with the ICS community because if you had that threat. Someone else might might get this soon so it is really it is really important and valuable. And yeah, I think I'm just on time. Thank you very much that for it for listening me if you have any questions, either you can send me a message for LinkedIn or Twitter or ask out. Thank you. John, this is in there. Okay, so everybody look forward to see john and ask questions and all sorts of different things. On the DEF CON discord server we're in the ICS Village subgroup. And john, thank you very much that was a fantastic presentation. Thank you.