 Hi, this is project obsidian threat hunting kill chain three. The logs are gone. Who am I? I'm extreme paperclip digital forensics nerd Linux geek infosec engineer lifelong student of everything amateur history buff spice fanatic loads of fun. Here are the things we're going to go over. We're going to go over how to develop a threat hunting hypothesis. Sources we could use for developing a threat hunting hypothesis. Research the methods attackers use to clear the windows event logs. We're going to find data sources we need for a threat hunting hypothesis. And we'll find evidence of an attacker clearing the logs. And we'll answer the how of our hypothesis. And finally, we'll, we'll go over how to document your threat hunting using a simple template. Okay, quick note about note taking pro tip. During and throughout the threat hunting process one should take good notes. These notes will be very useful throughout the engagement several times throughout this exercise, I will be sure to mention that some fact or discovery, we find should be added to the notes. This is generally a good habit to embrace in the world of information security. Okay, so what is threat hunting. Many people have their own interpretations and definitions of this but we could say threat hunting is the proactive practice of discovery in data. And it allows you to discover and uncover not only malicious activity, but new risks detection methods, or even visibility gaps that your organization was previously unaware of. So question. What guides the development of a hypothesis for the purpose of threat hunting. Here are some things to consider and take note of when building a threat hunting hypothesis. The IT environment. What are the operating systems in the environment windows Linux Mac. Does the org have active directory. What data gets logged windows event. What are the vulnerabilities on Zeke also vulnerabilities does the organization perform vulnerability scans. If so, are the vulnerabilities analyzed and remediated. Are there outstanding vulnerabilities that still need to be remediated. The MITRE attack framework offers a wealth of great information that can help build the threat hunting hypothesis. So, take some time to look at the enterprise matrix. And within each of the tactics are techniques that attackers use these techniques can be used to help inspire your threat hunting adventures. Also, what can your users employees coworkers tell you about things they have seen in the environment. As the threat hunter, we would add all of these sources to our notes and continue. Okay, in this scenario, we will consider the question. What happens when an attacker clears the logs in an effort to hide their tracks. So we should start with a very broad hypothesis. Let's focus on that middle section of that question. attacker clears the logs focusing on that we could make a broad hypothesis like attackers will try to cover their tracks. This hypothesis is broad. It only addresses the who and the what, but not the how. For the notes as the threat hunter, you would add your broad hypothesis here to your notes and continue. So how can we develop this further into a specific hypothesis for our hunt. Well, looking at our notes. We see that the MITRE attack framework was listed as one of the things we could use to help build a threat hunting hypothesis. And how would we do that. Well, a quick Google search for MITRE attack attacker covering tracks brings us to the following MITRE attack page. Indicator removal on host technique ID T 1070. Let's follow that link and read what MITRE attack says about this. And remember in our scenario we're looking for information about Windows event logs being cleared. We can clearly see something more relevant to our scenario in the sub technique listed above. It's the T 1070.001 clear windows event logs. So let's take a look at the details of this sub technique. Here we can see the details of the MITRE attack sub technique clear windows event logs. Note the methods listed here windows commands using w e v t util cl commands. Let's see there's also the event fewer GUI and PowerShell. So for our notes as the threat hunter, we would add the above MITRE attack technique, the sub technique. The attack methods listed and any source URLs, we would add this to our notes and continue using the info from the MITRE attack sub technique that we just saw about the specific methods attackers will use to cover their tracks. We can update our threat hunting hypothesis to be more specific. So let's take a quick review of how we got here we started with a broad hypothesis. Then we did some research and found some details from the MITRE attack framework to enhance our broad hypothesis into a specific hypothesis. So now we can make a specific hypothesis, like attackers will try to cover their tracks using one of the following methods. The event viewer GUI windows commands such as w e v t util cl or PowerShell. So for the notes we would as the threat hunter we would add this these specific items, this we would add this specific hypothesis to our notes and continue. Alright, so let's talk about data sources. You can know your data sources when conducting a threat hunt. So, which data sources could you possibly use. And where would we find the data needed for our hypothesis. Let's take a look at the data sources available to us in Splunk. A simple Splunk query. We are about to run basically does is to search the Splunk events event count. And then it's going to split the events by index, and that's the summarize equals F. F is for false. Then it's going to search for any index. That's index equals wildcard. And then we're going to dedupe the results by the field index using dedupe index. And finally we're going to specify the fields we want we're going to say fields index to just show the field index. And looking at the results, which of these indexes would be useful to us. So with the list of sources here, these indexes. We're most likely find what we need within the system on and win event indexes, because these data sources are going to show us what was happening inside a host. So for the notes as a threat hunter, we would add the Splunk indexes system on and win event logs as data sources to our notes, and we would continue. To begin our hunt, we should look for evidence that the Windows event logs were cleared. A simple Google search for Windows event logs cleared gives us a clue, a Microsoft article about Windows security event ID 1102. So let's take a look at this article. Windows security event ID 1102 the audit log was cleared. The article paid close attention to what Microsoft says for security monitoring recommendations. It says, typically, you should not see this event. There is no need to manually clear the security event log in most cases. We recommend monitoring this event and investigating why this action was performed. That's interesting. So for the notes as a threat hunter we would add this information about Windows security event ID 1102, including any source URLs, we would add this to our notes and continue. But question. Which data sources. Should we use going to pause here either. Yeah, what's up. Sorry. I might want to redo this little section right here because I just noticed that discord popped up and probably got into my screen. If you want to back up a slide and yeah, yeah, I can splice it together. Sounds good man sorry about that. Okay. I'm about to start. Just give a three second pause and then start. Okay, so I'm going to say right now, three second pause starting now. Okay. Windows security event ID 1102. The audit log was cleared. In this article, pay close attention to what the Microsoft article says. For security monitoring recommendations. It says, typically, you should not see this event. There is no need to manually clear the security event log in most cases. We recommend monitoring this event and investigating why this action was performed. Interesting. So for the notes as a threat hunter, we should add this information about Windows security event ID 1102, including any source URLs to our notes and continue. Question. Which data source should we use. So looking at our notes, we see that we have when event logs listed, which contains the typical Windows event logs found on a Windows operating system. And this is listed as a data source in Splunk. So let's go search for this event within our Splunk data. In this search, we're going to search the wind event logs index in Splunk for event dot code 1102. And we'll see if we get any results. We do. So we get 26 events here discovered showing proof that the Windows event logs were cleared. So the notes as a threat hunter, we would add this broad Splunk query, including the date time specified for the query, as well as some details about the results to our notes and continue. So, let's alter this Splunk query to make it look more presentable. So what we're going to do is we'll search the wind event logs index for event code event dot code 1102. And we'll use rename to rename the long field when log dot user underscore data subject username as simply user, because that's much nicer, right. We'll arrange the results in the query using table, and we'll specify the fields we want displayed time host name event code when log task user. And then we're going to sort the results by time from oldest at the top to newest at the bottom. That's just my preference, honestly. Okay, wow. Here we can see evidence of the Windows event logs being cleared on several hosts. Also, it's interesting to note that the search results here did not specify. We can also show if the event logs were cleared via the event event viewer GUI Windows command, such as W E V T util CL, or PowerShell, but in any case, we see that the windows windows event logs were cleared. So we would add the notes as a threat hunter, we would add this specific splunk query, including the date time specified for the query, as well as some details about the results to our notes, and we would continue. Okay. So are we done. Did we accomplish what we set out to do. Yes, they're not entirely wrong. We did find a way to detect when the windows event logs were cleared. But what if we wanted to find the source of this activity. What if we wanted to find which command or commands the attacker made of you might have used to clear the logs. What if we wanted to find the answer to how the attackers cleared the windows event logs. Let's explore further to answer those questions. Okay, looking at our notes. We can see in our specific hypothesis that one of the methods attackers used to cover their tracks is by using the W E V T util CL command. We should search for this command in our splunk data. But how do we do that, how would we find a specific windows command that was executed on a host. Google log windows command line and see what it gives us. And one of the first results is a Microsoft article command line process auditing. So let's take a look. Reading the Microsoft article. It mentions audit event ID 4688. So what is this audit event ID 4688. Let's find out. Quick Google search for audit event ID 4688 leads us to an article titled 4688. A new process has been created. So let's take a look. Okay, one of the fields we see in this windows security event ID 4688 is the process command line field. So it appears we can search splunk for the windows security event ID 4688. And we can specify the value. We are looking for in the process command line field. Right. For the notes here, as a threat hunter, we would add this information we have found about windows security event ID 4688. So we're loading any source URLs to our notes and continue. Okay, let's take a look at windows security event ID 4688 in splunk and see what the process command line field is going to show us. So this is just a simple search. We'll just search the win event logs index for event.code 4688. Now bummer. So looking at the results, we don't find any data for the process command line field in windows security event ID 4688. So for the notes as a threat hunter, we would add this specific splunk query, including the date time specified for the query, as well as some details about the results to our notes and continue. We didn't find any command line data in the windows security event ID 4688. Why? Well, according to the Microsoft article previously mentioned about a new process has been created. If you scroll down a bit and read, it says you must enable administrative templates system audit process creation include command line in process creation events. So you need to enable this in group policy to include command line in the process creation events. So now what are we dead in the water. Is there anything else we can do to find what commands the attacker used to clear the logs. Okay, checking our notes. We take a look and recall that we made note of the two data sources sys one and win event logs. So let's go take a look at the sysmon data. Okay question, which sysmon ID could possibly show us the command line details. A quick Google search for sysmon event showing command line leads us to an article about sysmon event ID one process creation. Okay, we see in this article that it lists the fields in sysmon event ID one. And two interesting fields pop up here. Command line and parent command line. So for the notes as a threat hunter we would add this information we discovered about sysmon event ID one, including any source URLs, we would add this to our notes and continue. Yay. Okay all is not lost. We have sysmon event ID one process creation, which shows us the command line, as well as the parent command line. Okay, let's take a look at sysmon event ID one in Splunk. We'll search the system on index for event code one. One important note here, this simple query, this simple Splunk query is only a query to see what the command line data looks like from sysmon event ID one. The actual values that we see in the command line in this specific search here, these results, they don't matter. Understand we just want to verify that we can see command line data. And here's where we get. We can clearly see data for the command line and parent command line fields and sysmon event ID one. Great. Now that we've verified that we can see command line values in sysmon event ID one. Let's search for the specific command. We're going to run a splunk search in the sysmon index, looking for event dot code one. We're going to specify process dot command line contains the command w e v t util cl nothing bummer. Okay. As the threat hunter, we would add this splunk query, including the date time specified for the query, as well as any details about the results to our notes and continue. So, again, are we dead in the water? Is there nothing else we can do to find what commands the attacker used to clear the logs? We're taking our notes. We take a look and recall that we made note in our specific hypothesis that attackers have also been known to use PowerShell. So let's move on to PowerShell. So question. Let's find a way to clear the Windows event logs using PowerShell. A simple Google search for PowerShell clear event logs results in a link to a Microsoft article about a clear event log command let's take a look. PowerShell command let clears all entries from specified event logs on the local or remote computers. So for the notes here as the threat hunter, we would add this information we've discovered about PowerShell command that clear event log, including any source URLs we would add this to our notes and continue. Interesting. So, could an attacker use PowerShell, this PowerShell command that clear event log as a method to clear the logs? Sure. Let's dive into Splunk and hunt for any evidence of this command. Okay, let's search this PowerShell command in Splunk. We're going to search the Sysmon index for event code one. We're going to search for process dot command line containing clear event log. 21 events. Now we have proof that the attacker use the PowerShell command that clear event log as a method to wipe the Windows event logs. This is the how to our hypothesis. So for the notes, as the threat hunter, we would add this specific Splunk query, including date time specified for the query, as well as any details about the results to our notes and continue. Let's make this Splunk query more presentable. We're going to search the Sysmon index for event code one. Again, specify the process dot command line containing clear event log, and we're going to use rename to rename the long process dot command line field as just command. Then we're going to organize the results in a nice table using table, then we'll specify the fields we want. We're going to sort by time with oldest on top to newest on the bottom. Looks great. So for our notes, again, as a threat hunter, we would add this specific Splunk query, including the date time specified for the query, as well as any details about the results to our notes and continue. So how can we document the final results of the hunt? Perhaps you'll see now how important it is to take copious notes throughout the threat hunting process. We can take our notes, we can document the final results of the hunt, and we can use our notes to fill in the details of a threat hunting template. And this process can be repeated using new hypothesis in the future for this threat hunting exercise that we just did. It would make sense to break these results into three separate threat hunting templates. Windows will name it like this Windows event logs cleared via event viewer GUI, which will address the Windows security event ID 112. Windows event logs cleared via Windows command, which will address the Windows command WEVT util CL. And also Windows event logs cleared via PowerShell clear event log, which will address the PowerShell command. To document these threat hunting exercises, we can use this simple threat hunting template. It means a title, date created, our hypothesis, MITRE attack, MITRE sub-technique, simulation details, if any, proposed search query are the limitations and observation notes that we can kind of summarize and expand on. And finally, our hunt findings with any recommendations. And then we'll have a proposed detection title, if any, and a proposed detection query. And what it's going to look like would be something like this. So here, using our notes to guide us, we can create the title, Windows event logs cleared via event GUI. To date, our hypothesis, MITRE tactic, MITRE sub-technique, simulation details, if any, in this case, there's none. Proposed search query and hunter limitations observation notes. Here's where we can summarize from our notes. So for this, I put the proposed search query did find evidence of Windows event logs getting cleared, but the results were broad. The search query was improved to display the relevant data in a readable format. And also put here, it's important to note that the search results did not specifically show if the Windows event logs were cleared via event viewer GUI. So it'd be prudent to test this in a simulation to verify that Windows security event ID 1102 would still be triggered, no matter what method an attacker use, GUI, command, PowerShell. So for the hunt findings for this, I wrote, Windows event 1102 is wonderful for detecting when Windows event logs are cleared. And the query developed below should be made into detection. So here's the proposed detection title. Alert Windows event 1102, the audit log was cleared. And finally here we have the specific proposed detection query. The second one. And using our notes. We create a title Windows event logs cleared via Windows command. WEVT util CL. The date hypothesis. Miter tactic, miter sub technique and note for these sub techniques for these three threat hunting templates. I'm adding some extra info at the end in parentheses. So for this one I put via Windows command. WEVT util CL. There was no simulation details. In my proposed search query. And for hunter limitation observation notes have the proposed search query did not produce any valuable results. However, it would be prudent to run a simulation test using this command and build the detection. And the hunt findings. So the query did not produce results. I believe that I believe the developed query should be made into a detection because attackers could possibly use the command to clear the windows log events. Windows event logs. The proposed detection title for this one is alert Windows command WEVT util CL used to clear windows event logs. And then finally here we have the proposed detection query with my specific splunk query. Okay, third one here. Using our notes to guide us create a title. Windows event logs cleared via PowerShell clear event log. We have the date or hypothesis. Miter tactic, miter sub technique. Again, notice here at the end I put in parentheses I added via PowerShell clear event log. No simulation details. The proposed search query. And for the hunter limitations observation notes. I wrote the proposed search query produced evidence that the PowerShell command let clear event log was used, but the results were broad. The search query was improved to display the relevant data in a readable format. See proposed detection query. Okay, so for hunt findings. Use the PowerShell use of the PowerShell command lit clear event log should be monitored for any suspicious behavior and so the proposed detection query should be made into a detection. Proposed detection title alert PowerShell clear event log executed. And then, of course, my specific splunk search. So, one final thing I'd like to mention here. There are many different ways one could conduct their threat hunting. There's no one way to do it. But this has just been a simple methodical way to approach it. In the end we have our threat hunting results documented in a simple template and shared with our team. And that's it. So, thank you. Thank you. Thank you. And feel free to join in on the conversation with the blue team, the blue team village on discord. There's the discord server right there. Thank you very much.