 Welcome. Thanks for joining this session of Blue Team Village Project Obsidian, Incident Response and UDA loop. My name is Julianne and I will be your guide for this walkthrough. It will focus on the UDA loop and how to use it during security incident response and illustrated by two alert case of the same incident. This is part of multiple walksthrough around incident response but also find sync, threat intel and various other aspects of cyber defense. We encourage you to attend another session between real life or virtually. One of those is around triage and scoping for which we will do a very quick reminder. Our agenda will be a quick introduction, the UDA loop, some key questions for incident response or two alert case, a lesson learned and conclusion. For incident response, we will start with the triage and scoping. This is developed in a different session but as a reminder, the six phases of a security incident are preparation, identification, contentment, eradication, recovery and lesson learned. And as a fundamental part, always keep in mind that you should document, document and document what you are doing. Be it for your colleague, be it for an auditor, be it for your management. This is necessary for any security incident response. Please note also this presentation and the corresponding notebook will be available after death count. Let's move to the UDA loop. The UDA loop has been developed by US colonel John Boyd from the US Air Force in the 60s following his experience during the Korean War. It was initially for dogfight, but the concept is widely applicable, most notably for us to security incident. What does UDA mean? Observe, orient, decide and act. It's a quick sequence of steps that during a dogfight helps you to assess the situation and evaluate where you are, what action you want to take and take the action. And it's usually operated multiple times inside a dogfight to help you win against your attacker, ideally because you are operating this loop faster than the attacker. It's kind of similar to the Deming Wheel in ISO standard, the famous plan to check act, but more at the micro level versus the macro level of ISO standard. Obviously, all the observation, orientation decision will be based on the context of outside information of cultural information and past experience, whatever has been analyzed during this event. I have a quick question. During an incident, what is very important is identified what you know, and what you don't know. To quote Donald Runsells, they are known knowns, they are known unknowns, and they are unknown knowns and unknown unknowns. What is often, is it a false positive, is it a known legitimate activity, known testing, be it a pen testing or something else. Do we have enough visibility to assess the situation and at a sufficient confidence level. It's very essential to say, we are not making confirmation bias bias and mis-evaluating things because we have just, for example, one source of information, which would maybe not be really able. To be sure that evidence can be preserved, to be sure that your colleague can do the same assessment, or if things go to court, that court can evaluate the same situation with the same information. We want to know obviously which user are impacted, which role and permissions they had. Same thing for system. We want to know what stage of the attack it is, and is it live active versus the attacker has left and you are only doing basically the cleaning or preventing the attacker to come back. How did it start? Who or what is the patient zero? Did lateral movement occurred? Do we have just a single system impacted or are there many? Which data attacker has access with some legal impact? And last but not least, what is the business impact? Where is the service unavailability, some information disclosure or data tampering? Let's move to an early alert. As part of project obsidian, team has simulated different scenario and kill chain. Now we will see what happens is during an early level of this attack, we got a notification. In this case, user brand social got an email notification that he logged in the company jump host, but he didn't. In a very good company member in notified security. You are security. What are you doing? So identification phase. I said for the older loop, we will do some question, each question will result in one, or maybe multiple observation. We'll orient investigation and a result in decision and action. One quick reminder as a start for any incident or alert. Have a ticket where you will document things. I can only see alert to the user when it's the manual reports and share the ticket reference. Depending on the context, you might or might not share more information. Document what's known, what's unknown, what's your understanding of the initial reporting of the alert. It's important to know that not all user are reporting things the same way. And one user can say, can share a lot of details when another will give you my computer is crashing and you will have to ask the question to say, yeah, okay, is it a serious incident and why what into you to make a security report. Following this documentation and as part of avoiding any bias, you want to regularly have peer review. First iteration of the other loop. From an observation perspective, we look at the ad authentication log, which show a password spray attack has happened and multiple accounts were compromised. We can orient ourselves by reaching out to it or help us to validate. There is no related activity pen testing testing or similar that could be associated to this behavior. After we want to decide do we want to move to the next phase or to more identification. Let's review how we can do this authentication. As part of our Jupiter notebook, we are using mystic by library to do a splint query. As an example, we collect successful and failed logon pair user account and source IP. If we try to visualize data, we have a nice chart on the ad authentication by user. And we can see this line, which is nearly always associated with the password spray. So normally this graph is interactive inside the presentation mode it does not render as well as we want, but we can change things. Let's see a snapshot of the rendering where we have more user but we can see that all the user of the company basically as an attempt to log in on the chart we don't see if it's failed or successful. But if we review another visualization pair IP user versus IP, we can see that all those login are coming from a single IP address, which for some reason I can't change in the presentation sorry for that. With all the information, it's easier to say from the log, I want to get all successful login from this identified attacker source IP, which from your security documentation and all your IT documentation, you can confirm if the source IP is related to your company or not. In this case, we can find that from a single IP, we have five different users that have login and public IP. So it's likely that the attacker is using this as an entry point. Just with authentication log. We know the attack vector is a password spray. We have the source IP of the attacker. Maybe there is a single one maybe there is more and we have identified that it's not just one account compromise but five. It's important here to have not jumped the gun compared to the user report where if you had actions that directly you would have only one user contained or disabled and which would not have blocked the attacker because there were five access. On a few things we don't know. How does account privilege. Are there any other entry point and what activity happened after that for further iteration of the UDA loop. We can check if there are where password reset if some MFA change happened if some cloud app token change. We may want to collect more information be to a VM snapshot or more granular artifact collection like process activity, network activity, auto runs. Depending on your environment. We may want to add telemetry if it's missing our stage and log collection network collection like ideas IPS. And depending on your assessment of the incident notify management in depending on your companies that may happen earlier that may happen later. Depends on your process. We won't dig further in this early alert case because we will review in the latest case. Now moving to the contentment contentment should be proportional to the type of incident and the business impact, you are unlikely to stop a service that brings million of revenue if you have a small fishing from a Nigerian president. And if you are if the attacker is extricating your customer data or your company intellectual property. It's very likely that action should be taken immediate. Those questions should be asking you to your leadership early on and ideally in preparation phase, not during the incident. First typical action for contentment lock disabled the affected user account. If you evaluate is it enough. If it's not enough, we can decide to content or isolate compromise system or suspicious one. If email propagation is used we may want to freeze the mail server queue or recall or delete some email. That's only a few of the actions that may happen or maybe needed during an incident. So for contentment, we are doing a eradication. Normally we always want to rebuild compromise asset because from a security analyst perspective that's the only way to ensure that an asset is clean. Sometimes business will decide otherwise because the asset is a legacy it can't be revealed or it will take too much time. So that's an executive choice decision to balance between the risk. After any rebuild, you should ensure that the system is functional and protected. Depending on the assessment, we ensure that patch or configuration has been fixed to ensure it can't be attacked. And we obviously want to be sure that the business service is operational for recovery. We declare return to normal service validate again that service are functional, normal monitoring is functional, no abnormal behavior observe for some grace period and ensure attacker is not coming back. Let's move to the last alert. We are more at the end stage of the attack. The previous alert has not happened or maybe was miss or misclassified. This is an automated security alert raised on the company jump post title mimicats execution detection. What are you doing. Let's do a quick reminder of what is mimicats. It's a very popular security tools that allow to extract plain text password hash and value secrets for memory. It can perform various type of attack, including past the hash past the ticket. And it's very often used by attacker as an open source tool. We start again from identification. We talk about it staff or business staff. It's very important that you have a good documentation, the inventory identity matrix network matrix that will help you a lot. But clearly, it never will never replace helpful staff. You still want to have documentation that will most often make your work easier and faster versus trying to wake up the engineering on call. And just wait somebody on a bridge. First iteration of the older loop. Again, we try to validate. Is it a pentest or legitimate activity by it or by some non user, depending on your company. You may have user more normal or more abnormal than others, and doing some activity which may resemble an attacker. So it's part of the knowledge and the baselines that you build over time in preparation of incident. Next, we confirm that the mimicats execution is not legitimate. We want to know which credential was stolen and are they reusable as is typically is it end user accounts service accounts some cloud accounts. Are they all credential versus domain credential. Are they domain admin or are they standard user. Are they some group policy object restriction in place, typically to prevent the use of a domain admin outside of certain system. We're using laps, which is a Microsoft tool to ensure that only each local administrator password will be unique. And this way they can't be reused to attack another system. We want to know what that action occurred and whether data filtration active or not. Back to our notebook and to a splint query. We try to check the process activity. Here to example query with either event ID 4688 net native process command line auditing in Windows or with this month event ID one, which is an extra tool but often recommended by security people. We can see the visualization perspective when we build the process tree of the jumpers. We can see various activity attached to one of the user, which is named Pat resist. And you can see that there is a cascade of activity that's mostly related to or shell but we will see other activity. More on this screenshot where you can see some task list call some run DL 32 with the mini dump argument. Who am I common? What we know from this process tree we can say reconnaissance activity has happened. We have some who am I task list, possibly other common. We have a process memory dump, which is unlikely to be any time legitimate. The one my or task list, maybe your system in can be responsible for it. But doing a process memory dump, very unlikely. And we know that lateral movement occurred. The cascading of poor shell and CSC with a computer list argument is a result of postploid invoke port scan. And CSC is because poor shell is calling a command let, which is using teacher. And as such, each call is resulting in a CSC children process that will do this live compilation of the blog. One thing that we don't know or we don't see in this process tree was mimi cats or some PS exec that you can be in in the financing was true. There can be various reason, but one thing obviously the data can have been tampered by the attacker or inadvertently, it can be wrong filter by the analyst. It can be a software bug or configuration is issue. That's one reason you want to have multiple data source and validate they all goes into the same direction. And if we review another log we have identified that the activity is occurring on the RDP jump post. The was deployed in this environment. We can try to filter some of the activity, either by some top DNS list top 1 million top 10,000. Let's say, what are the domain left from some filtering action, we can identify those domain interact s h dot com file pizza, financial magnum tempus financial dot com web warm all dot IO transfer dot s h. Likely the attacker has done an extra session or maybe he's on doing it currently, depending on on your timeline. The only case, and that's where it's important to know your company context is maybe there is a leg is limited legitimate business use case, but usually it will be for one service thing for different service is less likely to be legitimate. We also have seen some type of request on company domain magnum tempus financial, which suggests that the attacker was doing a manual execution. Sometimes it can be also a way to mislead the analyst by the attacker, but at least as a first appreciation that seems the attacker was doing things manually, not an automated, or at least not a fully automated that you can review the you can review the financing station works for more on containment. Depending on the pace of your investigation, you may stop the attacker before during the tax situation before data destruction and log whiting. Again, we want to lock or disable affected user account. We may also want to restrict network access bit by domain IP address to block common control access or to prevent exfiltration of data that will depend a lot of what are the option available in your company. For example, you have a proxy or only an ideas. If you are arriving very late during the incident and you already observe that we are on a stage where the attacker is doing event log clearing on many server and including domain controller, which means that the basically on most of your network. There is no time if not already to escalate to crisis. This will usually include someone from each executive branch legal communication HR potentially more. If you have not already done so but hopefully it is done in form the management. We may want to unplug from the network or shut down impact system. The network is usually preferred as it will help to preserve evidence, most notably in memory, but it will depend a lot again on your context and what can you do, especially in a really remote setup. In the worst case, more global network shut down, typically at office or data center level, maybe something needed. So how to switch to out of band communication. Usually if an attacker has control on your domain controllers that means they also have control on your exchange or mail system, your chat system, likely. You want to involve external partner, your insurance, your external returner, which maybe required or recommended by your insurance. Information sharing an analyst center, K Isaac, and nationals or sector set. Those last two are very important because they help structure your sharing of information and eventually share things in a way which are not liable for you and share this information in a way which is not necessarily identifiable. It's important that you can help that if other company have made similar threats, you can get more input to react faster. Typically to do the containment or to know that this threat actor is respecting their engagement when you pay them or they are not respecting those. You want to validate your backup. Normally that's an early recovery step. But, especially in run somewhere case in can influence contentment, meaning results should be known only early if you are attacker has destroyed your backup or if your backup are corrupt or I've never been tested. You probably have no backup and as a result, you probably want to make more care on protecting system preserve evidence, keep a copy of encrypted data. If maybe a decryptor is found, maybe in a month, maybe in a year, whatever. Ultimately, a very hard decision in a run somewhere case is, do you want or not pair run somewhere. That's a very trolley problem or Cornelian dilemma that you should think carefully and discuss with relevant parties. There is no right answer from a security or law enforcement perspective the recommendation will be always not pay. Basically, if your company life is at stake. That's how they're called, especially if you are at risk to finance terrorism or cyber criminal. You also want to monitor Internet, the different values media and form to ensure that your incident is not leaked to early and that you have time to prepare communication. We want like previously rebuilt assets, but in a run somewhere case, you have probably very high number of system impacted. And first, you need if you are not already prepared it. You need to establish your priority for service recovery. What should be repaired first. Is it your business system, your payroll system, your active directory, whatever else. Ideally, that's come from your disaster recovery plan or business continuity plan, and it's done before any incident. But if it's not the case, that's your first date after rebuilding assets. And like in the previous case, we want to validate system as functional and protected. We don't want the attacker to come back. Recovery stems and previously returned to normal service, validate. Finally, the lesson learned. You should have a final communication to close an incident. It should either include or be followed by the incident report. You should include executive summary, key findings with recommendation, what went good, what can be improved, a high level timeline. Depending on your organization, alignment with MITRE ATT&CK, MITRE SHIELD, NIST, SP800, 53 or ISO 27, maybe good addition. And part of all recommendations should feed your org risk register and be visible to decision maker and executive. You don't want to let an incident and use each incident is an opportunity for improvement. If you are not using it, you leave the door open to an attacker. It could be the same one, or it could be another one. More will be developed in the final reporting work. As a conclusion, security incident response is very often a marathon. You need to take care of your team and yourself, plan for overload, plan for helper, be it internal or external. Preparation is key. Establishing process, trust and collaboration take time, train early, train like you fight and fight like you train. Re-evaluate situation often during an incident. If you have better situation awareness than the attacker, the better. Know yourself, know your enemy, and you can fight a hundred battles without disaster. Uda loop is one method to structure this. Last, make it easier for you and your team, structure and automate the work so decision can be taken better and faster. Jupyter Notebook is one way to structure, to automate things, and it helps to document and show the reasoning of investigation and avoid bias. This is the end of this walkthrough. Thanks a lot for attending it. I also want to thank all the contributors to Blue Team Village Project Obsidian and to project Jupyter Notebook, Panda, Mystic Pie, OpenStrength Research. Enjoy Blue Team Village and Defcon, be it virtually or in real life. Thanks to you and join the conversation on all this call.