 Hello, the DS7s here, senior handler at the internet storm center. I wrote a diary entry about analyzing log FMT files with Cybershift and I also made a video for it. Now I'm going to do about the same but with one of my tools and because I usually don't use Cybershift to analyze log files like these, I use my own tools and I only use Cybershift when I cannot use my own tools for that case. The tool that I'm using here is Interactive Sieve. It's a .NET program that I've been developing for many, many years and it allows you to visualize comma separated files, information files with all kinds of options. Now the separator can be chosen here, comma. But here, since I'm using that log FMT format, I'm going to say that there is no separator. So there will be only one column with a complete line and also that there is no header. Like this. Now here I have the complete log file and it's 168 megabytes GZip compressed, uncompressed it's about one gigabyte and if you would load that everything into Interactive Sieve it would be way too slow. So a thing you can do is you can not only load a log file and it may be GZip compressed and my tool will decompress it but you can also load it with a filter and so it will only load lines that match that filter. You have a normal filter and you have a Riggs filter. So I'm going to do load with a filter of the log file here and now I just type the IP address that I'm interested in. So without source IP or anything just that IP address that will take some time. Okay, so that was almost a minute. Let's expand the column here. Okay and now here you see all of the lines that have been selected because they contain that IP address, source IP address 106. So that is the filtering that has been done. Now let's see what other source IP addresses we also have here because this could be for example also a destination IP address and here I'm only interested in source IP addresses of that value. So notice there are 5182 lines that have been filtered out out of a GZip file with millions of lines. So that's a very small amount to handle my tool interactive sieve. So now I'm going to run a regular expression here on these lines here, Riggs matches and I'm going to say source IP equals and then anything any character that is not a space character. Okay, so and I do see I have this as a source IP address what I wanted and also just with a 1. Okay, so not much 74. Now I could say I could select this here and say hide to hide those lines I'm not interested in but unfortunately this would also hide these lines and because it's exactly the same except for the 06 so I'm not going to do that. I'm going to do it a bit differently. I will select for example this line a right click and then use sift when I'm going to already let it populate with complete line here so that I can say I don't have to type it I just can use what was preselected. I'm going to sift for source IP equals this value and any line here that's the operation I'm going to do any line that doesn't contain that string I will hide it. So we have 5182 hide and now we have 5108 lines that are visible out of a total of 5182 if I run the rex matches again okay this is the only source IP address I have. Now let's have a look at the destination IP address against with rex match so destination IP and these are the destination IP addresses private IP addresses and public IP addresses I can hide for example the private IP addresses select hide and now I'm only left with the public destination IP addresses. Protocol UDP here so let's check what other protocols we have like this proto UDP seems to be the only one let's check the destination port destination port 53 service now if you had a CSV file of course you will not have to type e have to type each time that field you could just select the corresponding column service but here we have another format but you can still take a look at it with my tool interactive sieve okay service DNS okay and then we also have different entries origin underscore bytes and so on so let me shift for that too sorry to a rex match so we want bytes and it may be preceded with any character that is not a space character also I'm using asterix because it doesn't have to be the case just one zero or more like this and then we have these values all these values and a couple stand out here quite large 500 for example I can select them color them in red and then for example take a look at these lines and look at them in more detail