 This is a demo of my new tool, ByteStat. ByteStat is a tool that will give you some statistics about the bytes found in a file. So here you have the help. And like my new tools now, it also comes with a man page. So you do dash m and you get to the man page. So I have a couple of files here. Let's start with all.bin. So this is a file of 256 bytes with all possible byte values from 00 to FF ordered in increasing value. When I run ByteStat on that file, you get some information. First of all, you get the count of each value and you get by default the five values that are less, least frequent and five values which are the most frequent. Here, of course, since all bytes appear just once, all byte value appear just once, the frequency is the same. And then you get the size of the file and then you get here some calculated data for the file like the entropy which is eight. That's the highest entropy you can have. The amount of null bytes, so there is one null byte, the number of control bytes, 27 control bytes in the file, number of white space bytes, six, number of printable bytes, 94, and the number of high bytes. So that's bytes with the highest bit set to one. So that's 128. Another file here is the file not random and that is actually just a repetition of the all.bin file. Let's take a look like this. So you can see from zero to FFF and then it repeats like this, then you get similar data for this file. Another file is random. This is just a file that contains random values for bytes in a random order. If we try that one, here you can see that you have some bytes here like byte CE, that's the one that is the least prevalent. So this one appears only 242 times in the file. And here are the five less frequent bytes and here are the five most frequent bytes like 75 here appears 344 times in the file. The size of the file is 74,752 bytes. And if the size of a file is larger than the bucket size and the bucket size is by default 10K, so 10,240, then the file will be split in buckets of that size. And in this file we can have seven buckets. So this is the file size. This is the default bucket size. And this is the number of buckets, complete buckets of size 10,240 that are possible to find in the file. And then you have the calculations here again, entropy and other calculations for the complete file, but then also for the buckets. And here we have an entropy of 7.997180, that's the entropy for the complete file. But then if we calculate the entropy for each bucket, there is an entropy that is smaller and the smallest one is 7.981543, like while the largest one is 7.984125. And this bucket can be found at this position. So Xadismal F0000 into the file, while the highest entropy can be found at this position in X50000 into the file. And then again we have null bytes, control bytes, wide-spade bytes, printable bytes and high bytes, but each time the minimum value that is found in all of the buckets and here the maximum value that is found in all of the buckets. If those numbers are not that much different then you are dealing with a file that is homogeneous in its content. So there are no parts in the file where the randomness is higher than in other parts of the file. Well, if you do that on a file like this one here, picture.jpg ransom. This is actually a picture file that was encrypted by Handsomeware. If we look with byte stats at that file, we can see that it has overall high entropy 7.18559. So this could indeed be encrypted or compressed, but then there is also a bucket where the entropy is rather low, 5.15, so that's certainly not random. And this can be found at this location into the file. So this here, this overview, already tells us that although this file here has seemingly been encrypted by the ransomware and that it has a high entropy, that there are still places into the file where the entropy is not that high so that we have probably some structure into the data. And we can list the entropy for each bucket. You can do that with the list option, like this. And then for each bucket, you can see here the entropy. And here, for example, we already see that bucket starting at 7800, the entropy is getting lower. Now apart from this entropy calculation, so you also have a histogram. And so that's what you see in the first part here. Let's do this again on random bytes, like this. And you have these values here. Now these are just the five least frequency value and five most frequent values. If you want to see them all, you use option all, like that, and then you get the complete list of all the values, which is sorted by prevalence from the least least frequent to the most frequent. You can change that order by doing a descent option, the option, and then you have the inverse order. And you can also decide to not sort it by prevalence, but just by the key. Then you have here the values itself that are ordered. Now you can have files that have a very high entropy, but which are still containing bytes that are not randomly ordered. But you will not discover that by looking at the entropy. And for that, I also included the search for simple sequences. And a simple sequence is just a list of bytes where the difference between two consecutive bytes is constant. So for example, that the difference is always a zero or one or two and so on. And you do that with option S. And if we run bytes that with option S on all, we get here in the end the sequence that is found. And you can see that at position zero, we find a sequence that is 256 bytes long and that the difference between two consecutive bytes is just one. And here you have a dump of those first bytes. So you see zero, zero, zero one, zero two, zero three and so on. An increment of one. So this tool here, although it finds a file with the highest possible entropy, is still able to locate some simple sequences and thereby helping you to find structure into files. If we do that on the random, we get a sequence of four bytes. That's the longest sequence here that we can find because by default here you have the sequences that are found sorted in descending order and the length of the sequence and that in descending length. And the longest one, we found this four bytes and this is the value of that sequence. And if we run this on our picture like this, then we can see that we find some very long sequences of bytes here, eight zero, zero zero, eight zero, zero zero, eight zero and so on. So certainly not random data. And we can also instruct here bytes that to order those sequences by the key. So the position and then here you can find the first 10 sequences found in the file. If you do all, you also get to see all the sequences like this. Now a sequence of three bytes that is very common so we can say that the minimum sequence that we want to find is four bytes. That doesn't seem to be the correct option. Let me check. And the option is not dash N, but dash F filter. So the sequences of at least four bytes and then we get this output. Let's pipe this through less. First we get the histogram and here we can find some bytes. So you can see already at the position nine B D F, we find some bytes here which are not random. So that's where you would start to look into the file to see what you can find. Now always those eight zero, zero, zero sequences. Here's another one. And here we get all the sequences.