 Друзья и господа, вы можете поставить свои руки вместе для Максима Ледро? Здравствуйте, Вкон! Как вы делаете это? Нормально! Я просто был в партии рядом с моим домом всю ночь, так что я спал 45 минут. Так что... Я извиняюсь, если моё сердце меняет. В какой-то момент в этой презентации. Но надеюсь, что мы сможем сделать это. Итак, мы будем говорить про... 0 баксов. Делаю бирю ФЛ. Как mejorar короче-гадед фазин. И найти новые 0 дня в тяжелых таргетах. Так что рада видеть все вас здесь. Во-первых, дай мне спасибо, Дэфф Кон Гунз, за меня здесь. Я очень рада, что вы говорите на этом этапе сегодня. Перед тем, что я начну, дай мне интервью. Меня зовут Макс. Я спрашиваю поvarious-секретной конференции вокруг планеты, mostly about dynamic analysis of machine code. And last year I presented at DefCon, I talked about malware fizzing. So check it out, if you are interested in this topic. I have experience working on open source fizzers, such as VINFL, DRFL, contributing in open source framework, DynamoRio, and working on implementing extremely abstract, operating system for Windows malware analysis and IBM research in Israel. So currently I am working at Salesforce Red Team as an offensive researcher, searching for zero-day bugs, writing exploits and doing engagements. In my case, I need to identify vulnerability and write exploit in a very, very short period of time. And that's actually why our story about efficient coverage-guided fizzing begins. So we are going to talk about fizzing today. And what kind of problems we face when we use AFL. I will describe what coverage-guided fizzing is and explain how AFL actually work. Then we're going to talk about downsides of AFL and how each of them can ruin your entire fizzing campaign. And in the second part of my talk, I'll present a new coverage-guided fizzer implemented in PurePython that's supposed to address all of these problems. And of course we will compare it with AFL. I will show you a couple of vulnerabilities I managed to find with this fizzer. And so what is fizzing? And more importantly, what is coverage-guided fizzing? So the general idea of fizzing is actually a dream of any professional procrastinator. So you run a special tool, this tool mutates your input, provides this input into your target and you just see it, wait for a crash. That's basically all. Let's see it on example. Let's say we have a very trivial program that crush an input, pound it. We start string five a's. We cover first couple of lines, program exits, we do it again and again until we find something that can cause a crash in the target. In this example, our fizzer need to mutate the string from five a's to pound it. And this is very unlikely and can take hours or even days to finally find this null pointer dereference. So to be able to address this problem, people invented a coverage-guided fizzing. People also call it feedback-driven or smart fizzing. In this case, we continue mutating our input, but at the same time, we estimate coverage triggered in our target. So if we see that our input lead to some new path in the target, we save this input and then perform a subsequent mutation on top of this input. We do it again and again, we apply the same algorithm for the next finding, we save it, we mutate the next byte, search for potential new path, open by this mutation, then save it again, mutate the next byte until we finally find an input that can trigger a crash in our target. So this way we can cover the program in depth and actually it dramatically increase efficiency of your fizzing campaign. So instead of spending hours or even days doing blind fizzing, you would be able to find this null pointer dereference just in a couple of seconds. So you see the difference. And this algorithm is the basis of the most known today coverage-guided fizzer called American Fizzelop or AFL. So AFL was created by Mikhail Zalewski five years ago and today it's a standard de facto coverage-guided fizzer for box hunting. It's attracting people with its efficiency and successful stories of critical vulnerabilities found with it. And it's a very powerful solution for vulnerability research, especially if you have a source code of your target. So I have prepared a short visualization of how AFL actually works. So here on the screen we have four windows. In the left upper window we can see AFL status screen and just regular AFL status screen. In the right screen we see number of files already existing in our queue. So as soon as we continue mutating we increase in our files, number of our files in the queue. So in the left middle window we can see content of file, of actual file that we are going to provide in our application. And it's in hex. And I am indicating with red the actual change that AFL performing on our input. So you can see it performing a lot of random mutations in our file. And as soon as we continue mutating this input the number of path is growing. You can see it on the plot, on the top, on the bottom left plot with wide background. So you can see it's growing. And in this example we do fizzing against 7-th deep. It's just, let's stop this video and move forward. So for the past five years AFL proved itself to be a very efficient fizzer to find memory corruption bugs in many different projects on many different platforms both in user land and in the kernel of all widely spread platforms. There is even kernel version of AFL called KFL. I managed to find a couple of bugs with this tool but this is different story. People even search for bugs in very specific places like SCADA systems, switches, wireless hardware, medical devices where any problem with software quality security can cause serious consequences of human lives. So while many of you might think like, okay Max, memory corruption bugs are cool but we are running in the cloud, right? While this is true, C and C++ remains to be on the top of the most used programming languages. So our world is running on top of C or C++ fortunately or unfortunately. We can also look into statistics on number of research publications about fizzing year by year and we can definitely see the trend. So it's growing and last year it exceed 1500 publications. It's a very hot research topic today. A lot of people play with fizzing and a lot of companies invest in this technology. Google even has an OSS Fuzz Project where they integrated fizzing within software development life cycle of 170 open source projects. They use a lot of CPU power and they already identified 1000 memory corruption issues by generating half trillion test cases per week. So this high popularity of fizzing across security community means that it's becoming harder and harder to find new bugs in products and libraries already covered by security researchers or projects like OSS Fuzz, right? So we have to search for some new strategy and be able to perform fizzing more efficiently than other tools and to be able to perform fizzing more efficiently we have to understand what kind of issues already exist in modern fizzers and be able to address them. So let's start with the first one and it's a volatile path. What is volatile path? Let's say we have input eight A's that cover these three branches in our target indicated by green. We mutate one bit and now it covers three previous branches and two more shown in yellow. So if we, just a second, okay. So if we send this input again in the target it again cover only the first three paths. So we send absolutely the same input in our target but now it doesn't cover a path indicated by yellow. And those paths consider it to be volatile. They do not depend on input but parasitically executed in our program due to some randomness. And it happens pretty often, especially if we work with multi-threaded application. And our Pfizer actually has a dilemma. What to do with this input? From one point of view it can just run it multiple times, run, right? And then reject it if it sees some path appears and disappears after each run. But what if we actually discover something new with this test case? In this situation we would reject a new finding. Okay, we can keep this case and raise the warning for our user. That's what we actually have in AFL. But in this case, if our target is very volatile we can end up with a very large corpus that actually do not cover anything new at all. So we would just waste in our Pfizer time. Second problem is parallelization. Let's say you have a really slow target and you don't want to actually spend time on increasing performance of your target, right? So you are not a performance engineer, you are a security engineer. So AFL wasn't designed to be parallel from the beginning. Of course there is functionality that allows you to create master and slave instances. But let's discuss how it actually works. Let's say we have one master instance and two slaves along with three files in our corpus. AFL can perform deterministic mutation on master instance and random tweaks on slaves. So the corpus will be copied into each instance and all of the instances will perform mutations in each file one by one. If one slave finds something new this file will be distributed to other instances and they will continue mutating this, the same finding. So that's all actually what we have in terms of parallelization in AFL. It will work fine for fast targets and when we have small corpus but it will perform much poorly on slow targets, especially if we have really large corpus. So there is no easy way to distribute these corpus between instances and ask AFL to share code coverage in a smart way between them. And I believe we can parallelize better here and I'll show how a little bit later. Another problem is the lack of network file mode in AFL at all. So we actually don't have much fuzzer that can perform network fuzzing, coverage guided network fuzzing at all, especially on Mac. On Linux we can try some AFL forks or HONK Fuzz. On Windows there is network fuzzing mode implemented by me last year and we can try HONK Fuzz on Mac. That's actually all. Usually when you want to fuzz network application the general advice is to somehow make your application easy to receive input over file instead of actually doing fuzzing by sending network traffic, right? But it significantly reduce number of potential applications where we can use this approach, especially if we deal with black box binaries when we don't have source code and we can't just modify our target. Speaking about other platforms such as Windows we are significantly limited in number of potential tools we can use. Basically, if we don't have source code there is VIN AFL and if we have source code and our target can be compiled with Clang, we can apply libfuzzer which is basically designed to perform fuzzing of library API calls. And that's all. All of these tools have their own limitation, of course. On ISX we have even less tools that we can use to fuzz user land application. Basically, we can try to compile our target with Clang and use libfuzzer if we have source code. If not, there is actually nothing that can perform coverage-guided fuzzing for you. Well, at least I don't know about this. And of course I'm not the first one who wants to address these problems. There are a lot of researchers who try to address these issues and improve AFL in some way, such as AFL Fuzz, AFL Smart, VIN AFL, libfuzzer, honkfuzzer and many, many other tools. There are a lot of efforts to leverage coverage-guided fuzzing on kernel, such as CIS colors that demonstrate incredible results along with KFL, 3F and others. I just listed some of projects here and papers published in the last year. So we can dedicate entire day describing different fuzzing solutions and techniques. And there is a nice paper where you can find systematic research on all existing fuzzers if you want to go deeper in this topic. And I apologize if I forget to include some research or tool here. It's not because this tool is not great, it's just because we have limited amount of space here. And of course time. However, if we summarize all of our requirements and problems that I described earlier, we can see there is a huge demand for a new fuzzer here. This fuzzer should be very flexible in terms of scalability, parallelization and platform dependencies. And at the same time, it should be able to address volatile path problem and be able to support data mutation with multiple fuzzing strategies in one fuzzing campaign. Of course, this fuzzer should also support functionalities that already exist in modern fuzzers, such as allowing users to provide custom mutation algorithm to be able to implement specific fuzzing strategies or even enable a structure-aware fuzzing, which is a very, very promising technique today. So, ladies and gentlemen, let me introduce Manu, a new coverage-guided fuzzer implemented in Pure Python that tries to address all of these issues I presented on the previous part of my talk. Manu was born to be parallel from the beginning. It can obtain coverage from FL Instrumented Binary or from DBI frameworks like Intel Pin or DynamoRio. I'm trying to support Linux, Windows, and there is a better version support for Mac. I decided to call it Manu after the most fluffiest cat on the planet. Basically, in English, it's called Palace Cat, but in Latin, it's called Atacalobus Manu. So Manu is a very adaptable cat capable to live and hunt in very severe weather conditions of Central Asia. And I just like this cat. This is actually the second DevCon talk where I present exactly the same slide. And this template was actually made out of this picture. So I processed it through a neural network and then just applied black color. Okay, moving forward. Let's talk about architecture. Of course, Manu will employ model-based design. You have network model, instrumentation, models core, UI, mutators implemented as plugins. This way, user can ask Manu to enable or disable different mutators or even provide his or her own and ask Manu to use them. If we want to understand how Manu will address volatile path problem, it's pretty straightforward. Let's say we have two volatile paths indicated by yellow. When Manu sees that our input covers something new in the target, we will run this input a couple of times. It's called calibration in coverage-guided file. And then just if we see some randomness in our coverage, some random path, it will temporarily blacklist those paths and ignore them in the next runs. So first time I found this algorithm in KFL, Colonel Pfizer, and I just decided to integrate it with Manu. Probably it was introduced in some other Pfizer, but I first found it in KFL. Okay, moving forward. Parallel fuzzing is default feature in Manu. You just need to specify how many fuzzing instances you want to spin up using the command line argument. So Manu will automatically start all of them and split the corpus equally to each instance. So in this case, each instance will have only a small part of the corpus. So this way we can smartly distribute our corpus across instances. And we can even spin up a remote instance on some other machine available over network. So Manu will be able to share corpus and coverage with this remote instance, like it's running on the same machine. This way you can have your own Pfizer's cluster on top of Manu. And basically each instance is an independent Python process that utilizes its own shared memory. At the same time, Manu main process will be able to synchronize coverage and share it across other instances to avoid double work for them. So if one instance found some new path in the target, all other instances will at some point know about that and will focus on something else. So in this way we can smartly distribute our fuzzing campaign across all available cores. User can easily provide a third-party mutator via plugins. So it's only required to place your plugin in a special folder where Manu main binary located. And plugins can be implemented in Python and should contain init and mutate functions. So far I have implemented 2,000 strategies by default. So you already have it in Manu. The first one is AFL strategy ported from C to Python. And the second one is Radamsa, distributed as native library. Network fuzzing is an experimental feature. So you can try to fuzz TCP or UDP server. It works. And you can ask Manu to act as a TCP or UDP server to fuzz some network client. But this feature in very better stage, so expect problems. And Blackbox binary fuzzing is a very important part of Manu. So by default Manu supplied with 2 DBI clients for Intel Pin and DynamoRio. Manu will start binary using one of these frameworks. So you can choose what you want to use. DynamoRio or Intel Pin. Then DynamoRio or Pin launcher will start your target, inject instrumentation library. Instrumentation library will open shared memory. And will check coverage back to Manu. So pretty straightforward way. And I recommend to use DynamoRio, because Pin is only supported on Linux. And it's low. So this is how it looks when you run it on Windows. Manu will launch a lot of Python processes. Each of them will run DynamoRio. DynamoRio launcher will run the target. And this injected instrumentation library. And then instrumentation library will provide coverage back to Python process. So pretty straightforward. This is interface. We have information about volatile bytes, coverage, number of iterations, crush, exceptions, and many other useful things. And it also has its own logo implemented as Ascii to Art. So just to have logo. Each tool should have logo. So Manu supports seven most used options via command line and dozens of others that should be provided via configuration. So in this way you can adjust Manu for your own purposes. And let's see the next. So this is Manu main folder with a lot of files for different platforms. We have the main script here. NetworkModule, Utils, PrintingModule, AFLMutator, ported from C to Python. Radamsub binary for Linux configuration file. We can look into code. It's just straightforward Python code. Nice and fancy here. I tried to make it as clean as possible to be able people to contribute. So we have also AFL, ported from C to Python. We see a lot of different strategies from AFL. And we have a couple of folders. In Windows folder we have folders for DBI and instrumentation libraries for 64 and 32 bits. And same for Linux. So you can just launch them and run against black box binaries. And if you're looking to command line option, it's again like any other fuzzer. You have to provide a path with your files that you want to submit in your target output, number of parallel fuzzers, switch between dump or smart mode. You can provide path to configuration file with more options. And you can restore previous session. So if we look into configuration file, we can find much more options. So they should be provided in this file format. So just option names, basic, equal value. We can enable dictionary for AFL strategy. So AFL will inject tokens. And we also can assign weights for AFL. So in Radamsa, for example, we can say that Radamsa should mutate 20% of our test cases and AFL 80%. Or we can enable our own fuzzing strategy just by specifying its name. For example, we can say example mutator should mutate 20% of our test cases. We can run this deterministic seed. We can print more information. We can disable this volatile path suppression algorithm that I showed previously. And to be able to start black box binaries fuzzing, you have to specify which DBI framework you want to use. So it's Dynamo or Intel Pin. If it's Dynamo, you have to provide DBI route, where it's located, the launcher, then client, supplied with manual. And you can also specify for manual what kind of libraries you want to instrument along with your target binary. So this way, you can include them in your code coverage. This time out. Also, these options, NetConfig master, NetConfig slave, can be used to spread manual over network. You just need to specify port and IP address. You can run it in debug mode if you need more information. Synchronization between different fuzzing instances. And we have command line fuzzing as an experimental feature. Just in this case, manual will send it over command line. And to be able to enable network fuzzing, you just need to specify IP and port, for example, and protocol. So, for example, instead of TCP, or ADP, we want to fuzze it over TCP. And we can also ignore certain signals, for example, CIGABORD, which is quite annoying if it would be considered as false positive. OK, we can actually start it. So we just need to say, where is mine binary, input folder. Then provide output folder. In this case, we are doing fuzzing against 7-zip. Then number of parallel threads, like 20, for example. Then pass to our main binary, like in AFL. And then command line, like options. And that's all. Then just enter. It will spin up a lot of instances. Each instance will perform its own mutations. Then you can see the dry run is finished. So it's running. We already identified a couple of crashes. And a lot of other useful information, like coverage statistics, performance, executions per second, files in queue, timing, what kind of mode we are running, and strategy. OK, so I guess we can stop it and move forward. Now let's discuss vulnerabilities that I managed to find with this tool. So popular is an open source library for rendering PDF documents, mostly used on Linux. Three software products, like events, LibreOffice, InScape, used by million users across the world, integrated this library for PDF documents handling. And popular participate in Google OS fast program, which makes it much harder to identify new vulnerabilities in this target using AFL and libfazer, because Google already generated trillions of test cases. So we just don't have enough CPU power. OK, so I decided to try the same seed corpus with 491 PDF files used by OSS files, and run the latest version of AFL and Manu for 24 hours with 78 parallel jobs. As you can see, on average, AFL was approximately 25% faster comparing to Manu. But execution speed is not the key factor here. If you compare paths found by Manu and AFL, we can clearly see that after nine hours of running, Manu managed to outperform AFL and continue to discover new paths in the targets faster than AFL. And this was possible because of the improvements that I introduced in the second part of my talk. So our corpus parallelization plus volatile path suppression plus combination of two mutators gave us much better results, despite of lower execution speed. And, more importantly, Manu managed to discover three zero-day vulnerabilities and five non-security-related bugs in popular, previously, of course, unknown for developers. So NIST assigned the following CVs for these vulnerabilities from medium to critical severity rate. And let's actually discuss a couple of them. OK, so the first bug. The first bug is a heap overflow that exists in downsample row box filter, which is called by downscale image in libkirl library. And this function is actually used to downscale GPX images that might exist in PDF objects. I don't want to go deep into technical details here. It's not important. All we need to know that we can use, we can trigger heap overrun by controlling pixel coverage variable. I mark this red and force this function to read out of heap chunk to variables A, R, G, and B. And at line 40, we can write these values to the destination buffer. To the best of my knowledge, I don't see straightforward way to exploit it. But most likely, we can use this approach, this vulnerability, to get some pointer leak out of this vulnerability, which might be very useful if you want to bypass SLR. OK, the second bug is located in GPX stream I need. This function doesn't have a check for negative values. So I highlighted in red where popular read boof size, which we can control, and call to unsigned char function without actually checking size of boof size. So in this function, to unsigned char's initial size can be very large. So it makes possible to allocate a really large heap chunk of arbitrary size. We can use this bug as a second stage in house of force heap metadata exploitation, where we need to allocate a large chunk to be able to force malloc return a pointer to arbitrary place in our target memory. OK, the last bug is more dangerous. Here we control variable i, that can be negative, which might cause an integer sinus error, and heap out of bound read, controlled by an attacker. So if we can satisfy the first condition and avoid this really, really huge statement, basically what we can do, we can return an address to an arbitrary object in memory, which is already pretty scary. So the caller function will check if this object is not 0, and then call one of its methods, which is pretty straightforward way to get control over RIP. So I indicated in red, this is our function, get entry, and then we just call entry get flag. And instead of get flag, we can place pointer to any function that we want. The only problem here is to prepare this object in the right way and allocate it in the right place. But I believe it's very possible, so it's straightforward to exploit this vulnerability. OK, moving forward, Zeke, formerly known as Bro ideas, it's the world's most powerful open source framework used by thousands of companies and institutions across the planet. There are a lot of very powerful plugins for Zeke, such as J3, designed to fingerprint SSL communications, which is a very powerful technique to detect suspicious encrypted connections, usually performed by malware to communicate with C2, or rent team and plant to communicate with C2, depending on whom we are talking about. So this project even has its own conference called BroCon, which takes place in Arlington every October. So after Zeke is very aware of memory corruption issues and fuzzing technique in general. Because even a simple denial of service is completely unacceptable for such type of security products. So the code quality is pretty high, I would say, very high. And they have done multiple fuzzing campaigns in the past. So it was actually a serious challenge for Manu. Unfortunately, Zeke has a very complex initialization routine that usually takes up to 10 seconds to initialize and start traffic monitoring. And it was very, very early stage of Manu that it was just couple of Python scripts. So in this case, if your target takes 10 seconds to initialize, it's completely unacceptable for Pfizer to wait for 10 seconds. Fortunately, they have done a couple of fuzzing campaigns in the past with Leapfizer. So I managed to find an example of how to avoid this long initialization routine. So all we need is to implement a special wrapper for each protocol we want to fast. And an example of such Pfizer for SSH protocol shown on the slide, along with other 10 protocols. So I run both Pfizer for 24 hours with 70 parallel instances for each protocol and managed to find three previously unknown vulnerabilities. And IFL found zero bugs. This is the list of vulnerabilities I found. Two of them are located in Kerberos and the last one in RC protocol implementation. So while for the most application, memory leak is not a big deal, for ZIC, it might cause a serious issue if memory is not free in a pocket processing function. So memory leak, in this case, is a memory allocated on the heap, but never free. In our case, we have memory leak in the function shown on the slide at number 1. So each Kerberos packet will cause ZIC to allocate around 130 kilobytes of memory, which will never be free later in the code. And we need to send a lot of packets to force ZIC allocate a lot of memory on the machine and finally cause deny of service. I was able to write a simple exploit, which usually takes around 7-8 hours to force ZIC allocate 50 gigabytes of memory, leading to crash. But sending a lot of packets is a bit boring, right? What about one packet that can cause a deny of service and stop network traffic analysis? Remember RC protocol? Decade ago, it was a very popular way to communicate with people in RC charts using ICQ. So the protocol is text-based and pretty simple. You have an American code for command space and then user string. For example, using command 353, which is numerical code for nick, then space, the name, will assign a nick name in RC chart for you. But if we send this command without actual nick name, so the function part begins, part begin will return 0, then part erase will be called with this 0, which will finally, obviously, lead to null pointer to reference and segmentation fault. However, RC protocol is not enabled by default, and we actually need something else. It's usually up to Blue Team to enable or disable monitoring of RC. But the last Buck and Kerberos protocol was a bit more complex. So I don't want to go deep into technical details of this problem. I actually don't quite understand it at this point anyway. But it seems like there is an integer type mismatch in BitPak, generated parser code, and Zeke analyzer itself, that potentially can cause unintentional paths in some situations to be executed, leading to null pointer to reference in the Kerberos protocol parser. And usually it happens 90% of the time. So we just need to send our package three times instead of one. And let's see it in action. So we are on the Linux machine. We have two files. The first file is a Python script that will connect to Kerberos server, and it receives IP, port, and path to the file with malformed Kerberos packets. The second file contains actual data, so actual malformed Kerberos packets. And we have a virtual machine with Zeke. And just to make it simpler, our Kerberos server script is also placed on this virtual machine. So in this script we have to specify IP, default Kerberos port, which is 88, and file that we want to use for response. So let's actually start it. And we just need to specify where the scripts file that we want to answer. Then we can run Zeke itself. It's just we have to specify a name, interface that we want to listen on. And this is our IP address. Basically, it's just a local IP address of our virtual machine. But again, this Kerberos server can be placed anywhere in the network. And now we can just run this client. We just need to specify IP address, port, and our packet. So it already sent. And we can see segmentation fault and stop of network analysis. So this is a pretty straightforward way. Thank you. So this is really a pretty straightforward way to disable traffic monitoring in the network, and then communicate with malware, with C2, or do some rate team engagement on top of this finding. And I managed to catch more bugs with Manu in other applications, such as an open source tool 7.zip. It's Linux version, P7.zip, in Anarchiver for Mac. But unfortunately, these bugs are still waiting to be patched by maintainer. For example, for P7.zip, last time he was online, it was in 2017. I hope he's fine. So again, these bugs are waiting to be patched. I am not going to disclose them today. But I will definitely publish a post about this in future when the maintainers will fix it. Two of them allow remote, like arbitrary code execution. Of course, ManuL is now in beta stage. There are a lot of functionalities that wasn't tested deeply enough, such as anything related to network, any help contributions more than welcome here. What is very strongly required for now is AFL fork server to be able to speed up ManuL and perform network phasing more efficiently. I also want to add Intel P-trace on Linux and Windows. I believe that ManuL can be even stronger with structure ware phasing. So more mutation algorithms, again, is strongly required. And basically, on MacOS, we have a lot of stuff to do. Despite a lot of future work, ManuL already demonstrated that it's a very efficient coverage-guided phaser that can catch bugs in really tough targets, already tested by top security researchers. And I'm going to, I already released 15 minutes ago, it's already public. It's released under Apache 2.0 license. So you can already pull it from my GitHub account and find your own zero days with this tool. So you can subscribe to my Twitter account, add me on LinkedIn. If you have any questions, you can catch me later at DevCon after this talk. Or PM me on Twitter. So thank you for your attention.