 Welcome to VRC3. Hello and welcome on the Franconian.net stage. We now have the pleasure to present the next talk called Fuzzers Like Lego, held by Andrea and Dominic. They both are developers for AFL++, and they are creating a new fuzzing library. If you are new to it, fuzzing means to randomly input data into a program until it starts to crash. From there on, the input data will be changed to trigger more crashes or specific behavior. Have fun. Welcome to our talk, Fuzzers Like Lego. We, that's Andrea Fioraldi and me, Dominic Meyer, are going to talk a bit about building blocks today. Not like these, we all know that Congress loves Legos, but we're going to talk about our software and specifically Fuzzers, an automated scheme to test software for security vulnerabilities, can be built and brought together. So first, let me introduce ourselves. We're both academics doing our PhDs at universities around Europe, and we both play CDF, and we're both part of the AFL++ team. For whoever is not really familiar with fuzzing, AFL is one of the most well-known fuzzers around, and AFL++ is a fork that is maintained by a group of four people, asked to and Mark and had go. We managed to increase the fuzzing performance, overall performance, execution speed, and also just pathfinding and bugfinding over the course of the last one and a half years, pretty dramatically. Here you can see the FuzzBench experiment summary. So FuzzBench is a pretty good offering by Google, giving fuzzer authors the possibility to test their fuzzers against real targets. And here you can see AFL++ 268c in comparison to AFL++ 3.0, which is the latest version and which is pretty advanced in comparison to the old-school AFL when, at the time, we forked it. So yeah, would we want to say that AFL++ is the best fuzzer around? Well, we get many FuzzBench points. So yeah, we have a great success here, but the truth about fuzzing is that AFL++ isn't the best fuzzer. Actually, the best fuzzer is home fuzz. No. The best fuzzer is, of course, the Freedar API fuzzer, which uses Freedar to fuzz. No. Oh yeah, the best fuzzer is libfuzzer, which is included in the Clang project. No. The best is Unicorps, which can fuzz pretty much anything that you have the CPU at. No. Ah, the best fuzzer is Fuzili, which fuzzers. No. No. The best fuzzer is Tomato, which is specifically for browsers. No. No? Okay. Well then, ah, I know. Usually the best fuzzer is your custom fuzzer, which is tuned for that specific use case and adapted to your specific needs. So it may come with custom mutations that are only useful for weird XML dialogs of your target, or there's no off-the-shelf fuzzer for this specific weird architectural thing that you try to fuzz. Well, that leaves us with a bit of a problem here, because if I can't use an off-the-shelf fuzzer, how would it create a fuzz? Well, the usual way is to just fork an existing fuzzer. That's why we have a lot of AFL forks out there. Yes, I know it's funny coming from a guy who just told you all about their awesome AFL fork, but it's true. Our AFL++ fork actually tries to incorporate many of these, but of course it's impossible to incorporate all of them. Well, the other way is you can create a whole fuzzer from scratch, which of course works, but that way you don't reuse any existing code. You will have to spend a lot of time just like doing basic engineering things. You adapt different techniques from different fuzzers. You have to stock it up. You reinvent the wheel and in the end you will end up with a more naive design. So, getting to the point of specialization and to like a performance of something like AFL++ or home fuzz, it takes a lot of engineering effort. And for like a weird target, you are not gonna put that much effort into it. And then lastly, you will not be able to just take your one core fuzzer and scale it to many cores and many machines with ease without even more additional engineering. So, I mean, it's in the title of the talk already, but how would you go about building something that's reusable? Well, our solution is a fuzzing library. Our goal is it to build a library that can be used to develop custom fuzzers quickly and easily. The library offers you basic blocks that can be put together to a proper working fuzzer. And each of these blocks can be exchanged, can be amended. The community can add their own blocks and you can then put the perfect fuzzer for your target together with not too many lines of code. Now we're gonna go through each of these components in fine details. In the following part of this talk, we'll present the concept that we define to abstract the properties of fuzzers. We will give some example, a very simple example that related to fuzzers that you should already know, like a fuzzer and so on. So we relate this abstraction to some possible implementation. We will show some bits of this entity that is translated to code that is Rust in our case. And we hope that the community will profit and learn about this new vision of fuzzing as a building blocks. And all these parts should be swappable, can be revisited without any kind of problems with the other parts. So for instance, you can define a new type of mutator, swap it with the existing mutator and all the other stuff works without the problems with your new mutator. The first component that we will discuss is the observation channel or observer as a abbreviation that is the entity that provides some information about the specific run of the target. This information arrived inside the target. They will usually read only. So just, the fuzzer doesn't use observation channel to instruct the target. They just, the fuzzer just observe. So it's a passive channel. And it is usually deterministic, but not in all the cases. A very, very straightforward example that you should really know is the FL observation channel that is a map in Azure memory that logs in each bucket that represent an edge in the control program. The number of execution of such edges in the current execution of the program. Another very simple observation channel used by official shell fuzzers is the execution time when the other case is fit to the target so it's run fuzzer measure time, typically in milliseconds. And no for each of these cases, how much time is needed to execute it. But apart this very common type of observation that fuzzer does about targets, as the spirit of the library is fully abstract, you can define for instance, a new type of observation like in this case, reachability was the program point. You define the code target program point and the data of the observation challenge channel is just a Boolean that says yes or no if the current run reached this target program point. We discuss the observation channels that are deeply connected with the target and the data live inside the target, we say it. But all we can instruct the target about doing something and the component for this purpose is the executor that basically instruct the target about the current input and run the iteration, run fuzz case, runs the target with the input. For instance, if you target probably runs inside an emulator, the executor will place the input in a determinate area of the memory in the emulator and start the execution of the emulator to run the target with the given input. This is just an example. The executor is deeply associated with the observation channel in our design they are even contained inside the executor. And a possible example, the most simple example of executor comes from Lipsfazer is the in-memory executor that in which the input is just passed as an argument to one Arnaz function and the execution of the target program is actually the execution of this function is the most simple possible executor that you can imagine. Another more complex example that you know is the Fox server of the FL that is a more complex mechanism to control the target using inter-process communication by pre and between two process that one is the fuzzer. There is an intermediate process that is a copy of the target that each time that the fuzzer requests a new execution using the pipe fork itself in the target child that is the actual real target process that is fuzz. And when the target child exits is communicating again to the intermediate fork server that community gate the outcome using another pipe to the fuzzer. So there is the double interaction executor in FL. Now we discuss the feedback entity that is the entity that managed to handle the data inside the observation channels. The main purpose of feedbacks is to produce a fitness value that say if the state of the observation channels are interesting that it means that that this case that is related to the last execution of the executor is interesting that means that it can be added to the corpus of the fuzzer. This corpus is evolved with the during the fuzzing algorithm. It's for instance used for mutations and so on. So it's a score. Again, it's a function with a state that assign a score to the executions. And if the score is okay, that this case is added to the corpus. A very straightforward example of feedback is the one that is using almost all of the share fuzzer that is the maximization map. There is a map like the map that is in the map observer but it is inside the fuzzer this time. It is an history map that keep tracks of the maximum where you see so far in the all the commutative observations made in all the executions. In this case is defined as interesting when the state of the map observation channel is as an entry that has a value that is greater than the corresponding entry in the history map in the feedback. When this happens, the entries in the feedback is also updated. So it's always the state of the feedback is always a little bit different. A very simple usage of the maximization map is to maximize the coverage, the number of the execution of the ages in FFAL, for instance, the code is very similar to this one. If there is an entry that is greater in the observation channel that is greater than the one in the history map, the history map is updated and the fitness is increased. But the very same feedback, so very same code, very same implementation can be used to do a very different job that results in a very different subcommon of the fuzzer but changing a very few lines of code in the target to report instead of the number of the execution of the ages, the size of the locations for instance in one of the possible usage of an maximization map as we want to maximize the size of mallocs to spot out of memory bugs, for instance. Now fuzzing as an objective that in most of the cases is to find the evaluation of some requirements like crashes, demoes, evaluation of some invariance of the program and so on. But as we like abstraction, we define the objective of the fuzzer as a set of objective feedbacks that are just feedbacks that are just like the normal feedbacks that we discussed, but that provides an interesting result to add this case not to the corpus that we use in the fuzzer for the evolution of the state of the fuzzer but in the objective corpus that is the corpus that is not used anymore by the fuzzers just for the user that they say in this set of these cases, there are these cases that comply with your definition of objective for the fuzzer. For instance, in these objective corpus there are these cases that crash the acquisition in the normal case or in a more strange case where we reuse again the example of reachability. We can define the reaching of a specific program point as the objective of the fuzzer. So for instance, we can start fuzzing with a invalid input, put a reachability condition in a portion of the code that we know is reached only with a valid input. And so in the end, in the objective corpus we have all valid test cases for that. A very important entity, of course, in fuzzing is the input that we define not as the input that is expected by the target but as the input that is used inside the fuzzer. What this means that the input is the structure that the fuzzers expect to easily manipulate the input, this structure and then communicate the target an input also with another format. For instance, if we define an input as a thrusher with fields and amps and so on, maybe the target expect by the array, the fuzzer can easily manipulate this struct inside the code. But then when in the executor we instruct the target about the input, we serialize this input structure. A complex example of input can be the abstract since a tree, imagine a grammar based fuzzer and we store these test cases as a tree, not by the arrays. Then maybe the target we expect by the array but inside the fuzzer we manipulate the tree with three operations so I can wink and append and so on. For instance, in a mutator, we can swap nodes and so on. And when the input must be put inside the target, this is serialized to the format that is expected by the target. Another component is the corpus, we discussed about it already. We talked about it as a set of test cases but it's not just a set of test cases. In our model, a corpus is the place in which there are stored interesting inputs and they are metadata, what are metadata? Metadata are all the information that are not internal of the input like execution time, the new entries of the coverage map that are discovered the first time that this test case was added to the corpus and so on. So these properties are related to the test case that is in the corpus and just to the test case in the corpus, they are external, so they cannot be stored in the input structure. But the definition of corpus doesn't end here because it defines also a policy about how the fuzzer should request the test case to the corpus. The fuzzer, each iteration requests a test case from the corpus and the most naive implementation of this policy is a random policy. When the fuzzer requests a test case, the corpus give a random test case. But you can also follow the after approach and each time that the fuzzer requests a test case, you can serve a test case with a first in first out approach using a queue and so on. The mutator component of course is very important because it's the most used way to generate inputs in feedback driven fuzzing and the definition is very simple. One of multi-key inputs are taken and a new derived input is generated. It can be very simple, just one mutation, a bit flip, or complex with like a scheduler mutator that apply multiple mutation and also some scheduling policies that can be defined to operate this mutation. And the two policies can be how many mutations the mutator should apply each run and which mutation it should apply. In a basic fuzzers like FFAR, this policy is a random but in more advanced solution like MOPT, scheduling algorithms can be applied like MOPT which mutation is selected using an history of the effectiveness of each mutation and so mutation that lead to interesting inputs are more priority in the selection of mutation. Deeply connected to the mutator, there is the generator that generates inputs this time from scratch and take as input some parameters. For instance, the probability to expand some rule in a grammar when generating a test case from scratch using a grammar. Generates alerts are used to generate initial corpus in case in which the user doesn't provide the fuzzers in an initial corpus or part of the mutator has a mutation. You generate from scratch a part of the input not just entire input and so on. In some strange cases can be used also as post-projects step in fuzzing. You evolve not a real input representation but a set of probabilities and before sending the input to the target there is a post-process stage that generates the actual input taking these set of probabilities as a parameter. The most simple policy to generate an input can be a random array with some other requirements like just printable bytes and so on or even complex generators like grammar-based generators that are used for instance in out-us also to mutate inputs because one of the possible mutation on out-us is to take a node in the input replace it with a new generated separate tree from scratch scratch. So the mutator use itself a generator as a mutation. A very abstract component also is the stage that is defined as the entity that operates an option on a single test case that this can mean of course nothing is not explained. The idea is that the fuzzer records the test case from the corpus each iteration and the stages are all these actions that are upweighed to just this single test case that was taken from the corpus. The most simple stage that you think about in fuzzing is the mutational stage that there is a loop and in this loop the test case is mutated using a mutator it is executed and it is evaluated using the feedbacks and then if interestingly to the corpus. But it can be also more complex it can use a scheduling algorithm to intelligent we know how many iterations this loop must do. This is what we explore the topic in the literature for instance there are a lot of words that try to maximize the effectiveness of fuzzing selecting these number of iterations in a file it depends on the first core and there are for instance words like I felt fast that find several scheduling policies that depends for instance on rare portion of the codes and so on to give more iteration to interesting input less iteration to shallow inputs. An example of stage is the analysis stage that is a stage that can for instance execute a different executor that perform the debugging and collect the information with the observer and store this information as metadata and then after that you can for instance run another stage that make you of this metadata that can be for instance comparison values extracted from the target to do notation for instance that is a common analysis stage using fuzzers. If you know a file another example is the trim stage that try to minimize the size of these cases while maintaining the same coverage and so on you can define also other different type of stage like calibration and so on. Now I represented in theory what for us are the core concept core entities behind feedback doing fuzzing in an abstract way so they can be almost translated to code not so it's more easy to say that to do but they can but of course they are just theory and to provide a real implementation we need additional components related to the implementation that Dominic will discuss in the next slides. Thank you Andrea. So these additional components that are not really theoretical background of fuzzing things but I actually needed for this library to work and for fuzzers to work let's take a little look at this so in our Ligo house there are these parts on the left you see here already there's a display so we need something like output for the user you see on the very top the blue stuff which communicates with some outside world and then there is also internals of this green thingy which is the random number generator for example so we actually benchmarked a lot of random number generators it makes a huge difference for execution speed which RNG you use then we have of course you know the state pretty easy to explain right it's the state so it contains the corpus and the feedbacks and all entities we had to split it up a bit so the corpus is like a separate thing so it works well together with REST but from a core concept you have the state and each time you run an input it may evolve the state and parts of the fuzzer and then last but not least and pretty importantly we have an event manager so each new event that occurs during fuzzing will be sent out using an event manager we have different implementations of the event manager one of which just displays the changes to the user which would allow for single core fuzzing and another one which is called low-level message parsing event manager which can pump out each event so each time a new test case is found or meta data is added to a corpus it can pump out this event to all other nodes that are fuzzing as well this makes it easy to scale to all cores and in the future may even allow us to scale across different machines the nice things about LLMP is that it is implemented using shared maps so very rarely does any fuzzer instance need to go and ask the kernel for something usually everything is inside shared maps and all of the nodes that are fuzzing just listen for changes on the shared maps so then code well the initial implementations many of these well we involve them a bit but many of these concepts was done last summer by Rishi during our google summer of code A++ summer of code and that may allow us to test out all of these ideas however C is just it's showing its age let's say right now so it doesn't have generics which is pretty useful for this kind of use case and it doesn't really allow an easy way to object oriented programming or any abstractions so what we did was we rewrote the thing in C++ which is the logical next step that leads us to a lot of virtual functions which is slower and then many different templating craziness to get away from the virtual functions so here we are today talking about rust implementation to conclude after a few weeks in depth rusting we can say that some language features are still missing but in overall it's more legible and still performant in comparison to C++ but we looked around the rust community and we found a pretty good keynote from RustConf 18 about game development and the concepts we found inside this keynote we then translated to rust patterns for our fuzzing library and then we ended up with a game state kind of thing which is called the fuzzer state and the fuzzer state contains feedback, executors, corpus and stages, all the things you heard before and then one very special unrusty rust part is an enemy map so the reason is that there are no real downcasts in Rust so what we have now is a hash map that we put anything in and hence the name enemy and we get out the type that we put in later so that way each part in the fuzzing pipeline, each part of the fuzzing pipeline can store and retrieve data at any point in time and then we evolved this concept further to Haskell-like tuples with the nice benefit that accesses are already checked at component what we really came to love is CERDE which is the serialization and digitalization in Rust that takes a lot of effort of writing digitalization away from us so let's go back to scaling we saw that it's just spawning a single thread on the G-Lib C will enable footex on write which makes fuzzing potentially slower than the target prints so we just said we don't have any threads we have a single process in our example implementation and whenever an interesting test case is discovered it's synced lock-free over these shared map channels we have a marginal serialization overhead but afterwards no more syncing is involved this is pretty cool we even added an option if all of servers are the same for each client we can reuse the results and we don't have to rerun the input for a target on a different fuzzer so you can still have clients that are not the same shape so you could have one client that has additional instrumentation but if they're exactly the same fuzzer they will just take over the test case from the other client and go on fuzzing with the new test case you can see here that everything on these ADCores is green green means there's not much kernel involvement which is what we strive for in fuzzing because in AFL band it's known that fuzzing will usually slow down your fuzzing if you use it a lot and you can also see here in this chart in the middle that we actually scaled pretty well so it's almost linear for libpng we have over 10 million xx on this machine you were probably wondering well now you chose rust fine but I don't like rust can I only use this library if I'm a rust person no so the cool thing is this is only the core our libpng test harness already used gcc++ we already have a version using QEMU mode which also works it integrates flawlessly according to our tests plus the library is completely no standard lib and alloc which may tell rust people something it basically means you can include this potentially in a kernel later and almost all targets that any static clang binary can be built for the bad part is it's not done yet we tried to finish it for the talk but there's still some little engineering details here and there that we want to finish before we want to release it to the public so you can already look at the old C-lib which already has the scaling aspects of it and you can expect the rust rewrite to come really really soon if you're super interested shoot us a message we will let you know as soon as possible we can also give you early access maybe and appa++ will stay around for the normal use cases so if you're just one of us you can still do it with appa++ thank you all for listening we've gone through the main fuzzing building blocks and we showed you how we translated to our library right now which will be out very soon we think that we chose good defaults and we will provide examples like libfuzzer like example that you can just use and adapt to your own harnesses and so follow our Github project and us to know more about updates and enjoy the rest of RC3 and thank you so much okay thank you very much for your talk now let's have a look at the questions with Andrea and Dominik right now as I see there are no questions asked in the chat so I have like two short questions that I would ask and if you still have questions we have a little bit of time so please ask them right now so the first question you already somewhat asked it seems to be possible to fuzz applications that aren't written in Rust right? yes totally so the core is written in Rust the library but any place that you can link Rust against which is almost anything you can fuzz in the end and the library is no standard lib so you don't even need to have an operating system around it so you basically could also fuzz like embedded projects yeah for sure that's part of the plan that sounds very good and one more question for me is right now you said you didn't release source code is there like a place where you can update on the status? yes the status as soon as it pops up on our Github maybe also the discord channel the fuzzing discord channel which we usually post information and updates about the project okay so there's a special discord channel I'm not sure if we somehow can add it here maybe we could put it later down now we can in the slides that we will release we can put a link I see there are some questions coming in I'll look at the pad if they're already sorted I'll look directly in the IRC so will the new thing be a tool or is it just a library? it can be both it is a library and alongside the library we will provide some default configurations, defaults, tools that can be built with a library that you can use as a tool you can scale it on for your own tool that is based on the library for example right now we already have a libfuzzer clone running so anything that takes LLVM the standard LLVM input function that fuzzer people will know you can already fuzz that using well it's the library in the end you can get those fast snapshots that is very fast and can be used to fuzz binary only targets on Linux but the idea is that in the end you write your own few lines of code to wrap it up and not have all the command line flags that you now specify for the existing fuzzing tools okay cool the next question from the chat is did you try it yet on bare metal systems? not that far yet we know that it builds but there are some things left for example to use this LLMP message passing you need some place that gives you a new shared map and all this harnessing has to be done and we've not tried it yet okay so one more question I'm curious how pluggable mutators will be if I want to for instance for instance tie in some more protocol grammar yes you can just if you want to do structured fuzzing the two components that you want to override is the input because you need some representation of your structure that is not just a bad array and the mutator is just think about an object in any object oriented programming languages that is just define a new class and instantiate a new object of this mutator class the difference is here that we will use generics and classes that are not comm-setting rust there are traits that are quite similar but not the same thing alright I think right now there are no further questions and yeah so basically that sums it up if you don't have anything left to say I think we're done one more thing about this plugging in mutators we're thinking about making it possible also to write prototyping in other languages for example Python and then plug them in in like a generic fashion but then if you really want to do fuzzing in the end you should definitely use a low level language instead because the jump to Python is super slow it is a long time long term object that we have to support Python because yes it is great for prototyping and experimenting a new fuzzer that sounds very good alright then I'll say thank you a lot again thank you and thanks to all the guys on the tech team or the guys on the RC3 team have a nice day and enjoy the rest of it bye