 Okay crowd, so these are Antonio Jacopo and Andrew from the shellfish team, which is from the UC Santa Barbara and they will Talk to you today about a dozen of years of shellfish and I have to read the sub-table from Defconn to DARPA cyber grand challenge and Yeah, how they automatically Exploits stuff. So thank you. Give a warm applause All right Thanks a lot for the intro So I like to continue with saying who the hell is shellfish so Shellfish is a team of security enthusiasts that has taken has started in UC Santa Barbara some time ago now the original version of this slide actually said that Shellfish is a hacking team and there are lots of Italian in that but then my colleague Antonio Pointed out that putting hacking team and Italian could have maybe they they shoot us or do something bad So we are not related to that hacking team. We actually most of us do research in system security What it means is that they are we are based many of our team members are based in research labs And they publish papers that try to improve the state of the art of what is known in computer security But we also play as team shellfish Many capture the flag competitions as a matter of fact We have even released a couple of the tools that we use both in system security and in capture the flag competitions Some of you may may have used in the past the publicly available anubis and whip a wet so you could submit a Binary or a website and see how they worked and today We're also going to speak in in a little bit more detail of anger a tool that we have released that Team members in the lab that I worked on it have released recently So just to brag a little more who is shellfish shellfish was born in that little building in the UC Santa Barbara Well known as the only university that owns a private beach and that has a as a reputation as a party school I actually got this comment from a flight attendant on a flight so it definitely has some bad reputation but Believe it or not. It's actually a very good research university that has employs several noble prizes none of which are in this room and The shelf is team has also expanded through its alarming to other Research labs, which has people in Boston who never see the Sun and you the common France in Europe and As you can see we are relatively international team We have the majority of players are originally from Europe Of course, we have a good representative of people from the US some very great hackers from China and Asia Brazil Senegal and unfortunately, we have no one from Oceania if you want to we are very open to recruit people Join one of our research labs As a team as I was saying we play CTF So I'm sure many of you have already played capture the flag competition So I'm just gonna give a very minimal primary is gonna be useful for us later CTF is a security competition in which you do You are presented with a challenge can be a binary program a service a website sometimes even a hardware device and The base there are many variations But the basic idea is that you understand how that works you find where the secret is You steal it from the other guys or from the challenge you submit that to the organizers and you get points and Shellfish has participated in many CTFs including a DEFCON CTF It has even won the DEFCON Finals in 2006. None of us were actually on the team at the time, but it's in a sense good memory, but We do not only play CTFs If you you may be familiar with the UCSB I CTF I CTF is an attack defense CTF in which you not only Crack programs to steal the secrets from others. You also have to defend yourself and patch the binary so that other teams cannot steal secrets from you, but the service is still functional and Giovanni Vina Was the originator of the I CTF it used to be as a matter of fact the final of the security class of Giovanni and it has Evolved many many times over the years The I CTF has in many ways tried to innovate with a different style every year some years have been very Work have worked very well. Some other years have not worked that well We have in a sense. We have had many many discussion even internally I can assure you of that but we have in a sense embraced the nickname of chaotic shitstorm and we have gone with Very in my opinion very brave and very good attempts to innovate If you have played in the last years for instance, you know that we have tried to give Higher importance emphasize the importance of network defense for instance or try to see what things would do if presented with a huge Number of 42 services and to see would they try to sort of defend all of them with the same approach attack All of them with the same approach and in the latest year. We've even gone with the full player versus players Player versus player even the program themselves were written by the players of the competition itself So if you want to hear more about the I CTF our base framework is The base framework that we use for the competition has been released in open source We're in the process of preparing the new version and you can also read Our paper of ten years of I CTF to see to get a little glimpse of what has been done over the years But now let's get back to the juicy star. So why the hell are we here? Why should we be listening to us? Basically, we want to show essentially how you can go from this to this and We've been told is a very American style slide and say Yeah, I'm telling you guys have really gone American if you're showing this But I like to have you know that most of these North are actually one dollar and five But I think there's a five dollar yet that's down there So it's not actually that much money, but shelf is actually on a more serious note shelf It's actually has won some money by participating to The diaper grime and grand cyber grand challenge and we're gonna explain a little bit to you what that is What it involves and then we're gonna do go a little bit more in detail and give you an idea of how we decided to Play the competition and we're gonna give not all the components obviously are public for obvious reasons We don't want to make the life too easy for our competitors But we're gonna do give you an idea of some ways that you can use to automatically discover vulnerabilities And in particular, we're gonna give you a live example using anger Anger is an open-source binary analysis framework that can do many things even more than we have used for the CGC and obviously many more than we're gonna show you in the live demo And it's an open-source framework that you can use yourself as a matter of fact if you're interested in anger And so also becoming better each you can join us Later after this talk in we're gonna organize a worship and you're is gonna be in charge in Hole 13 she's one three. It's on the first floor. So if you want you can join us later We'll give you more details about that later and Finally, we're gonna give you a hint of what are the next steps that we're gonna take the teams are gonna take to Participate to the finals of the cyber grants and Cyber grant challenge as you can see we're also very fun of acronyms like DARPA so Let's start with a basic idea So I think most of you guys have heard of DARPA DARPA is a research agency By the United part of the Department of Defense of the United States of America Many of you may be familiar with the role that it has played in the creation and evolution of the internet and DARPA's mission is in a sense to among other things push forward research in approaching interesting problems and Occasionally the probably the most famous one that you may have even heard in the press is And in 2004 they started a grand challenge No one was able to do it before Make a car that can start at a certain point and alone completely alone autonomously drive from point A to point B and You know nowadays we hear there are many very good Projects on making self-driving vehicles many of which are actually realized in 2004 believe me It was not easy at all as a matter of fact in this 2004 grand challenge none of the vehicles actually reached the finish line. So in a sense DARPA as in a way Found a problem that no one was solved and tried to push forward to see I challenge you Can you show me show me what you can do to face this challenge? But now as I was saying there are there are cars that do that So now in 2014 DARPA has decided to start a cyber grand challenge And once again it means completely autonomous thing completely autonomous Exploitation and patching other programs and yes what you see right there is a competitor So as a team you'll have this array of servers free to do whatever you want with them But you have to play a game and the game is what we're gonna talk In these little next slides, but first let's just give you an idea of the current progress of the competition The qualification has already taken place. It was in June this year seven teams originally joined the challenge and Seven teams have been selected as qualifiers after this after the qualification event the finals will be August next year's and will take place in at Defcon in Vegas the day before The Defcon CTF finals and as a matter of fact the winning system will play Against the humans and that's all for the best that the humans still win because otherwise we're gonna have quite a bit of problem and We're gonna go a little more into details But if you want an intro in a sense a more usable into DARPA has a very nice website very flashy in a sense at cybergrandchallenge.com So let's talk about a moment more about what is the CGC so No surprises an attack defense CTF exploit patch and Again, you probably as you can probably imagine the way you participate as a team It's not that you're given a program you look at the program and you exploit it And it's not even that you look at that program and you've write the program that Exploits that guy it's not that you've write the python script that exploits this specific program What you need to do is you need to develop a system that is gonna be fed Some programs and automatically without human intervention will find vulnerabilities exploit and patch them, okay Sounds simple sounds easy It's not that much It's actually interesting. I like To you to take a moment and how would you play what would you do it's kind? It's an interesting it's an interesting question and it's what in part led us to To work on this but I would like also you to reflect for a moment that this is also not that trivial to organize And I think DARPA in a sense Merits some praise in the sense that they have had to take some interesting choices. For instance, how will you prove that the Binary was exploited now as you were saying before, you know In classic CTF you're given a binary and you know, maybe the binary There's a flag. There's a file called flag You need to read the flag or you examine the thing you see oh, there's a winning message and there's a losing message I need to make the program print the winning message or maybe it's a storage server and the secrets are the Secrets that are stored in these files that have certain names and so on and so on and so forth as You can imagine this is a lot more complication for a program to analyze This is useless complication in a sense So DARPA introduced for the calls for the qualification around a very simple way to define what is an exploit You have exploited a program and you demonstrate that you have exploited it by giving an input that will crash the program segmentation fault whatever if you create the benign program will not crash If you make the program crash good you find you found an exploit and you're gonna get the points for the exploit Easy is it easy? Well, don't forget that the competition also includes a defense component. So What are you not what do you think is gonna be the defense is it just gonna be that you know The program just needs to run until the end. I mean if it if it doesn't crash and it doesn't time out It's good. I Mean obviously you could say well, yeah, that would work but then I can just submit this as a patch program and Believe, you know, this is the what is usually if you have played something CTF's you may have heard a term the Superman defense Usually in this competition you want to avoid organizers and even players of course want to avoid A defense systems that work every time no in a sense no skill required a very little skill required or in a sense It's working every binary. So how do you prevent this from happening? Well This one is relatively easy. Basically you need to put in some functionality checks. So that means that whoever rides the program Not only you write the code of the program not only you put in a vulnerability in the program You also need to put in a Some scripts that will check the functionality of the service for instance if the service is a palindrome finder You need to write some script that send words and see if the program correctly says palindrome Not a palindrome and goes on and on and of course You also want to make sure that these checks are in a way Meaningful and not easily patched for instance the program just admits two Good value of inputs and two easy answers to those two or even complicated answer then you want to avoid that, you know, the the team can pass just by thinking and If this input then this output if these other input then this output otherwise exit zero So again, this takes a little bit more skill than than it sounds This was a secret. No What is another way that you can build a Superman defense another way is that you could have what is called a Out of band error handling. So you're running the service. You're running the program the program for instance tries to use a Unmapped address and the program is going to get a segmentation fault and ordinarily this would be considered a crash But if I can install a signal hander, then I can turn all these crashes into benign Exits and I since I know that for benign inputs the program will never do a six egg I'm sure that I can do exit zero without compromising the functionality checks. So once again Superman defense my program cannot be exploited and Solution for this is that in a sense or at least a solution that diaper took for this is that There is no out of band error handling. There's no at least no easy way to do out of band exception handling error handling But as many of you may be familiar with terms such as dbt or an emulator like the mommy for instance the emulator for consoles What would prevent me from writing an interpreter, okay? So I don't actually run the x86 instructions You understand like I can run these instructions is my little VM my little VMware or virtual box or mommy and Whenever I detect that the CPU would raise an exception for instance because it's again segmentation for out of band, you know accessing unmapped memory division by zero whatever Again, I'm gonna exit zero Now, please take a moment. How would you prevent me from doing this? It's not as easy as it sounds and that by my opinion took the elegant and also very correct solution You don't This is perfectly allowed in that pa cgc The the the point is that you pay for that. So sure you can do whatever you want You can even write an entire binary translation platform in fact One of the qualifying teams Non-qualifying been sorry trail of beats actually did something close to these we obviously don't know the full details But they did an transition and LLVM intermediate and they went back and I encourage you to read the blog post from them because they No, no, but not because it's a stupid way It's actually a very good way and you will see that in a sense you run into a risk that You need to make a very good estimate of What's the cost of the defense that you're putting in if the defense that you're putting in ends up consuming too much CPU or maybe too much memory then You may lose points in your even if your defense is perfect in itself You may have lose you may have lost points in this and I'm just going to give you a couple of extra ideas of the competition Obviously you want the competition to work on Realistic programs you want programs that can do more or less what the programs that we are on our laptops can do So no hello world only, but you also don't want extra complication and What do I mean by this it's a little fuzzy as a term, but basically you need to think of this that the Advantage the work that the teams are gonna put and the advantage that you're gonna take by looking at how they do things Things have a limited amount of time So do you really want the modeling that the team to spend a ton of time modeling the entire? You know horrible things like interruptible syscalls or an entire model of file system and so on and so forth so Usually not you need to find some sort of compromise and again. I think DARPA found a good compromise here The architecture that is being used is regular Intel x86 All opcodes all user land opcodes at the very least are fair game So all the problems that I'm sure you're familiar with such as problems in disaster and Lee and so on and so forth are all there So it's very realistic but the OS is a somewhat limited form of a Linux like Unix like operating system as you can see you have some basic syscalls transmit and receive they're basically read and write from from the standard in started out and Sockets that connect programs together. You also have a form of select. It also allows you to sleep weight you have some form of Manlock and and unmapped so to speak but this can even allow you to write to map Executable code pages. So if you want you can even write a just-in-time compiler in this in this in this architecture You have randomness. You have the way obviously to exit the program Probably the biggest Simplification with respect to the capabilities of a full Real-world Linux program is that there's no shared memory no threads with shared memory Particularly interesting at least to me is that DARPA took the bring-your-on-defense approach really to the to the maximum extent so You do not even have by default the usual so to speak the defenses that are common nowadays So the stag is executable like it was ages ago. There's no asli. There's no nothing you want this stuff. You have to Bring it you have to Patch your program modify your program so that it has them. I I think it was an interesting Interesting idea from DARPA and as we have seen in the case or in the trail of bits case This brings we did sell some non-trivial choices on the on the team side So now we're gonna give you I'm gonna pass the torch to my colleague Antonio And who's gonna introduce a little bit our cyber reasoning system. So our the way we play the game So I'm gonna introduce the cyber reasoning system we developed that what is a cyber reasoning system is just the way DARPA defined The system we have we had to develop to participate to to cgc And so as the aquapo introduced as an input we have a list of binaries and for for each one of these binary We need to produce an exploit that is an input that is gonna make these binary crash and a patch version of this binary that is immune to these exploits, but still preserve at least some functionality and So let's see our system a little bit more in details. So we have an out Basically the the vulnerable binaries are passed to an automatic of vulnerability finding component That is of that is as the name says try to automatically find Vulnerabilities into these binaries and then both the exploits generated by the system and the vulnerable binary binaries are passed to an automatic patching component that is trying to Generate the patched version of these binaries and of course and some of the patching strategies actually require exploits to generate patched version of a binary that are immune against a specific exploit so that's the reason of that hero and both Proposal exploits and proposed patches are evaluated by another component that is evaluating them in terms of our How good a patch version is is it preserving functionalities is crashing because there was some bug in our system So we are evaluating this and of course the best exploit and the best patches The best patch we have for a specific binaries is finally submitted to to DARPA for for their evaluation so that's the general idea of our system and For the next of the presentation, we are gonna focus on the automated vulnerability component and so let's see what we did so So here for for the qualification remember that An exploit is just a program that is making it is just an input that is making your binary crash. So That that's the question we want to answer How do I crash a binary? But this question is a some somehow a subset of a bigger question of a much more difficult question It is how do I reach a specific state within the execution of a binary? So this is an undesirable problem in the in the general sense but there are some approaches that can solve this problem in a good amount of cases and So I'm these are the two approaches. We we we use so one is is a fuzzing and the other one is a symbolic execution, so let's see First as How these two approaches are a work in general and then how we we use them So about fuzzing so The quest the so as I say the the question we want to answer in general is how do I reach a state? Within a program and if you see at these code that I put in the in this slide We have a very simple problem program that is getting an input from the user Putting in a variable hex then it does two different checks and then it can either print you win or you lose So suppose that we want to understand how the text you win is printed Which is how which is the input we need to provide to set to reach that state So of course the easiest thing we can do is just to test a lot of inputs So we can try with one and you lose is printed we can try with two still you lose But when you try with when when we try with ten we will reach that instruction So our we will reach the state. We were looking for so fuzzing is in the in the simple way It's just trying a lot of different inputs, but specifically for for cgc. We used We use fuzzing and we use a way of a method called coverage guided fuzzing that means that the execution of the program The program was executed in a instrumental environment environment that was taking track of how much a given input was Covering of the entire code base of the binary and these system was focusing on Basically on those inputs that were more promising So those inputs that were increasing the most the general coverage of the of the program so our system was based on an already existing system called AFL that's the link to the original version of AFL and This system was really was was effective, but There are also there are specific cases in which this system cannot work or at least cannot work efficiently So let's see an example So here we have the same code as before But they are the only there is one difference in the second in the second conditional instruction So yeah, X needs the the square of X needs to be equal to that big constant value So of course, it's really unlikely that just by trying a lot of different inputs We will actually find by by luck or by any method the specific solution of that equation so As a possible solution we can use symbolic execution So let's go back for a second to the for to the original example So the general idea of symbolic execution is to interpret the binary code and replace user input with symbolic variables So the first instruction is getting some input from the user We don't know what the user is putting here as a in the general case And so our symbolic execution engine will just keep track of this by putting a variable X Within the the current state of the execution so then we go on executing the program normally and But of course if we if we if we reach a Conditional instruction like this we don't know which kind which of the two branches We need to take because we don't know the value of of X So what we do here is that we take both branches We take both branches, but we we keep track of the constraint that a branch is Is is putting on a specific path? So in the path in which in the in the first branch of this a conditional instruction Of course if we take it X needs to be bigger or equal than 10 and in the other X needs to be less than 10 so The execution goes on keeping track of these constraints So suppose that now we symbolically execute starting from the state on the left So now we encounter a second a second conditional instruction and as before we generate two different states in one X We have an additional constraints We have an additional constraint that says X needs to be less than 100 and in the other state We have a constraint that says X needs to be bigger or equal than 100 So but now we reach actually the condition we were looking for that is in this case the text you win is printed So when we reach a condition that we were looking for we can ask to the symbolic execution engine to actually to Concretize this input that is to give us a To give us a a concrete value that satisfy both constraints in this case 99 and if you in fact put 99 as An input you will reach that condition and of course if we go to the more complicated example for the symbolic execution Is not engine is not that complicated to deal with this We just have a different constraint, but still any symbolic solver can still solve these Equations and give the correct answer that is one two three four five So we use a symbolic execution for CGC in particular We use the symbolic execution engine of anger that is the binary platform develop the binary analysis platform develop at UCSB And in particular we were looking for two for two different conditions. So one is memory accesses outside the Allocated regions and the other was Unconstructing instruction pointer So I'm constrain instruction pointer means you have for instance a jump instruction that is jumping to the value of a Register but the value of that register is coming from the user input So the symbolic execution engine doesn't know where to jump So why we were looking for these two condition well, because if you reach one of these two condition It means that you also found a crash an input that will make the program crash either because the program Given that input will access a not allocated memory or will just jump to non-existing code and So as in a general idea, so we we for for the final we are exploring the idea of combining the two approaches and In particular a paper has been has been accepted at the NDSS conference and it will be published in February and It will be presented in February at the NDSS just to give you a Preview of the results of that paper. So if these are the entire binaries We had to solve for for CGC the for the qualification these In scale more or less. So these are the amount of binaries we found We can find a so an exploit using fuzzing. This is the amount of binary. We can find a An exploit using symbolic execution, but combining the two we can find even more binaries even more exploits and so the bigger circle is the amount of Binary of binaries we were able to exploit using this new approach. So I suggest you to to read the paper as soon as it will be public so Okay, now Andrew will introduce anger and he will give us a live demo Hello, I'm audible good I'm gonna talk about anger. I'm angry one anger is a pretty It's a Python library you use it it has code you can run the code and it does binary analysis It does all the things that Antonio just finished talking about and more It's open source because we believe in that sort of thing and It's interesting to note that anger started because of one of the other graduate students in our lab Wanted to do a project on firmware analysis and got carried away and since then that project has grown and become adapted to Cgc and Linux binaries and all those different architectures. So it's had its the multi-platform nest baked in from the beginning and It I said I said it is written in Python. This is just because we're a research lab We need to move fast and break things. We don't really have the luxury to program and something like C++ or OCaml not that we'd ever want to and because of the magic of Linux and the Python package index you too can have it with those one or maybe two commands that One is if you're willing to forego the support. Yeah, we are providing you So what I'm going to do here is I'm going to Demonstrate using anchor to automatically find vulnerabilities. The automatic part is I will see So we're going to show off the finding the grub back to 28 vulnerability Which you may have seen recently There's been a lot of news coverage about a vulnerability in the grand unified bootloader versions 1.98 through 2.02 that if you press backspace 28 times and then return on The grub password username prompt you will get a rescue shell which is an interesting vulnerability and It is due to an unchecked integer under flow in the username and password input functions So as you can see on the screen There's a link to the actual write-up of this vulnerability which you'd copied a lot of stuff from for the explanation and the Specific vulnerable part of the code doesn't check under for those. It's just simply decrements the indexing to the current input buffer so then later in the function and what it does is it clears the unused parts of the buffer with this memset command and The problem is that is that you can make The current length negative. So if you make the length negative all of a sudden the It's it's memsetting Other parts of the stack like the return address So if it sounds it sets it all to zero then it's good the program is going to jump to zero and crash except that This is in x86 Very low-level stuff and somehow this is possible the explanation of why this is this can be exploited successfully as well outside the scope of this talk, but please read that write-up. It's very nice so Finding this automatically is going to be very difficult because See on there on the screen is a control flow graph of the vulnerable function in the grub binaries And as you can see it's got a big loop with a whole bunch of paths And if you take a symbolic executor and throw it at this problem and say finding me a crashing input The universe is going to grow old and die before anything interesting happens It's too much state. You're going to go through the loop once you're going to produce ten successor states each one of which has gone through The loop then each of those ten needs to be put through the loop again they will all each generate ten children and you're going to have an exponential state explosion and Your computer is going to run out of RAM realistically so I have a demonstration here of The naive approach of symbolic execution the grub naive the grub naive script Which is just a using anger as a naive symbolic executor so most of this Most of this is simply setting up the grub state because we I'll be realistically cannot Reckless acute symbolically grub from the very beginning of program boot. So we chose we choose to Start our execution in the program. This is something that anger can do you can Construct a you can construct your own state, which is the land that does the state construction It's down here. It just constructs a blank a blank slate blank state that you can use to symbol symbolically execute as symbolically as you like and then simply simply Steps the path group until it finds until it finds something that has jumped to zero and As we see here I've also said it to print out its current internal anger state at every step We can see that very quickly. It's Starting up. It's executing lots and lots of Symbolic states you can see that the active number is becoming larger and larger as the program slows down more and more And if we look at my Memory consumption and you can see that it's slowly going up and it's going to run out eventually We're not going to let it get there because I like this computer In So in general the security community has attempted to solve this kind of state explosion problem in loops with a technique called vera testing That's to make by Carnegie Mellon University paper Anger implements this I tried it. It doesn't work because You see it simply generates too much complexity for the constraint solver to handle It's a bit more of a technical discussion that we can have later in the workshop however anger does let you solve this problem because Because Symbolic execution is powerful, but it's really dumb. It's going to take spend all this time Computing things that don't need to be computed but and so You on the other hand you are very clever And you know how to smartly navigate the state space of this program in order to not waste your effort But there's no way that you could realistically do that with the precision and care of a symbolic executor So the solution is anger anger provides an interface for you to channel your anger effectively Search your feelings. You know it to be true So what you can do in order to solve this vulnerability this is what I did last night to prepare for this demo is to exit to manually examine all the surplus of Symbolic states that anger produces and figure out What's going wrong? And you can see that it's doing all sorts of things you would never do if you were Personally exploring the state space of this program like entering several letters in a row and then building a press in the home But multiple times in a row. You don't need to do that Yes, yes So What you can do is you can just tell anger to not do these things and anger how it provides The power of anger is it provides reasonable interfaces for you to do this more easily I took my previous example the naive approach added ten lines of code and It finds the bug as we're going to demonstrate. So we say grub bug and Still prints out but now we can see that it's finding a whole bunch of not unique paths that it's discarding And now the computation load is very easy for the symbolic executor process and it finds the crashing input Which is 28 backspaces in a return? So that's a lot of the demo just a couple of side notes that This is I am you look if you look at the If you look at the script itself you can see at the top It's straight up loading the grub executable out of my operating system I haven't upgraded done a system upgrade in several months. I'm still vulnerable. Please don't hack me But what I thought I was previously doing is I was previously using trying to exploit the 64-bit version And it didn't work because this exploit doesn't work on 64-bit systems because of some the way the compiler uses Rx or cax and it's a mess, but I didn't know this until I had written the entire exploit and Found that it didn't work So after a little bit of manual analysis, I realized this was the reason So I just switched over to the 32-bit version and all I had to do in the anger script was change the word sizes and The program address anger is incredibly versatile and it can handle all different kinds of programs It can handle binary blobs. It can handle firmware images. It can handle PE and elfin's maybe Mako files and I don't know there's My opinion is the power of anger in the in the interface and I recognize that this is a very Dissonant message from what we were talking about earlier with the automated hacking stuff But the point is that this is why the cyber grand challenge is hard the secret sauce in our CRS is not something that we can talk about under penalty from my professor can't do it and That's so do so. I'm emphasizing how User how user-friendly anger is because that's the way that you're going to be using it We're going to be using it as an automated system, which is a lot of really complicated algorithms That's all I have to say back to One of the two I forget which to talk about the finals of the cybercrime challenge so Let's go back to something a little bit less technical so We we so we played it with the qualifications seven teams best as you have to say Sheffish was one of them in particular. We exploited 44 binaries out of the 131 provided by DARPA and every qualified team received $750,000 as a price. So that is a good thing to have and Okay So for the finals the setup of the competition will be different In particular many different many things will be different So the the finals will be around base attack defense CTF so for the qualification we just receive all the binaries and then we had 24 hours to Automatically find exploits patch them and submit these these patches and these exploits to to DARPA instead here It's a round-based attack defense in which at every round we receive new attacks from other teams and we can submit new patches So the this is not 100% finalized right now But there will probably be no possibility of human intervention or very limited possibility of human intervention Whereas during the qualification phase. It was at least possible to fix bugs for instance if you forget to put the file as executable Because you didn't sleep for the previous week so something like that and Maybe there may be some possibility of back fixing you even during the finals because of course everyone wants to avoid that you work For two years on something then you forgot you forget some a semicolon and all your work is pointless Since something like this happened for the for the grand challenge with cars So they may want to avoid this somehow, but they don't want you to adapt to Exploits sent by the other teams. So you need to balance these two things So another interesting point is that data about previous rounds is available. So first of all You submit exploits and patch and patch version of binaries and you can get from the system the performance There are they have so you can adapt based on that But also anonymized traffic sent by other teams against you you can receive it and you can analyze it And you can even receive at every team patches that other teams a patched version of binaries that other teams Submitted so this brings it to a lot of interesting Scenarios in which you may be able to steal patch binaries from other teams You may be even able to steal exploits So there is some probably some metagaming you may want to do about deciding how to patch what to patch Maybe you can just steal a patcher and and wait Before the value before deploying your patch. We don't we don't know yet, but there may be some meta gaming we need to reason about it and Yes, also this important so exploits are going to be different So for the qualification and exploit was just an input that was making your program crash for the finance There will be two different types of exploits. So the first type one is basically an exploit is is a Is some code that receive a specific location and specific register and a specific value from the organizer And then this code generates an input for a binary that is making the binary crash at that specific location And also setting a register to a specific value So this is somehow a way to the DARPA has to check that you kind of reach a state in which you You actually exploited the binary. So the binary is under your almost full control also another kind of exploit is type 2 exploiting which you have to leak data from a specific memory page provided by the organizer and And so for this we we will need some more realistic exploit generation And we are working on using hangar to do something different that is to automatically generate a rope chain as a part of the exploits Also as in addition to all of these there will be the possibility to deploy network level filtering rules so during the competition there will be an Intrusion detection system from for which you can provide rules that prevent traffic to reach your services and For instance rule can be if a packet says the string just block it But of course you want to deploy these rules But in a way that you are not impacting the functionality of your services So these are a completely new area. We have to explore So every team has access to a cluster of 1,280 cores 16 terabyte of RAM and 128 terabyte of storage this is a picture of one of it and So right now we can access to this through SSH But for the final they will bring seven of these to one for each team to the Hotel where Defcon is and so it seems that they will also bring a half a megawatt generator to power all these things and cool them and And so of course the first thing we did when we we got access it was to run a stop And then we spent other two hours trying to run a stop on all of them at the same time Using using terminator Yes, that's a huge screenshot of like 5,000 pixels something like that so Okay, so finance will be we'll take place in august 2016 at Defcon Las Vegas there will be money prizes So the first place is $2 million second place $1 million third place is a $750,000 and as Jacob will say the winning team will compete against human teams at Defcon city at finals And it seems that the winning team will have access to the entire Seven cluster so the winning team will have access to something around 100 terabyte of Rams something like that So and it will try to compete against humans So we don't know so because of these because of these it seems that Next year as Defcon CTF finance all the binaries for for the finance will follow the CGC format so that humans and These automatic systems can play against each other and so I just want to conclude with a picture with all of whole of Shelfish persons that contributed for the qualification you can also see my Photoshop skills and and so then since the qualification other two members Start started working on this and as you can see we are from all over the world and Italians the number of Italians surpass the number of Americans and okay, and So we will have a workshop after this talk In all 13 and so let's bring your laptop and we will show you a lot of things so all 13 I have a map, but it's really hard to find rooms here. So just follow us. I think that's the easiest way and Yes, really not that hard Good and we are gonna show probably some easy example that you can you can just do it by yourself And then probably we're gonna maybe show more in details what anger what Andrew show us during the demo and With this I put some references the DARPA presentation at Defcon. It's on YouTube That's pretty interesting if you are interested on CGC. I put our emails some Websites and so we are ready to answer to your questions Thank you. Thank you very much. That is good thing You can leave if you absolutely need to please be quiet while doing so and only do it through the stage Exit so Keep the voices down. You don't need your voices to leave the room so that People who want to ask questions can ask questions so I There's a bit of Turmoil now we wait a few seconds until that clears up The same Just for the info There are how many questions from the internet. Okay, I get that Okay, so not waste any time for the Q&A. We will just start so Let's have the first question from the front microphone on the right side, please Hi, I'm from one of the other teams who have previously participated in Defcon my question is Previously it has been kind of hard to get to get funding for IC security research at least at least if you're doing CTF like stuff. Why do you think DARPA has? Put this huge amount of money Into into this great cycle challenge is what do you think their motivation for doing that is? Do you have an answer I mean on one side I guess you could say that maybe the situation has evolved enough and I mean if you follow for instance the For instance AFL has definitely proven that it can find many many bugs almost automatically if you follow the Conferences you'll see that static analysis has made Many many improvements or maybe DARPA thought that you know the It's ripe enough to start challenging people, but I guess ultimately it's a question for DARPA Okay, so let's have the next question from the internet signal agent Yeah, a question from the internet is what kind of answer you got for your freedom of information act request So can you repeat? In one of the last slides we saw an example of a freedom of information act request and Internet was asking if you got an answer on that or what what kind of answer you got a freedom of information request That's what the internet is asking Good, okay. What can I say is that I think after the competition? I think we are gonna release most of the code if not all the code so But I don't understand the question And and also not another thing so all the submitted binary all the binaries Submitted for the qualification and all the so all the patch binaries all the the exploits They are available on the DARPA website So if you even want to understand what teams all the teams did you can just get that those binaries reverse them And you can have an idea what teams are doing and that's all I can say yes and DARPA also has Man not all but most of DARPA's code such as the Linux modification and so and so for every all of that is public is public and Yeah, so if you want to take a look I believe at the moment you you can get a good look and certainly after the final This would probably improve a lot more. So I think there is a good amount of release of information obviously I mean we don't control everything. So and the competition is still ongoing. So Okay, is that a question on my right side on your left side. No, that's no question Do you have a question you're standing people are just saying okay, is there another question from the internet? There's no question from the internet That's what be then the perfect point to wrap up and oh there's there's one one microphone question over there One last question do you think two million is a fair price for an automatic exploit creation tool? Do you mean more do you want do you think we should get more? I think you should get more Yes, it's very this is very difficult and it's this is it's consuming the time of Basically, it could consume the time of all of our researchers It cannot consume the time of all of our researchers because there are other projects our lab must be doing But it's an incredibly difficult program. It is ten and out to creating The first I it's not quite the first step to a like a terminator-esque dystopian hacking robot future, but It's very good inroads if you're into that sort of thing. No, I mean you're giving them a tool to Some improvements exploit almost every binary program for only two million Well If anybody in this room would like to talk to DARPA about extending the grand prize I would be happy to help them with that But at the moment that even the $750,000 is going to fund shellfish going to Def Con and eating pizza for the next 20 years Okay so Thank you Antonio Jacopo and Andrew for this very nice talk and be sure to join them on their workshop