 Good afternoon It's a wonderful honor to be here today tonight or this afternoon. I'm gonna be talking about capture the flag Capture the flag is a contest. We like to run at DEF CON DEF CON you're here. We've run capture the flag at DEF CON in Las Vegas for several years now in 1996 the first capture the flag game at DEF CON was held and In the year 2000 so after four years they formalized how it was run So for the first few years how capture the flag worked at DEF CON was players would either bring software that they wanted somebody to hack for them or They would hack somebody else's software and the human judges that were Authenticating and running the contest had to make these decisions every time something got hacked whether it was easy Whether it was hard who got the points and it was very very confusing So starting in the year 2000 the DEF CON goons running capture the flag created rules and they created services So starting in 2002 and running through 2004 a group called ghetto hackers took over capture the flag And this is kind of how capture the flag became a name brand event where the ghetto hackers had a different brand name Slightly different from DEF CON they had continuity from year to year Starting in 2005 a new team called kenshoto took over and this is also when I started playing capture the flag Kenshoto had a much more difficult game that made a lot more people more interested in capture the flag starting in 2009 the previous CTF team of School of Root turned into this team called DD tech and started running capture the flag themselves And then starting in 2013 and running up until last year my team the legitimate business syndicate took over and Starting literally tomorrow the new team order the overflow is taking over running DEF CON capture the flag So the way we think about capture the flag and how capture the flag works with DEF CON is that there are two distinct formats This format is Jeopardy format named after the US game show and how this works Is teams pick a question or a challenge from this grid they receive a prompt and solve the challenge for some points The main part of this game is the scoreboard that we just saw on The scoreboard you find a prompt in this case the prompt is giving us a network address Including a port number and a file to download From there we solve the challenge in a lot of cases these challenges involve a lot of assembly programming or reading assembly And they involve Python Once the challenge has been solved you return the challenge or the answer to the question to the scoreboard for points and Then you go on to the next challenge right here We see a handful of challenges that have been solved by somebody with different point amounts and one challenge That has not been solved yet and that will give the Person who solves it scoreboard control So DEF CON capture the flag qualifiers is one of the most famous Jeopardy styles CTFs in the world But I'm going to talk about a challenge from a different event the Shaw 2017 capture the flag contest so Shaw 2017 was a Outdoor computer festival held in the Netherlands last August the week after DEF CON and I went there and had a wonderful time And part of the wonderful time was a contest that the Dutch team Einbosen ran at this event I'm going to talk about a challenge called ASB With ASB we get a file This is the prompt it has a little bit of backstory where their team member named ASB got a reputation For solving challenges the wrong way So once I downloaded this file, I discovered it was a Windows executable and Because I just you know downloaded this executable randomly from the internet. I decided to run it and figure out what I had to do so the goal with this is we need to enter in the Flag and the executable will tell us if it's correct or not and starting from here. How many characters are correct? So I need to guess the correct input So I have three ways that I can do this I can either reverse engineer a Windows binary and I'm not very good at reverse engineering And I don't usually use Windows so that was kind of out. I could guess each character by hand which Looked like it would take a long time or I could write a program which I foolishly believed would go really really quickly So I chose to write the program and this is the program It's a little ruby script and mostly what it does is it opens the wine emulator with the ASB executable I Hand guessed a bunch of the flag before I tried writing the program So I have that part of the flag at the top and then we just loop over and over Guessing which of our flag characters goes into the executable and tells us it's correct If we get to a point where something weird happens we print out what we have so far and exit In theory after a long time writing this program and switching back and forth between oh, I can't program computers anymore I'm just gonna do this by hand. I eventually got the solution The solution is right here right above the stack trace, which is cool like I solved it I got some points the organizers gave me some beverages while I was sitting there and I had a lot of fun So that's Jeopardy style you get a challenge you solve it and you get some points This is the visualization from our finals game, which is an attack defense format capture the flag It's very very different here. We see teams Shooting successful attacks at each other in a soundboard theme or like audio mixer themed event So how this works is all these steps happen at the same time Teams are reverse engineering the challenges or the services They're patching the flaws in the services at the same time. They're trying to exploit other team services And in all of this they're trying to not break their own services to lose points So here's a small example of how attack defense works at the top We have the scorebot or the scoring system of the capture the flag game on the left We have PPP's service called at mail and on the right. We have a team called shellfish so Every round the scorebot deposits a new flag or a new piece of random data in PPP's at mail service and everybody else's services as well Sometime during the round shellfish You manages to send some network traffic to at mail that causes it to disclose this flag or as we call it Shellfish has stolen the flag Then they take that flag submit it to the scorebot or they redeem it to the scorebot and get some points in The meantime scorebot is checking PPP's at mail service to make sure it's working. Okay Can it send messages? Can it receive messages? Is it happy? And if their if PPP has broken their at mail if they've you know Turned it off or deleted it or just written a bad patch that makes it do things wrong that's a failed availability check and We assume that if there's a failed availability check that shellfish is also not able to steal from it So we make a failed availability check Lose a whole lot of points and again. This is the format we used for DEFCON capture the flag finals Here's an example. It's a service called Rubix. This is a service. We put in place last year How it works is teams connect to the Rubix service on another team and It simulates this Rubix puzzle cube where you rotate the cube and try and get all the colors on the same side But it's a virtual version of that teams submit instructions to rotate the cube into a valid configuration and Once the Rubix cube is happy it executes the instructions as if they were CPU instructions Which is kind of clever So we're gonna follow along with a write-up from a team called lab rats They publish this write-up and we can follow along So the first steps to solve this the computers we used for DEFCON CTF finals last year was one We made up and it used 9-bit bytes instead of 8-bit bytes to make it miserable for teams So they first had to write software that could deal with 9-bit traffic, which nobody has normally Then they had to figure out which what parts of these binaries were part of the standard C library and What parts of these binaries were which functions in the C library? After that they can figure out how the main gets called and what the part of the program that they care about actually is Once they've done all that work, which took several hours they could actually start analyzing what the service does So what are they analyzing? How is it supposed to work? What is the benign traffic? supposed to do Once they figure that out, how can they attack it? How can they make the service release its secrets and After they figure that out they can try and figure out how to defend it How can we prevent the service from revealing its secrets to other teams? So all this is in support of the game goals you get points by capturing flags You lose points by having flags captured and you lose lots of points by failing availability checks It's extremely complicated. It's extremely frustrating, but it's a lot of fun for both organizers and most importantly It's fun for players But I should go back every every kind of capture the flag game is extremely ambitious Jeopardy style. They're extremely ambitious to run. They're difficult to play on purpose attack defense also difficult So some of our goals for capture the flag games are making them run smoothly Making them a fair contest and having fun challenges If the game doesn't run smoothly players are going to get frustrated and find something else to do If it's not fair players aren't going to put in a legit effort You know if the same team is always going to win because they're friends with the organizers why play and if the challenges aren't fun Then the challenges aren't fun So let's talk about running the game smoothly and just like any big ambitious project running a capture the flag game project starts early and The most important part of it is who you're running it with who is on the organizing team So legitimate business syndicate is about half of us are friends from university that competed together then and about half were people that worked together in 2012 So here's my friend gyno kind of the team captain and I 12 years ago We were participating in a college level contest and we just solved a difficult problem and we were very very happy about it and also a little bit weird from being in a windowless room all day So how did we start to run Defcon CTF? We found out during closing ceremonies at Defcon 2012 that the previous organizers DD tech were stepping down in December gyno realized that hey I Should make a team that runs CTF because that's my dream So he started talking to us and I remember at a chicken wing restaurant The day after Christmas is what he told me about this and I was in So in February 2013 We all got together in the same room wrote down a proposal for how we wanted the game to work and Found out in March that the proposal got accepted and then the real work began So what is our group made out of? So about three quarters of the group are people that I would normally call reverse engineers But people you know people aren't just their skills and all these people have different specialties So we have a radio specialist who did a lot of work on our 2014 challenge Badger We had several people who were good just with computer hardware in general, which was real useful for 2015 the year we brought I think 40 different computers to Las Vegas and In 2017 the game ran on the work of our esoteric computing expert for the clemency system But besides reverse engineers My friend Salir is just a genius when it comes to computer infrastructure of any kind Networking he he can type commands into routers faster than I could just mash my fingers on a keyboard. I don't understand it I worked on the database backed web application because that's Something I really enjoy But the most important part is that people grow and change I'm not the same person. I was five years ago. None of us are So whenever somebody, you know grows into a new skill set or gets excited about something Don't say you're the web app person. Just deal with that Like if they're excited about something, that's a big benefit for everybody And that means that players role or team members roles also grow and change over time So if you are putting together a capture the flag team either for playing or for organizing Think about who you know and who you know that you trust and who do you like who do you want to hang out with a lot? So once you figure out who's on the team you need to be able to talk to your team and that's what communication is for We after the first year we ended up using Slack or the the chat program Slack for asynchronous chat basically all the time It's great because you don't have to you know talk to somebody immediately. They'll just see the message later We did eventually settle on having meetings every week which is really really useful to you know Have these little iterations where you commit to something and get it done But whenever we have an actual contest to run we like to be all in the same place So this is the rental house we had for our 2017 qualifiers and this is right before the game starts where we're figuring out what order challenges get unlocked in Once we get to Vegas we find it really nice to have a shared hotel room where we set up all the noisy computers and stash all the snacks and stash all the drinks including lots of water because it's important to stay hydrated and And you know that's kind of our home base when we're in Vegas and The snacks and the drinks are really really important because your team is a team made out of people and people you know People require food and snacks So in 2014 We were having computer problems one of the teams Instances their actual physical hardware was acting really flaky And it needed to get unplugged and plug back in a lot like more than any other team should and It was causing a delay of the game because Processes would be trying to talk to that machine and it was just not on so I was on stage trying to figure out why the scoreboard was going slow and Salir was walking back and forth between his computer on stage to read logs and the rack of hardware off the stage To unplug and plug it back in the power cycle the machine And I was kind of like getting impatient with how things were going and I just kind of asked him how much longer is this going to take and Whenever he walked off stage. He didn't go to the hardware. He just kind of left the room and Salir is normally like the most stable level-headed friendliest person and that's not something that ever happens and that really really scared the rest of us running this game and Eventually after about half an hour He came back with a drink. He came back with a sandwich and he solved the problem basically instantly and What I'd kind of neglected was to think about what he needed You know both in emotional terms and also it was lunchtime and that's why we were both kind of crabby so Supporting the CTF players Means supporting the organizing team and making sure everybody's well fed The final point I need to talk about with smooth operation is that capture the flag software is still software and Just like any kind of software or any other computer system The way to make this software work reliably is really screwed up By the end we could have another machine spun up in about five minutes And we usually had hot spares ready to go anyways, which was great so next up I'm going to talk about running a fair contest and a fair contest is really challenging because Because capture the flag is a game about computer hacking But Def Con CTF is also a computer system Which means that we have to be really really paranoid and we have to worry about teams trying to hack the right service the wrong way We also want to make sure that teams don't hack the wrong thing. We want them to hack a vulnerable service We don't want them to attack the scoreboard and for attack defense games we have another problem where teams may want to fix broken software the wrong way and the way we eventually You know solve this is by restricting what players can do more So for qualifiers we tend to run services on separate hosts and we eventually got to running multiple hosts in multiple locations so for The different challenges we have in qualifiers each one would get at least one machine per data center So teams were in the US would connect to a data center in North Virginia or something teams in Europe would go to Ireland Asia Pacific wherever Amazon's Asia Pacific hub is Which was great teams got lower pain times and we didn't have to worry about the entire service getting screwed up globally Next Each connection to a vulnerable service would get a separate container Using the Run C container runtime and XIN at D to actually manage the connections Using XIN at D also has a really nice benefit where instead of writing each service having to have its own like network handling code We could write all these services. It's just standard in and out executables and Then finally in 2015 what we started doing was limiting what system calls each service would be able to make using the set comp tools and Linux Finals is a much more complex game with with much more complex problems And one of the problems that we had was we kept trying to keep the game about reverse engineering and not a game about operating system administration Starting in 2013 what we did was each team whenever they connected to their machine Instead of getting root or super user access They'd get an unprivileged team account that all it could really do is Overwrite the service binaries that then ran under their own accounts So we had a problem starting That year and it got really bad in 2014 and it was a phenomena We called the Superman defense after the comic book character who just Like one of his special powers is oh, he just doesn't get hurt ever So some of the Superman defenses would be like a firewall rule that says this traffic coming from the scoring system Is okay let it through but this traffic coming from an opponent team is not okay So just block it and we never let teams have firewalls that way However a Superman defense that was very successful was preventing the flag being read from disk So teams would run the service in an emulator and the emulator wouldn't have flag access and that was that and we tried to solve that with Making legitimate pollers read the flag, but it didn't always fit into a service nicely So what we learned Was that there were the US defense advanced research projects agency or DARPA started this project called cyber grand challenge in 2014 and One of their goals was to be or their one goal was to build a capture the flag game for autonomous computers with extremely formal rules So for their thing instead of having you know, just services on a disc that would run They came up with new jargon for it called challenge binaries or CB's The CB's were all 32-bit x86 binaries But they used a special executable format that had limited system calls and couldn't store state So what this meant was that teams couldn't just break out of the jail or they couldn't wrap it in emulator They had to solve these things the quote right way Similar to that instead of just launching exploits over a network CGC gave us this idea of a proof of vulnerability Which was it the same kind of executable, but it had a few extra functions that could do it could declare that it was going to Demonstrate that it could control registers of a target program or leak memory from the target program and the scoring system Instead of the teams managed how they ran So these were put in place to support this goal of offline evaluation Where teams connect to a website and they get the binaries the challenge binaries that other teams are using And whenever they go to patch it they upload it to the team interface website again Whenever they have an attack they write a proof of vulnerability and submit it So the scoring system then runs the availability checks and proofs of vulnerability and isolation and The value there is that we can run those Offline or we can run them later in a way that's designed to be reproducible and auditable So once we learned about that in finals we could bring some of that technology to our game Starting in 2015. We would restrict what system calls Vulnerable services could make and in 2016 the day after the cyber grand challenge We ran our game using their same format with challenge binaries with proofs of vulnerability And then finally for our last year We ran the entire game in a limited emulator that didn't have a lot of hooks into the messy real world of Linux Another part of our fair contest is releasing the scoring information after the game zone for qualifiers we release the You know scoring information and a little bit of that for finals we release Just here's the database dump from Postgres. Here's all the binaries teams uploaded here's packet captures to the entire game And that meant that a lot of teams could spend time after the game Analyzing what was being used against them Third parties could audit the scores and figure out how the actual scoring system in the game worked An important part of running a fair contest is thinking about accessibility and Accessibility can mean a lot of things and all these things are important Our first year we had a challenge called diehard which required sensitive timing this was a problem because The name of the challenge was supposed to be a clue based on a popular movie in the US About what players had to do and the timing was difficult if you were on a You know internet connection that wasn't super close to where the game was being run And what we had to do in future years well that year we you know had a really bad solution We told teams to rent space near our servers and play from there and write a program Which is like I don't want to be in the middle of a CTF game renting new servers Long term what we had to do was we had to rethink how a lot of our challenges were being built We had to remove this idea that You know teams should be familiar familiar in you know United States online You know hacker culture from our game We had to eliminate the whole idea of precise timing for our game, and I think our game improved for that Some of the other accessibility issues are just the languages we use online in our official communication because not everybody's a you know fluent speaker the same language as you are and A lot of people are going to use machine translation because it works But you have to you know use shorter sentences use smaller words And it's difficult to get in that frame of mind, and I don't know if we did a great job, but I We thought about it at least so the last aspect of capture the flag and what I've learned running it is this idea of fun challenges and The fun challenges that we spent a lot of time on That we think were the most successful we went and broke expectations One of the ones from 2015 was a challenge called DOS fun for you and there's one write-up for this online and the write-up starts with Discover this challenge is a DOS binary the disk operating system from the 80s and They discovered that the Ida Pro disassembly tool was Doing it wrong so they ended up debugging their disassembler and patching it before they could actually start the reverse engineering and Some teams found out that they had to use the emulator We told them to use because if they use something else if they use DOS box It would get some of the subtle timing wrong and they'd have a bad time another fun one was our Badger challenge from 2015 finals and what it was was a service running on MSP for 30 Microcontroller on physical hardware that communicated with other teams with the game network using a custom CDMA radio network So this is it right here We had two version with I think three different versions of this badge. We have the one for competing teams We had the one for organizers and VIPs that showed game scores And we had the base station which was hung up on a string over the game area and had a bunch of wires hanging out of it So one thing that we learned from cyber grand challenge was this idea of consensus evaluation So with most with a traditional attack defense game whenever you patch a binary on your machine Team your opponents don't get to see how the binary has been patched. They just noticed that their attacks don't work anymore and What CGC did was they introduced this idea called consensus evaluation where whenever you patch or you replace a binary with a fixed one Everybody else gets to see that which means that if your patch isn't complete Other teams, you know can still attack it and it also means that if your patch is really good other teams will start using it But the important part of that is that now instead of us releasing eight services over the entire weekend and only getting eight binaries Every time somebody patches it. There's a new binary that needs reversing So what I wanted to do with these quals challenges was I wanted to push teams into this idea that automated program analysis is Important you can't just you know open up IDA Pro solve a binary and then you know you're done You need to start thinking about programs not just as software that runs, but as input for other programs So I worked on this thousand cuts category in 2015-16 and on a category called crack me 2000 in 2017 that each of these challenges had hundreds of binaries that required automated analysis and I feel they were successful. I saw a lot of blog posts on a lot of write-ups about how to solve these that Mentioned they'd never done automated program analysis before consensus evaluation and finals gave us some really funny moments So a player from a team that I'm going to refrain from naming walked up to the organizers table and asked why they were losing points You know their service was being attacked We looked at the score where it's like yeah, your service is being attacked. That's why you're losing points It's called capture the flag but they claimed they were using the same binaries as the winning team and that it couldn't be that and we just kind of like looked at them and Then they just kind of did this and walked away Because what they realized was that the winning team had put a secret code in there for a backdoor in their binary To allow them to attack anybody that didn't that just use their binary without thinking about it That was really funny, you know, we still laugh about just the expression on that guy's face all the time So the rubic service from 2017 a Few minutes ago I mentioned how it worked you upload the instructions to rotate a Rubik's Cube And then if you've done it successfully those instructions get evaluated as just CPU by code so how this worked in practice was teams would start to write code to block the evil Whenever shell code would come in the scoring system would submit some that didn't steal the flag But attacking players would submit shell code that did try and steal the flag So defenders would start using the extra space in this binary and they patch it by saying this shell code let it through This shell code block it this code let it through this code block it So what that meant was whenever a team would patch Rubik's Somebody that was already successfully attacking them would disassemble the binary see their previous attack Reflected in it and have to build a new attack which felt like The comments we got after the game were that this felt like a multiplayer game against other people We were having I was having a back-and-forth with somebody on this other team. They patch it I'd write a new exploit. They patch it. I'd write a new exploit and we also had to update the scoring system to you know Create new good shell code, but it was a lot of fun for the players and it just fascinates me So we've talked about running smoothly. We've talked about running a fair contest and we've talked about fun challenges This is not everything that we know about capture the flag and this is not everything that there is to know about capture the flag There's still a lot more to learn and there's a lot more work ahead of us building new capture the flag games That create new opportunities for new players CTF is not a huge community yet How many people in here have never played a capture the flag game? Okay, that's actually Really impressive, but I think in a lot of the world. There's still a lot more work that we can do to Make capture the flag for new players The best way to learn how to do capture the flag is to do it either play a capture the flag game Or once you're happy with that Build your own capture the flag game for your friends and for people that you like to play capture the flag has given me five years with the best group of people I've ever worked with and It's given me an amazing five years building a contest for the CTF community around the world the friendliest and smartest community I know Thank you for making it amazing And if you're curious about Defcon capture the flag Order the overflow the new Defcon CTF organizers They're starting their game tomorrow morning at 8 a.m. Beijing time. You can learn more about it at OO overflow.io