 So it's now 10 o'clock. Welcome to the Packet Hacking Village Talks. Thank you so much for starting your day and your morning in DEF CON with us here. So if this is where I see a lot of you folks here, this is your first time here at DEF CON. And so you may be asking, what do we do here at the Packet Hacking Village? What's our mission? Our mission here at the Packet Hacking Village is on security education. It's to inform people and to be aware on a variety of broad range of items in security. And on a personal note, for me, when I first started, as I said earlier, when I first started coming to DEF CON in 2006, the Wall Sheep and the, well, which is now the Packet Hacking Village gave me all the foundation on networking and security that I never got out of college. So that's our mission here is on security education. And for the whole duration at DEF CON, we'll have a number of events and learning out our activities at the other end of the hall. But without much ado, to kick it off our talks today, we have a fantastic talk that epitomizes the spirit of what we do in security education. And when we were actually doing the CFP, we looked at the talks and I was like, this talk got to start off, start off the Packet Hacking Village talks. And without much ado, I want to give the warmest and warmest welcome to Tom Kopchak and Dan Borges of National CP TC. So Ming, thank you so much for the opportunity and for the warm welcome. It's a privilege and an honor to be here and have the opportunity to speak to everyone. So thank you so much. And we have a lot of exciting stuff to share with you in this talk about the crazy things that we do that consume all of our lives when we're not working and a lot of our lives when we are working. So, yeah, absolutely. Without that, let's get started. All right, guys, so we have a lot to cover today. First, we're going to start with what is the National Collegiate Pen Testing Competition, CPTC, and really what makes it different from a traditional CTF. Traditional CTFs are very technically focused and you're going to see we are technically focused, but there's so much more to this competition. And then we're going to get into some of the technical details. How did we put this all together and build this amazing competition year after year? Yeah, and then we're going to go into some of the technical detail about how we hack together this competition, what we do to build the environment and make it interesting for everyone who's competing. And then we'll talk a little bit about the data that we collect and what we contribute to the research community at a whole. And we're going to end the whole thing with a call to arms. We're really trying to get a lot of people involved. This is a product of the community. Kind of like Ming said, we're trying to further education and we really want to get that feedback from the community. So what is CPTC and why is it different than a traditional CTF? Just from a mile high, a traditional CTF focuses on technical skills. There's usually an objective and a very clear challenge and you have to do some kind of hack to get through and deliver that flag or whatever. In CPTC, it's much more open-ended. We have tons of vulnerabilities and it's how you express those and communicate those to the client that makes it different. Yeah, so how many of you have done a CTF before? Okay, that's good. I know that I see some people have competed in CPTC before, so welcome. The thing that we really try to make this is different than a CTF and we're going to go into a lot of detail as the things that we do that make this more relevant. But the formula that we have on this slide is really something that I like that captures the spirit of CPTC. We take offensive security. We use pentesting as the vehicle to teach security. We build a custom environment with custom tools and custom products and even commercial products as well to replicate a business. Add in the business element, not just the technical but the non-technical aspects and then you get CPTC. So that's our formula and it's worked really well for teaching the skills that we're hoping to do. So just a little bit about us, like why are we up here talking about this? For CPTC, I'm a director of the OSINT and the World. You'll see what that means later, but we try to make this as holistic and realistic as possible. In real life, I'm a senior internal red teamer, so I have experience doing this and delivering this kind of information to clients. And then for me within CPTC, I run the white team, which is basically the rules and making the competition exist sort of group. I work with a couple other members of the advisory board as the core group to build the competition and direct how it's going to operate. And then I also run the monitoring team where we collect data for research purposes. And in real life, I direct a team of splunk implementation engineers at Hurricane Labs in Cleveland, Ohio. And the common theme that I really like between Dan and me is we both have the opportunity to work with clients in our real job. We have to be able to explain things that are technical to non-technical people and also really the things that we do in real life apply directly to what we do in the competition. Now it's not just us. There's two of us here and a much larger village that makes up the entire competition. So this is a picture of some of our advisory board that works together to make this competition happen each year. And as we've said before, there are a lot of crazies that build this together and dedicate a ton of time to make it happening. And then on the competitor side, we have tons of schools that play and every year we get more. So there's 50-plus schools that are registered right now for the 2019 season and we're actually expanding internationally. So we have some schools in Dubai registered as well. The regionals are massive. I think last year we had to host upwards to like 30 concurrent environments. So each school that plays gets their own environment. Yeah, and the other thing I'll say is as of right now, we're wait-listing pretty much every region except New England. So if you know anyone who can get to University of New Haven who wants to compete, there's still some spots available, but otherwise the wait-list is pretty much the only option. And we're looking at expanding that and handling it more better next year. But to get to the meat of the talk, at this point you're probably wondering a little bit more detail about the competition. So let's get welcomed in by the Stanford team that won last year that are pictured here on the slide. So why does CPTC exist, Dan? Go. So CPTC was actually the brainchild of Polstack Pol and Bob Kafka. And this was about what five years ago they thought this up and there was just this need in the industry to address offensive security as well as teach like more of a holistic approach to security. Yeah, and the big thing that we wanted to do is find a way to take industry experience and turn it into a relevant event that everyone can learn from. So the advisory board has been the core group in turning this into something that really reflects reality and what we deal with professionally. And then we also wanted to make this competition something that wasn't just small. We wanted it to be something that could expand. And as we've grown, including this year, we've grown internationally. So it started a couple people who showed up in Rochester in the middle of October, which is a beautiful place to be. And it's expanded to five concurrent regionals in the U.S., one in Dubai, and then an international competition of the top teams. So I know I asked how many of you have done a CTF and that was pretty much the whole group. So CPTC is different. It's not a CTF. In fact, it's a competition named only to the point where teams compete, but the goal that we want everyone to win from is to learn something and to be better prepared in the industry. So we have awards that we give out that are specifically for highlighting things that teams do well or relevant skills that we want to teach. So they don't necessarily know they're competing for that, but we like to highlight things that teams do. Additionally, when you're interacting in the competition environment, we don't want you to talk about it being a competition. In fact, if a team says something to one of us, like in the competition, we're like, what do you think? This is some kind of game? We'll talk about how in-character and out-of-character behavior happens. But our emphasis is when you're working with a client, you're not going to be referring to that as a competition because why are they competing to try to break into your domain controller? That's not appropriate. That's not how you handle it in business. Yeah, like Tom says, the things we score aren't direct, like flags or hard results in a traditional CTF. What you do is you find the technical vulnerabilities and it's how you present those to the management team that really makes the difference, which we don't normally see in CTFs. Yeah, exactly. And CPTC really should be a professional consulting engagement where you are hired by our client to perform a pentest, deliver a report to your management, and also work with them to get the problems fixed. So you're doing a security assessment for a real-ish company that we create and the important thing is it's not your smash and grab, try to score as many points as you can sort of scenario. If you find every single vulnerability, spoiler alert, you won't. But if you do, you won't win necessarily because you still have to communicate that clearly. And it's really the good analogy that we have for this is you're performing surgery on the network. You are a doctor, you're working with a patient. You can't kill the patient. You have to try to fix the problems with the patient and make sure they still live to tell another tale the next day. So again, how are we different than a normal CTF? Our machines don't have single vulnerabilities. We put multiple vulnerabilities in each machine, and then the machines are all interconnected. So there's not one attack path that lets you own the network. There's multiple attack paths. And what really makes CPTC different is showing how the vulnerabilities in these systems not only can be used to get domain admin, for example, but can be used to leverage against the goals of the business. So you'll see we have very goal-oriented competitions. Like the business will be a voting company or a self-driving car company. And how can you impact that business model, not just take over all the machines? Yeah, exactly. And the vulnerabilities that we put in here are not necessarily always planned. And I would say we spend less time these days actually trying to come up with a tax scenario as opposed to just having realistic problems built in. So we may have weak passwords that exist. We may have configuration issues. There may be configurations issues that allow you to get access to a system that allow you to find more information that you can use somewhere else. So we build in some of that, but there is just a ton of vulnerabilities. So we guarantee that all the teams will not... Even not one specific team will be able to find everything because we won't be able to find everything. And teams need to understand the significance of the environment and what each system is doing during the event. We don't tell them what the significance is. You walk into a company, you have to figure out what's important. You know what the business does, but you don't necessarily know why each system is there. So you have to figure that out as you're doing your assessment. And then when you ultimately present to your executives and give that reasoning as to why this is important, why you should care, you do have to consider the risk to the company. And that's something that CTF doesn't often capture. And the other thing is teams have to consider that they are operating in a business environment that's doing something and is producing whatever that company is doing. So like when we did the election theme, there were elections that were running. You couldn't take down the servers that were collecting votes because then there's not going to be votes that are collected. Same thing for the autonomous vehicles. When you cause a car to crash, that's really bad for the company. So you don't want to do that. So teams have to consider their impact to the business. And that's something that you don't have to do in most CTFs. But as Dan, you can definitely say when you're doing internal red teaming, you can't break the company you're testing. It matters so much. You don't understand this stuff until you get into the real world. But the relationships between these people at these companies matters more than the vulnerabilities. That said, we do have technical vulnerabilities. We have a ton of them. The competition really is split 50-50. Half of it is technical. If you compete, you'll probably want to take a picture of this because these are like real. And you'll see, you see the host column? We have multiple ways into each system. And then these systems are all interconnected. So you might get creds from one system and then move laterally to another. And that's the goal, right? Show us how you exploit the environment. Yeah, exactly. And I see some of you are taking pictures. The CIA did get a hold of this slide ahead of time. So some of that's a little bit redacted. But it gives you a general idea of how we break this down where we have hosts that are having various vulnerabilities and services that are configured incorrectly. And a lot of this we actually build as teams are producing their reports. So we have an idea of what's messed up and what we planned out. But teams will find things that we don't necessarily plan on being there. The environment will evaluate it and make sure it's actually a real vulnerability and then add it to the scoring system just because we don't expect something to be found. Not a problem with that. So the other thing that we really find important to teach is soft skills. And as we've said, technical is not everything for this competition. The team deliverables are as important as the findings themselves. And that's the way we design the scoring absolutely by design. So the reports, we score them based on how the findings are presented, how you relate that risk to the business, and how someone who's a manager can identify this is important, these are the things that I need to present to my team, as well as how well you explain and help the technical people fix those problems. Because that's what you want in a good pentest report. And then we also do presentations. So at the national event, and even actually at the regional events now, we give a presentation to an executive team. And they need to talk about how it's, the things that they find in the environment are relevant to that team. And we think about that from the perspective of you're walking into someone that had hired you to perform a security assessment, and how do you approach that? If you do something and say like, boy, your IT team really sucks, your customer is not going to like that. And those are sorts of things that we take into account when you're giving that presentation. And I love the fact that we bring in industry professionals to do that. I was just going to note, we get a bunch of volunteers, and then those create the executive board for the presentations. So it's not even us creating the presentations. It's people from the industry that, you know, get these presentations delivered to them often. And then we look to them for feedback too. And the other thing that we added, and we'll talk about this on the next slide, but we actually have the coaches participate as well in part of the scoring. And that's one of the mechanisms to ensure that there's fairness for that, and you know, they're not skewing scores in favor of their team. But we wanted, when coach came to us and said, hey, we want to get, learn more about what teams are doing so we can teach this better in the classroom. And I'm like, absolutely let's make this work. And an hour later it was part of the competition. So like we said, education, that's our goal. I would say actually only one team can win. We understand that, but every team can learn. And that's absolutely our goal. We're here to the packet hacking village, where we all are here to learn something. And CPTC absolutely tries to do that. Yeah. And you know, we've been talking about business all this time, but we absolutely try to teach technical skills through this competition. Even the volunteers, like I learned something every year I do this. We build new tooling, and we always push ourselves and we push students. Sometimes we even have like tears of the environment. So we'll have like easy network and then it gets harder and harder. So I think there's really something that we're going to learn here, both on the soft side and the technical side. Yeah, absolutely. And really the fundamental thing, as I've been saying, is you're learning how to be a consultant and work with clients. And nothing else that we've been able to find provides that experience to people who are in a collegiate learning environment. So I've had, even walking up here on the elevator, someone who competed in CPTC stopped us and said it was the best experience that we had learning how to do this and prepared me for the real world. So I just love hearing that feedback because it means we struck a chord and we're doing something right. And we just want to keep on making this better. So to kind of move on to the next section, let's chat a little bit about what we do to build this environment, how we hack it together, and what we do to make the environment realistic and also pretty complicated. So I kind of mentioned this before, but the business goals drive a lot of the infrastructure. So in 2015, we were just kind of getting started. We had a corporate network. We had a production network and like some several web apps. 2016, we really started getting thematic. So we did healthcare and their high availability is the point of the systems. We had to maintain patient confidentiality and high availability of the systems. From there, we moved on to elections. Elections was actually pretty funny because we made a mistake and I think every team got domain admin within like the first few hours, which yeah, like that sucks for us, but that's also kind of realistic. Like sometimes that happens. Pentes team comes in and they just roll the environment in the first few hours. So the reason for that is we screwed up our build and didn't clear Bash history. But that's something that you would find in an environment and although we didn't plan on it, it was something that happened. That said, the teams that took that and did a lot with domain admin were the ones that were very successful. The teams that said, hey, we got domain admin, yay, look at us, they could have done a lot more. Yeah, it was such an interesting year because now every team had domain admin, right? The playing field was leveled and all of a sudden it became well, how can you impact the business? What can you do with all that access you have? And elections, that was very difficult because we had all kinds of cryptographic verification. So they had access to all the systems, but now they had to start analyzing crypto systems. And then last year, we did autonomous vehicles, which is the wheels company. And that was just this crazy set of microservices where you had these small little services shooting network data all around to each other. And the students had to figure out that kind of microservice architecture. Yeah, so now we'll get to this year. So we've been spending an insane amount of time coming up with this. In fact, we had a meeting over dinner the night of the 2018 national conclusion, where we started planning this out, debated it internally for quite some time. Almost every flight that me and some of the other advisory board members have been on, we've been writing up drafts because we can't find time to do this any other time. But we've been spending a ton of time working on this and trying to build something that's relevant and coming up with ideas that are appropriate years in advance, not necessarily years in advance, but months and months in advance. I think what we've chosen, especially given recent news, is very appropriate. Now I'd say we had nothing to do with the recent news, so don't blame us for that. But here we go. DinoBank will be our 2019 theme. In fact, we have a logo. But we'll give you a little bit of information about DinoBank, but we think this is something that's going to be incredibly relevant and expose students to an aspect that quite frankly, security needs to be improved for a lot of industries, as we've seen with some companies and some issues recently. So... So if you've ever done pen testing, you may have done some kind of PCI consulting or some kind of financial industry pen testing. And that's the goal here, is we want to get students to this experience that dominates so much of the field. So DinoBank is going for their memorandum of understanding. They are trying to become an official bank and they need a pen test to do this and then they need to remediate their vulnerabilities and basically have a clean bill of health. That's what we are... We want to bring the students in to do that financial pen test for this bank. And we've seen incredible growth in this event. So we added an international region in Dubai in 2019 and we're looking to continue expanding that to larger opportunities and involving more schools. And we want ultimately everyone to have an opportunity to compete in this and for interested. So we're going to find a way to make that happen. That all said, we're actually already working on 2020 and planning that out and we're partnering with a major company who's dedicating people to help produce it. So this is going beyond just the crazies of us who take our experience to actual companies that are dedicating resources and full-time people to help us build the environment. So that's all I can say right now but we're planning on making it the most awesome and interactive environment yet for 2020 as well. So we're looking forward to come and it'll be exciting. So we kind of talked earlier about like technically what goes into these environments. Alex and I talked about a tool last year at B-Sides LV called LaForge. Basically these environments are so complex that we had to generate custom tooling. LaForge is a really interesting tool and you can go back to that talk but essentially what it does is it allows us to design one competition environment and then we can scale that up and then here you see not only do we have rich networks with interconnected hosts that transfer data between each other but we have apps that rely on each other and that's kind of the voting components there and you see a little bit of a microservice architecture where these apps are shooting data back and forth. And then not only do we have fake not fake but infrastructure we have people so we generate hundreds and hundreds of fake employees some of them we role play as during the competition but the majority of them are just going to be like data or employees of this company that make it look and feel real. One of the things I really liked about this 2017 voting app stack is just the layers of complexity that exist in there. Teams thought that they could modify votes by just looking at where they were initially recorded and yes that was basically a database server that was very unprotected but what they didn't necessarily dig into is some of the cryptographic signing that we did. Now it was a terrible, terrible cryptographic signing mechanism based on like RGB colors or something ridiculous but that was something that I think one team actually managed to figure that out. And that was awesome to see them dig into that and really see what we did so a lot of this stuff is something you can figure out but we want to try to make it something you just scratch the surface there's a lot more to it. And that really helps us differentiate the teams in terms of how good they are at sorting out complex problems that we throw at them. But let's just say you wanted to build a CPTC environment. To help you out we created the simple diagram to explain what goes into one of our environments. But this is actually from the Forge. The complexity here is unreal. This is a graph of all the objects at the end of a Forge build. Really at the top is where we design the competition and then the tools scale this out and this is just like at the end of the day we have hundreds of thousands of objects in AWS and they're all related and we couldn't build it without tooling. So here's an object model that the tool generated when we were done to show you the complexity of these environments. Yeah so if you're a human you might find this a little bit more understandable. This is a diagram from our 2017 event and you can see there's multiple networks. We have a mix of Linux and Windows hosts. They're all doing different sorts of services and they interact in different ways. And some of these networks you have to get through from openings and other hosts. So like you may only have access to the networks on the left side initially until you get a foothold in one and are able to get into the networks on the right side. Like some of my favorite techniques is a pentest or a lateral movement. So you may find a vulnerability on one system and you can't get into another system but if you could reuse creds and laterally move then all of a sudden you have white box access to that system and you may find vulnerabilities in source code or something. Yeah and then separately one of the things that we do besides the technical side is the world and Dan has done some amazing work on building this and making it something that is really a signature element of the competition I think. At that time making the company exist we have social media that we create for the company's employees just like you would if they were a real company. And then we see the environment with all types of information that can be used to help the teams out. Yeah this is my baby. The world building team started as an OSINT team where basically three months before the competition we wanted to build out all these personas and then put them online and let students in this company and start to build this idea of what they were going to be attacking and we just took that to the extreme we made these people so realistic that then we started role playing them in the competition we started adding like storylines like I think last year we had an insider threat scenario so and the pentesters discover this as they go through things they find logs like hey this isn't right there's malware on this box and then they start to put together this like story that we're building and they're like oh I was planting malware in your environment so it's really cool stuff and the other thing I'd add is we found in a lot of CTFs you might have like a wiki server that just runs media wiki without any content in there it's like so you get into wiki server but oh well our wikis have content that you can use to understand how systems operate they might have details about configurations that you won't find anywhere else and you can use that to piece together on another attack path or get some more information about context for the systems so we have a lot of things that help give teams some ideas about what to do and then they can run with that and discover some more interesting vulnerabilities or issues so like we were saying we build these stories on the left here you can see one of our OSIN stories where we set up an entire fake online forum like hacks forums or something and then we had guys basically leaking PII and intellectual property to this forum that we created kind of talking smack on the company thinking they were the brains and you know the company's no good and they could do it themselves so you start to build this story before you ever come into the competition that like hey you guys have data leaking on the internet you know something something's not right here and then on the right you can see that they do leak their API and points they do start leaking documentation and one of the other goals here is we're giving the students something to research before they get into the competition so the idea is they're not coming in blindsided they kind of know some of the technology that will be in place they kind of have a good idea how to hack it before they get there there's actually really cool I had the opportunity to stop by where stanford does their team meetings and they had on the wall like printouts of our stuff that we put out there and like lines between it was awesome to see that and I'm like yep that's right that's right you missed that one and just kind of piecing that all together so that was a good time also we sometimes produce applications with custom APIs and sometimes actually remember to document it and sometimes if we remember to document we find these weird endpoints that we put in our APIs like something that'll just arbitrarily run whatever command you want which is a nice feature but you know it's not unlike normal IOT firmware where things are just horribly broken so yeah backdoors exist and we will create them into our app sometimes and teams get to discover them and then like we were saying before these apps are all richly interconnected so here you have a database host but the database actually supports the chat for another application so you know the students have two ways to get to the information they can either log into the chat server directly access messages or they can hack the database and get to the messages that way and like in the case of the wheels environment we had databases that were full of car data that you were able to just find and like that is personal information that shouldn't be something that someone on your network can just get to likewise our emails actually contain emails and those emails have information and stories about the company that you can get some more information from and then chat I love the stuff that you do with this too yeah in in chat like again as you get access to credentials and users you will find more and more of the story it will start to unfold and then we also start to like create internal politics so you know certain users are resistant to the pen test they don't really want it to happen other users are kind of fighting for the pen test they're championing the project so that kind of matters as a pen tester who you're talking to in the company right like who has your back who really wants you there and we think that stuff matters yeah so next you want to talk a little about the role playing that we do so it's not just teams show up and they do a pen test and don't have anyone to interact with Dan me other members of the advisory board all play roles in this environment and students are expected to interact with us as if we were industry professionals or maybe not industry professionals but customers and that is something that you don't get exposure to outside of this competition but it's modeled based on what we experience in real life at our jobs so we interact with students and character throughout the event so we'll walk into students rooms like we're the director of the security or if we're I don't even remember what you've been the past couple years but director of incident response and they've triggered some alert and I come in screaming is this you or you know is this us yeah so you know what I think let's do a run through of what this looks like so you're gonna be director of whatever you want to be I'm gonna be a pen tester okay okay I'm packing away I don't have the ski mask on so hey you guys attack the dot seven system no that wasn't us are you sure it just went off line and we're just trying to make sure that you know it's it's nothing on the network no no no that's us okay okay so that's an example of a bad interaction and we've seen that from teams so let's do an example of another one okay hey what's up Dan we just had the dot seven system go offline oh yeah it's a patient system we were just wondering did you guys touch that you know let me check with him I'm not sure if that happened but is there someone on your team that's tracking that that we can talk to tell us at our normal mail when you figure out and then we can resolve the details of it okay great you know I'm gonna have some of my team work on that to try to figure out what's going on I don't know if that's us I can't say if that's anything that's happened yet but we're gonna investigate and we'll let you know what's going on so let's work together and get this resolved yeah I really appreciate that awesome thank you cool that's an example of a good interaction so we've seen examples of both so and then we also try to have like physical competitions too so in this one it was really cool we brought all the students into a room and we had kiosks and they had to kind of like break out of the kiosks which is always a fun competition and then they could take it to like ad infinitum the kiosks had a machine behind them they could shell that machine get callbacks to the room like stay persistent in the kiosk network and you know some teams didn't even realize they could break out of the kiosk yeah I really love the hands-on element of this and there were teams that you know basically took the kiosk and had them being able to execute commands on that back in their room and some teams that didn't necessarily even realize that there was this operating system there so I like the opportunity to expose students to something that's different and kind of make it a little bit more complex than just VM in the cloud somewhere so bringing in the physical element making you have to travel to on-site engagement have a little bit more experience of what that entails and how you should prepare for it so next I want to delve into a little bit of the competition data that we collect and our methodology for that and some of the research that we're driving behind that so one of the big things we've been able to conclude is that hackers like typing LS in fact 13 some percent of the commands type last year were LS you can also see some people's favorite combinations of LS like LS LA is more popular than LS AL so these are very important contributions that we're making with the data that we collect there you go we have someone who feels very strongly about that so get a team get them all the type it that way and then you'll be able to change the future so joking aside some of the monitoring is some of my favorite stuff like we run like splunk alerts and then also like we collect like LS query data and we'll have like bash history so we have this really rich monitoring in the observation room where we'll show like commands that recent teams are running not just like statistically prevalent stuff because that isn't the interesting stuff but yeah we'll do like what is the best end map command who's generating the coolest payload like all kinds of weird stuff yeah and we we actually do a ton of monitoring in the environment and it's for two purposes competition integrity and also for research so after the competition ends researchers are pulling over or pouring over the data set to get more information about how systems are attacked and what it's like to be on the attacking side and the targeting side because we collect both sides so some of the sample tools that we use we have IDS and all the hosts both on the attack side and on the defense side we have smoke universal forwarders everywhere forwarding to a central environment collecting data we use OS query sysmon monitor log files you name it we'll collect it and we try to make this entertaining so we will provide real times graphs and charts kind of like that well she does with credentials but we want to see what's going on in the environment so we use our tools to do that one of the interesting things that we found actually just analyzing IDS alerts and how many are generated by each teams the winning teams at the bottom were actually the ones that generated the fewest amounts of IDS alerts so that that speaks a lot to us where you're not just scanning and generating a lot of noise in the network you're identifying something and then you're actually digging into it yeah exactly and this is a lesson that I've seen in real world pentesting um flowing doesn't really help you in pentesting kind of wildly scanning, definitely information gathering and doing research but then you really are executing like explicit exploits very targeted specific things so throwing around tons of scans isn't going to really help you explore the environment other than recon so the data set is public anybody can download past data sets and they can do their own research we just hope that you attribute it back to us and then we are also doing research so one of the things that our research team devised is they basically found that the techniques that teams are using to access these box do map to the MITRE attack framework and they're building full kill chains to kind of show this is how a team moved through the environment um so that's ongoing research right now yeah and some of the stuff that I've seen is really awesome where they have actual log data from whatever host like the host where the attack is run so we see an attacker runs this command and then this results in this process being executed on this target system and nothing else really provides the information of what an attacker is doing using log data as well as what the target is receiving uh plans for 2019 is to expand this even more and make it closer to all of our development processes where everything that can produce a log will get logged and grabbed and then I'm hoping to make the data set that's publicly available next year actually have frozen splunk data that you can just put into a splunk instance and run your own searches on as well so that's the long term goal to make that really something that's accessible to anyone who wants to do this sort of research and make CPTC the vehicle where we can do that we've also released our tools so LeForge is open source if you want to check that out and try to build a similar competition and we've talked about releasing LeForge config files so that way teams can stand up their own environments it hasn't been something that we've gotten to yet but every year we really push for that and one of these years we're going to have like a plug and play environment that schools can stand up on their own yeah exactly and the other thing that we've done is we're trying to have teams contribute more to the just the general security community as a whole so one of the new additions in the 2019 rules is actually allowing teams to stage their own tools in a repository that we provide they have to be able to explain how these tools work they have to have them document it and they have to be able to make them something that other teams are able to use and other industry professionals are able to use but we want to remove the veil of secrecy for a lot of this we want teams to be able to make contributions to the security community and UCPTC is the vehicle to do that and if anyone wants to do any work with the data set you're definitely encouraged to do so the only thing we ask is that you at least attribute it to us as where you got the data so we're just going to cover some quick success stories this is really the point of the competition is the students and it's the people coming out of it and we've had so many amazing people come through this program so we're just going to kind of rapid fire through these and highlight some of this and again the whole point of this is for education we're trying to get more people into the field these are people that are already passionate about it and we're just trying to guide them to you know be more successful when they leave school yeah actually going just going back to that we saw a need and talking to people in industry we saw that there are opportunities that people need to have to learn the skills that employers demand and a lot of students come out of college very strong on the technical skills but they don't have an opportunity to learn the business skills especially in a lot of the technically focus programs and just having a bunch of students walk up to me and say CPTC was the best thing that I did in college that's just awesome and we hear that quite a bit and everything that we're trying to do is supporting education and make things better and make students better at what they do so we've had students land jobs through CPTC and quite a number of them have said just this alone was what I was able to say is that the big experience that got me that and then Stanford's team actually reported a zero day in a real application and they were working on getting that resolved but it's the great example of something that we definitely didn't plan and being in the environment that we were able to reproduce once they told us what they found and that's another good example of good reporting where looking at the report we were able to understand what the issue was and validated in our environment so to wrap up call for action yeah this is really only possible because of volunteers like we saw we're passionate about this stuff and we saw the need and we know there needs to be more people in the industry so we commit our time to trying to teach and make this better and really we need more people like it's Tom and I and a handful of other people and we could really use volunteers we have so much work and it's such a good cause so if anything here struck a chord with you please reach out to us and you know there's just tons of work to do yeah absolutely so call for action be an influence in helping the next wave of cyber security professionals help support us if you think that you're able to do it start a team encourage students that you know to become involved or compete we're also looking for people to help us run this year's event not only from a helping build the environment to also helping with some of the scoring that we need to do and the grading and all that because it's a pretty big undertaking and as you've seen there's a place for everybody at all levels right we have non-technical positions we have technical positions so really anybody can help and that's what it's all about is collecting that industry experience and trying to put that into the competition yep absolutely so I'll leave this last slide up with our contact info we have a form on the website for sponsors if you're interested in not sponsors but for volunteers you know sponsors too if you want but you can reach out to any of us additionally we're going to be speaking with a bigger panel of the advisory board at ethics village at four o'clock on Saturday so tomorrow and we're going to dive into some of the ethical issues that we've run into while teaching pentesting and have a wider panel discussion with our group as well as having more audience participation so if you like what you see and want to learn more stop by there if not we'll open the floor for questions I also have CPTC stickers up here if you want to grab any of those but thank you all so the question was for those of you didn't hear there's a lot of custom information and artifacts of data that exists in the environment how is it created a good amount of it is hand generated right now we set up the systems in advance and then we kind of enter the data and then we scale that across all the teams but we do clone like a bunch of pages or documentation if we're trying to build documentation very fast and then things we're getting into this year on the world in osan team is we're starting to get into like deepfakes and creating images of people putting those like totally like normally we go and we steal images or whatever but totally unique creative images and then putting those like in places with other people and then same if we could ever do that with docs and generate real ish enough docs that are relevant because what I don't want to do is just have tons of red herrings right I want it to be relevant but great question great question thank you and the other thing we have to sort of kind of deal with is there are personas that we create for us so like I have an identity in the environment that's not my real identity but it still is a version of Tom for example and Dan has the same sort of thing so we have to create a separate persona that we can use in the environment and play that in character but we don't want you to also be like confused by our real job so to speak so the question there was how do students land jobs through CPTC so we have a lot of industry sponsors that help us either through providing expertise volunteers or dollars and they have opportunities to interact with the students as part of the competition so they some of them will have the opportunity to interact as a role player in the environment and they actually get to see the students operating professionally or watching presentations but we also have career fair type things where teams have a couple hours to meet with the sponsors and have conversations like that it's not like it's not a competition in the sense that that's why we throw it we throw it for education but man the winning teams like the second they get off that like podium and in the announcement the sponsors come up to them offering them jobs it's pretty insane so the question there was have we thought about deploying a version of this as a honeypot so it's a great idea now we have that's a really good idea and especially with the data that we're collecting I think that could be really awesome it's very expensive to run all the infrastructure you have like our daily burn rate so we have had to secure so our cloud provider that we use is very much dependent on what cloud provider gives us enough money so we switched between Amazon and Google and other just depending on what's available but I think actually Lucas you could probably tell us an answer on this but it's probably like five to ten thousand dollars a day for a region so obviously that's if it's sponsor money it's monopoly money but we still gotta be responsible about that so but we're also looking at the ability to take this and maybe you throw it on your own ESX server and use LaForge to build it so you don't have that cloud cost yeah yeah it would be if anyone you know cloud providers that can give us a lot more dollars you know we can make that happen so the question there was when should teams start OSINT and when should they stop so OSINT hasn't started yet this year it will probably start around mid August I usually start after DEF CON and basically you start stop probably the week before the competition I like to have everything leaked the week before but you know if we run late it could be up to and there will always be some kind of technical finding so there will always be something you can use in the environment and it may even be a case that you don't necessarily know what a thing is called in the environment like a custom tool but you'll see something like that in a server you'll see a web page for something and then you can potentially Google that and get documentation for it or there might be another path to get that through social media as well so the question for that was have we had any success stories scaling the data set down to a high school or a non-profit level and I would say a lot of the research has been more generic right now in terms of like business miter framework type stuff that's a really interesting point the thing I would say is probably a little bit more challenging is it's not necessarily designed to show a certain thing where like some of the other exercises there's a clear path you know something exists in the log file a lot of times our researchers are taking a team's report knowing they discovered something and then trying to find that based on a much wider data set so it's probably more sophisticated than a lot of that but I think once we identify examples and then maybe even write it up in a way like you can download this and find this sort of thing that'll make it a little bit more accessible but it's a lot of data right now so I think last year the raw data collected was like 350 gigabytes of data in like 8 hours of competing time so it's quite a bit and we're probably going to be seeing more than that this coming year and then even analyzing that you have to stand up huge Splunk instances yeah so we do use Splunk for analyzing the data that makes it pretty easy but we're trying to make that more accessible in where if you only want to look at a certain like windows logs for example we want to make it so you can just download that throw it into Splunk let it churn away and then do something with it so I think that approach is going to make it easier for like what you're saying high school level, college level because if you're a college professor and you want to look at teach students how to work with windows logs when an attack is going on you can download the CPTC windows log index index that into Splunk run searches on that do reporting all that kind of stuff so that's kind of my long term vision for what we want to do this coming year but I think that will probably be something that could help for what you're looking to achieve but yeah hit us up or grab contact info here and we can chat about maybe some ideas about that too the question on that was what kind of resources do universities provide and also how do they get involved with this so a lot of it's actually been word of mouth initially so this started out with a couple schools in the Greater Rochester area or upstate New York and then as people have talked about this experience it has grown and actually part of the reason we're talking here is to just get the word out there about what we're doing and as more teams learn about that and have the opportunity to compete or at least know this exists they're signing up and we're looking at ways to make sure we handle that demand yeah and in terms of resources we've partnered with numerous universities specifically five in the US and one in Dubai to host our regional events and they provide space for a weekend for a regional event they provide meals and they help teams out not necessarily financially but at least getting them arrangements to show up and stay a weekend and compete they provide us with people who can talk or give presentations in the industry and they also provide us with volunteers to staff the rooms make sure teams know where to go make sure that it all operates smoothly and then on the national CPTC side we provide representatives from our organization to show up they basically run the event for the school so they don't need to provide any of the setting up the environment sort of thing they just need to provide labs and they also we also provide all the infrastructure all the scoring and all of that to make it happen so from a university perspective that's hosting it yes they have a lot of things that they need to do to make it happen but it's not as serious as where they have to build infrastructure and all that and they use their lab space they use their classroom space and they provide food more or less and then the students that are competing they provide the willing and able bodies to stay up all night and write reports so the question was what tools do we provide to deploy this environment on its own so we have a tool on github.genocide with a zero instead of the O and then LaForge and there's a b-side talk about it last year but basically the idea is you can use that to build the environments it takes YAML configuration files and an API key for a crowd provider unfortunately we don't have any of those YAMLs like the example environments because we reuse them so we just haven't put out templates yet but that is the dream we've been talking about it for a few years and then maybe this year we're going to retire a subset of systems and we can publish those and we want to get to the point where we're providing a lot of information about the 2017 environment because that's well past at this point we want to get to the point where we can release older environments and teams can use that for practice or qualifying type environments but right now the question was do we is there a reason we use LaForge versus Terraform and we actually do use Terraform though so LaForge is just a custom Terraform wrapper basically except it's like way complex and fancy and has a bunch of bells and whistles but it's a Terraform wrapper it's custom designed to write Terraform in the way that we need a write Terraform to build this environment oh we screw up things all the time so the question was have we made other mistakes that have affected the competition we are we're human we do make mistakes and kind of think of some notable yeah we actually last year announced the wrong winners yeah they wanted me to score so the way our scoring system was set up is you have teams one through whatever in each region one through ten and for whatever reason there was a dependency between the way it was presented on one page in the scoring system versus how it was on the main page and if you like updated the URL as opposed to going to web UI you put the score in the wrong section and there were teams that the score was good enough that it didn't look like they had a zero for part of the score when we sanity checked it and then we realized when we started putting in the later time zone scores that they had scores already and something really bad happened so we actually before we figured out exactly what we were going to do about it we decided and this is actually going to play you should go to ethics village we're going to talk about this stuff too but we decided that we were going to do the right thing and whatever team actually won would be what we announced even if it changed of course the event completely and then we figured out how that affected individual teams so and we there's a couple series of tweets that we have about this where we explain what happened and how we fixed it but our emphasis is we are transparent about what we do if we make mistakes we admit it and we don't want to hide anything about what we do so that we do stuff like that happens and that's what you deal with professionally and we think that it's not making the mistake that's the thing it's how you handle it so that's very important for what we do so thank you so much Ming we appreciate it and thank you for all and stickers