 So, as I said, my name is Robert Wood. I'm going to be talking to you guys about threat modeling in general as an assessment methodology and how it applies specifically to the gaming industry, mostly because gaming is a very unique industry in that the business risks are oftentimes very, very different than financial services or healthcare or ISVs. So that was the primary driver behind evaluating the gaming industry because in any kind of security assessment, we need to make sure that we can map risk back to business risk or vulnerabilities back to business risk. And being able to accurately and correctly do that is going to be what helps us as security professionals change our industry and actually improve things instead of just saying, you know, you're all wrong. So this is our high level agenda for what we're going to cover today and we'll do questions at the end. So, threat modeling. Who here just by show of hands has ever been a part of, you know, building a threat model within your company or for another company? All right, excellent. So a number of hands went up. So threat modeling is a threat model, not threat modeling, but a threat model is a depiction of a system's attack surface and how a given threat actor will attack a system to compromise an asset or a set of assets. And a threat model identifies, you know, functional and logical components of a system and how those things fit together, trust boundaries around those components, a trust boundary being a system boundary or a network boundary. And what we want to use threat modeling for is a way to identify all of the various, you know, mechanisms that a threat agent can use to target a set of, you know, critical assets. And it's mostly focused on identifying design flaws. And from what we've seen in practice actually doing threat models versus more actionable and interactive assessment things such as source code reviews or pen testing, reverse engineering type assessments, is that design flaws end up constituting about 50% of the overall vulnerabilities of a given system. And so, you know, if we're just doing pen testing, then we're only finding about half of this stuff, which is more focused on implementation-level, you know, bugs, like, you know, like a buffer overflow, like cross-site scripting, like SQL injection. So obviously, you know, we want to make sure that we have adequate coverage across an entire system. So this is some of the, some of the verbs that we're, or some of the syntax that we're going to use throughout this conversation and, or throughout this talk. And I'll go through and describe what some of these things are throughout and right now. So an asset is anything that's sensitive that we need to protect. So that could be either sensitive information like credit card information, like source code, like healthcare records. It can also be a set of sensitive functionality that we need to protect. So consider Amazon's AWS. They have internal APIs that they can use to do EC2 instance memory snapshots, for instance. That's a very guarded API that you want to make sure that, they want to make sure that, you know, somebody on the internet doesn't have access to. So asset, something we need to protect. A control is something that is in place to protect a set or a single asset. So input validation could be considered a control. TLS on network connections is considered another control. A threat. So there's a lot of different definitions out there for what constitutes a threat. In this context, we're going to refer to a threat as an agent who uses an attack factor to compromise an asset, you know, through some component. Okay. So throughout our system, if you consider a much bigger system, as opposed to just an attacker, this little simplified diagram here, an attacker might go through multiple components and bypass multiple controls to get to an asset, but this is the general workflow that we'll work within. So as we all know, but it's worth throwing out there, is that all threats are different and we need to be able to quantify our threats so we know what we're defending against, who we're defending against. So some ways in which we can actually quantify the threats that were... Something funky is happening. Let me try. Oh, maybe it's not me. Okay. So some of the ways in which we can quantify threats are by the various capabilities that they have. So if you look at, you know, stepping back into financial services land or, you know, defense contractors, for instance, or government contractors, you know, they may be concerned about threats who have, you know, a great degree of physical security skills or physical security exploitation skills. And so they'd have to worry about, you know, corporate offices and data centers and stuff being broken into the threat actor that's much more technical and can sit on the other side of the world and attack the same set of assets in a different way. The skill level associated with those capabilities, so you have everything from the script kitties who are just running point-and-click tools to attackers who are able to, you know, do their own fuzzing, do their own exploit research, build their own exploits, and actually deliver on those in a, you know, undetectable or near-undetectable manner. The resources and tooling associated with that, with a given threat actor. So in some cases, considered like organized crime or well-funded nation-state attackers, they may not have the capabilities or the skills in-house, but they have money that they can throw at the problem just like any other business. And, you know, there's a lot of good people that work in the security industry, but there's a lot of bad people in security as a whole, too. You know, much in the same way that a business can throw money at a defensive problem, a threat actor who is well-funded can throw money and, you know, hire people to work on their behalf to go ahead and exploit things. And, of course, the level of access to various components within your system is another big consideration. So if you think about an external attacker who's out on the internet just, you know, doing their thing, interacting with public-facing websites, whatever the case may be, versus an insider threat who may have, you know, domain credentials, they may have a company-issued laptop, they may have, you know, access to internal databases, things like that. They can sit on an internal network, they have badges, all of this stuff that they can use to possibly compromise different sets of data or compromise it in a different way. You know, we need to make sure that as we're designing our systems and placing controls throughout that system to protect our assets, that we consider threats, different types of threats, and these are just some of the ways in which we can do that. So this slide, it's a little bit of an eyesore, and I apologize for that, but this slide covers the threat modeling process at a high level. And so a lot of it is starting with understanding the system structure. So we need to understand what the components that make up our system are, network protocols that fit together or piece those controls, or those components together. Sorry, I'm stumbling. I haven't had enough coffee today, obviously. And so once we understand that system structure, we'll overlay on that system structure assets, controls, identify where threat actors or threat agents live within that system. So do they live within a trusted network zone, i.e. your internal corporate network, or are they out on the internet, or are they a malicious client slash user, things of that effect. And once we understand where everything is, where everything fits together, we'll start to enumerate doomsday scenarios, the business ending events, and canonical attack factors based on the technologies that you're using. So consider a standard web application, a standard banking web application. It's going to have a backend database that has a front-end user-facing application that people are interacting with. It has probably some backend services. It might have another internet-facing service that a mobile app interacts with. And so based on those design patterns, we know that a certain set of attacks can just automatically map to those design patterns. And we can just kind of include those, or we get those automatically. And then we just kind of go and iterate through this second set, and keep looking for different ways in which a threat actor can go through a component or set of components to get to an asset. And so that's the process at a high level. So like I mentioned at the beginning of this talk, we need to make sure that we as people engaging or building a threat model, or designing a system, consider business risk. And so specific to the gaming industry, these are some of the high-level business risks that we frequently see when interacting with clients. So of course you have your account hijacking. That's one of the biggest. You have cheating, service and availability is a big one. But in other games, where there's money tied to an account or money tied to the virtual economy, there's the possibility for fraud much in the same way that there's fraud in our regular economies. And depending on the type of game, piracy and downloadable content might also be an issue. And we'll talk about a little bit later why this is not standard for everyone and why every business needs to consider their own business risk. So one of the interesting things about games and another reason why I chose this particular industry to give this talk, is that not everything needs to be super secure. You don't need high assurance design and controls for every little thing in gaming. And this again maps back to business risk. So some of your assets are definitely worth investing more security in than others. Others are, you have to kind of prioritize what assets are business ending if they were compromised or would cause an enormous amount of revenue loss versus those that are more of a pain in the ass, if you will, if they were to get compromised. And some assets are subject more to monitoring and controlling in a sense than they are just outright preventing them from being compromised. And so this table tries to convey that in some cases, and this is of course specific to a given business or a given title and set of business risks associated with that title, but we've seen this particular table cover a lot of use cases. So cheating, piracy, botting, things like that often fall into the control or allow set of things or control and allow spectrum. And so games like Titanfall, for instance, they don't really do much to prevent you from cheating, but if they detect you cheating, they'll just place you into death matches and stuff with other cheaters. Versus some games, they'll outright block your account and ban your IP address, your email address, whatever. They will kick you off their servers for good if they catch you cheating. And in some cases, cheating just doesn't really matter in a single-player game. Who cares if you cheat? It's like the games of old on Sega and N64 that did not have internet connectivity. You would just open up your secret menu that wasn't secret at all and just type in God mode or something and you would get unlimited health. And then of course you have your assets on the left that are these are the things we need to make sure that we protect. And this is pretty standard for all businesses. So denial of service is never a good thing. Payment information much in the same way that financial services, that it affects financial services or other retailers, never a good thing if that gets out. And user privacy obviously a big issue. So circling back to the threat modeling process, again we need to make sure that when we're building the threat model that we build it off of a valid and accurate system model. And so you can usually build that system model by working with, talking to developers, architects, designers, business owners, people who understand how the system is built, how it was designed, things of that effect. And I'll actually show a very simplified threat model and system model and it might be the next slide and so you do this by analyzing documentation but you cannot rely on documentation. Anyone who's either created documentation at their company or read documentation at their company probably realizes that you go look at the last updated time stamp and it's like 2009 or something and it's grossly out of date. And as soon as you actually start looking at code or talking to people they're like, oh yeah that should change months, years ago. Nothing is like that anymore. So you want to validate through interviews, code, whatever other artifacts you can get your hands on. And your diagram should ideally visualize the way in which control flows through components throughout the entire system and the protocols or whatever interaction mechanism that components are using. So that way you can see if a user is passing control over to a front end web application that they're doing that over a secure channel. If it's not an insecure channel then you have a potential and there's assets involved then you have a potential vulnerability that needs to be addressed as an example. Okay, so what's the next slide? So this is a very, very simplified system model and so if you look at from the left to the right we have control passing from the left over to the right and up in the top left corner is everything that's residing on our user's environment. So you have their browser that they use to interact with title websites and payment portals, things like that. Then you have the game client itself which does some state management. It's got some configuration files and then of course physics rendering, logic, all that good stuff. And then the big block in the middle is all of your back end services, your account management stuff, your payment management stuff, monitoring, cheat detection engines, all of that fun stuff that lives in the middle and then you've got your customer service tools or their own game clients that they use in some cases to play the game, interact with the game, monitor language on the title or in chat rooms, things of that effect. And this was taken from, well, this was kind of stripped down from a threat model engagement that we did for a large MMO that any of the gamers in this room I'm sure have played. And so trust boundaries, we had that term pop up in our syntax slide at the beginning. A boundary can be defined, oh, there isn't necessarily a set definition for a boundary because it changes in, you know, you can look at things at different layers. But think about boundaries in terms of when one component passes on control to another component, that would be considered a boundary. Or if you have a network zone, which are these kind of dotted zones around groups of components, that's going to be when a logical grouping of components needs to pass through additional control to get into another zone that contains another block of components. So if I'm crossing over networks and to get access to another, like an internal network, I would need to either route through some kind of load balancer or firewall or VPN or something to that effect. That would be a good place to place a trust zone. So some sample assets. These were specific to, and there was many, many more, but to avoid things becoming a complete eyesore, we stripped it down to these six. So these are some sample assets for this particular MMO that we reviewed. And of course we have the game content itself. This was a paid game with some extra downloadable content that you could purchase. You had your account information, pretty standard. Your billing information for ongoing payments and downloadable content purchases. You had your actual player assets. So players invest so much time into some of these games. And research has shown, actually, that a player account on some of these larger MMO games are more valuable than a single or even a set of credit card records that are stolen and sold on the black market. So for instance, a World of Warcraft account, in some cases, goes for up to $500 on the black market, if compromised. So let's say Blizzard was breached and somebody was able to access 10,000 accounts that were not protected with multi-factor authentication and they were to sell access to those accounts on the black market. That would be a much more valuable hack at a much smaller scale in terms of accounts compromised than some of the big banks and such that get compromised where they have all of these risk management and insurance and all this other stuff to protect them, and ultimately the records end up going for much cheaper. So it's just an interesting spin on how to look at the value associated with a given account. And of course, if an account is compromised, all the assets within that account are compromised. Then you have things that facilitate fraud and cheat detection. So they descriptive metadata that services are collecting and monitoring on. If an attacker was able to tamper with that, that could facilitate cheating. And then you have access to, this is more of that functionality piece, access to customer service representative accounts. So in this particular case, they were using the same version of the game client. They just resided inside the internal network and used their own account to log in. It was a domain account. And the customer service account, all that functionality was bundled inside the same game client. So if somebody was reverse engineering that game client and found a way to unlock or trigger that same set of functionality, they could do things that would facilitate compromise of other assets on this list, such as giving myself more assets, more inventory, more experience points, level ups, free downloadable content, or just spam out or mess with other players' accounts. All in all, just nasty stuff relative to this game. And so getting started, we will overlay, as I mentioned before, overlay assets onto our system model where they end up lying. And then we'll kind of keep a running legend at the bottom. And so you kind of go through various discussions with folks who are developing, maintaining, designing this system and figure out what all of that sensitive information is and just continually add to your threat model. You do the same exact thing with controls. So this is, again, a shortened down list of controls relative to this specific game. And you do the exact same thing with your controls. You overlay that onto your system model. And an interesting tip or useful tip, rather, is by overlaying controls and matching them up with the asset that they're supposed to protect, if you get to the end of the exercise, the end of your interviews and artifact analysis, and you have assets on your system that do not have a control paired up with it, then that may be an easy way to identify a potential design flaw. So for example, with the CDN down here on the bottom left, we found that there was no particular authentication or access controls, anything like that associated with the CDN, and they were doing all of their pushes out to patcher applications or installer applications on a user system over clear text HTTPs, and all of those local apps ran as admin on a user system. So there was an easy way to get administrative remote code execution on user systems if you were able to either compromise the CDN or compromise the traffic if you were on the same network as a user. So the particular threat agents. So this was, you know, this particular system had many more threat actors than just this, but some pretty standard canonical threats in the gaming industry is, you know, you have your basic Internet attacker who's just out there attacking externally facing services, maybe doing social engineering attacks, things of that effect. You have your malicious user, which many gaming security talks will exclusively focus on. They'll put all of their eggs in this little basket that's on the top left and not really consider anything beyond that. Obviously a big issue, but it's not everything. As you can see, there's much, much more going on relative to how a game is put together. You have your other attackers on the LAN. So external from the internal network and such, but still on the same local network as another player. So that's where, like the CDN updates over clear text would come into play. And then you have the insider threat. So, you know, anybody on that internal corporate network where the game services are hosted that may come into play, be able to sniff traffic, if you have payment information being passed around at HTTP, you know, poor access controls on player payment databases, things of that effect that might be easy wins for either a malicious insider or a compromised insider, because, you know, both are very real possibilities and likely possibilities. So this is a good place to start when considering threat actors relative to your system. But of course, specific to your gaming genre and your business risks, you may have more or less, or, you know, you may need to tweak these. And we'll talk about that in just a little bit. So, you know, again, overlaying on the system model, putting the threat actors where they belong, and from here is where you would start to, where you would start to use, you know, you could use big red lines, you can do each one individually, you know, whatever floats your boat really, but you would draw lines from a threat actor to an asset. And depending on what controls you need to go through, what components you need to interact with, if you find that it's an easy way, that it's easy for a threat to access to an asset, either by circumventing a poorly designed or implemented control, or by not having to go through a control, then that represents and constitutes a design flaw. And something where you should consider putting in defense and depth controls to protect that asset. And so you can iterate through this, you know, to your heart's content, putting in as many attack vectors, brainstorming as many attack vectors as possible to identify all the ways in which your assets are, either are under attack or could be attacked. And again, you could use this same approach on any type of system. It does not have to be a gaming system. So places that it makes sense to do a threat model. So in the beginning stage of a systems development, this is more focused on, you know, generating abuse and use cases that drive software development and system design. On the latter case, when you actually have a fully functional system because, you know, code has been built and things have been implemented and installed on servers, and you can actually go interact with things, that's when, you know, you're going to end up with the more complete model that we saw a couple slides ago. So as I alluded to, different game genres are going to have, the threat model associated with different game genres is going to be very different. You're going to have different threats, you're going to have different assets, and the controls protecting those assets are going to be different. And this is where the business risk comes into play because different revenue models associated with the game are going to drastically impact the threat model. And so, I am a huge fan of Batman as my arm will indicate, but who here is familiar with the, either of these games, the Arkham games or servers, games. Yes. So, for anyone not familiar, the Batman Arkham games is you play as Batman, it's a single player game, there's some leaderboard stuff, but that's not a huge component of it. You play as Batman, you run around Gotham, beat people's asses, save the city and be awesome. DC Universe Online is a fully fledged MMO game where, you know, you pick your own superhero that you want to be and in this whole world interact with other players, very very World of Warcraft-like typical MMO. And so, in the Batman Arkham games, because it's self-contained and you pay, you know, presumably $50 for a console version of the game, excuse me, or, you know, you buy it on Steam, whatever, actually getting access to the game is a big issue in terms of getting access to it for free. Protecting the game's content with, you know, DRM, whatever the case may be, is much more paramount than something that is freely downloadable such as DC Universe Online. So that's one big example. Also, cheating is not an attack vector that you would be concerned about in the Arkham games, because everything is self-contained in that little single player instance, who cares if you can cheat? Who cares if, you know, all that really does is degrade the player experience with regards to, you know, being awesome in the Batman storyline and, you know, working through and, you know, beating all the villains that you have to beat. Whereas cheating in the DC Universe game is much, much more apparent and other players interacting with that world are going to see, you know, that this player over here is cheating, they're getting free inventory, free level ups, whatever the case may be, or they're able to just build a bot easily and, you know, have their player churning through missions or quests or whatever on a 24-7 basis and, you know, they're leveling up a lot faster than anyone and it provides an unfair advantage and so that can decrease the player buy-in and ultimately the revenue associated with the game. Downloadable content is another big one. So that's another way that the Arkham games end up making money after your initial sale as they'll sell you on downloadable contents and new missions, new storylines, things like that. And so they have to worry about how users actually download that content, how it gets how they make authorization decisions to ensure that if that content resides in your system that your player ID is allowed to access it, things of that effect, whereas in the DC Universe game it's not really as big of a deal. I think there's some DLC, but it's not really as big of a deal as it is in the Arkham games. So another thing to consider is that gaming platforms that your game title is going to reside on, and of course this again extends to anything that you could theoretically threat model is going to drastically impact what your threat model looks like. So if you consider things that are in a user's control, so, you know, PCs, mobile devices, web browsers, things of that effect, that's all in the user's control, they can pick it apart, they own their machine, they can do whatever they want to it. There's the common bit of security knowledge that if an attacker has physical control over a system or something that they will be able to own that thing, given enough time and skill and all of that, versus things that are cloud-hosted. So NVIDIA has rolled out this cloud-hosted gaming console service called RID, and basically you interact with a gaming PC through a web browser, and so you don't get access to the underlying PC itself. And so they've attempted to remove the risk associated with, you know, an attacker having access to, you know, the underlying game client and all of that by forcing you to route all of your traffic through a web browser and interact with it that way. And so, you know, going that route would drastically change how you're going to end up protecting yourself against all that stuff on the left, the top left, so, you know, the reverse engineer attacks, the network tampering attacks, things of that effect. So, again, we can, of course, model all the things in the world, so and then, you know, once you build a threat model on a very granular piece of a much larger system you can fit those things together. So I've been involved in threat models on gaming consoles, on much, much larger MMO games, on mobile games, on mobile devices, all of these crazy things and you do enough of these things and you see enough of the results, you know, security vulnerabilities and just security intelligence post it out in the industry, you can start to piece all of these things together. So you know that if a particular system is using design pattern ABC on technology XYZ, then you're going to end up with, it's a deterministic way to identify potential security issues. But the thing to keep in mind in this is, regardless of how many things you threat model on a much more granular focused basis or, you know, assess whatever your assessment methodology is, threat modeling or otherwise, you still need to step back and consider your system and how your system uses all of these things because there will be some contextual stuff that comes into play. So this is just building, again, on the reusable components throughout game design. So, you know, if you're using, you know, this kind of ties back to the vulnerable libraries discussion, if you're using frameworks or libraries that are inherently insecure, so if you're building a Java application on the old version of Spring Security, then, you know, we know, without a doubt, that you are storing your passwords insecurely because Spring Security will use, I think it's, I think the old version of Spring Security uses just basic salted hashes, versus the new Spring Security, Spring Security 4, that uses Bcrypt by default. So, you know, unless you're doing something custom as a software engineer, which is probably not the case when you're pulling in a framework or using a library, then, you know, we can default to and have easy wins for vulnerability identification. So, one of the biggest challenges that the gaming industry faces, and this is one of the ones that most folks end up focusing on, is running trusted software, supposed to be trusted software, on a busted platform, so i.e. things that the attacker controls, their mobile device, their console, their PC, whatever the case may be. And one of the biggest challenges has been how do we make sure that we run trusted code in an untrusted environment, because ultimately we have to get stuff back into our system. We need to take input from an untrusted source and do something with it. And so, you know, for whatever reason the gaming industry, in some cases does, you know, they do things very well with regards to security, in some cases they don't. There's still an over-reliance on placing trust on the client, because for whatever reason we think that you know, it's difficult to reverse engineer software. As the last talk showed us, there's so much tooling out there there's a lot of people interested and engaged in this stuff that it's really not all that difficult to reverse engineer software in today's day and age. And so, you know, much lower skilled attackers can do more advanced things. Of course, you know, higher skilled attackers can do much cooler things and bypass, you know, white box crypto implementations like ArcSan and Metaphoric things of that effect, but the barrier of entry into these kind of attacks is getting lower based on tooling and research that people are doing. And so, you have to ask yourself as a system designer, what kind of risk do our users present to our game? And what kind of trust do we want to put into those users? And if the answer is not a lot, because it's an untrusted source, then we need to make sure that we have back end controls to protect against to protect against the crap that those clients are potentially sending us. So, you know, if they're hooking network traffic or intercepting network traffic and tampering with it, then, you know, we need to be prepared on our back end web services or whatever we have that's protecting or that's accepting that input. We need to be prepared for that to be malicious and potentially bad input. So, you know, all of the general secure system engineering stuff comes into play here. So, some higher level program considerations. So, obviously, defense in depth is clutch. If we, you know, circle back to again the example threat model diagram, you notice that placing all of your eggs in the in the game client trust basket. You know, let's say implementing something like white box crypto on your game client, that's only one small area of your overall attack surface that you need to consider. You need to consider your network protocols that go not only from the game client to the back end, but also network protocols and such that are passing network traffic around your internal network. So, defense needs to be everywhere. You can conceivably put defense. And you need to understand how data flows throughout a system and where you can protect that data in certain places throughout your system. Integrating security touch points into an SDLC. Regardless of what your SDLC looks like is clutch. And the earlier that you can end up doing that the better shape you're going to likely end up being in. So, if you're considering things right at the onset of a new title or a new software project and you say, you know, here are all of our use cases. We know exactly how we intend users to interact with our system and we can compliment those use cases with abuse slash misuse cases. So, all of the ways that a nefarious user could potentially use our system and we want to understand how our systems could break based on that set of use cases or anti-use cases rather. Using threat modeling as a means to drive other security activities such as static analysis, pentesting, whatever is a great, great thing to do. So, because threat modeling takes into consideration the overall system as a whole, as opposed to a pentest which is much more focused in many cases or a code review which looks at, you know, a set of libraries or the use of a framework in certain places or just code, not network protocols. We can use threat models to identify clusters of high value assets or clusters of potential vulnerabilities and we can say this area really needs some TLC. You know, we really need to invest more dollars, more activity, more attention over here using this particular assessment methodology or just defensive dollars to throw more technology at it, throw more engineering hours at it to help fix things. We want to understand the overall system as a whole and then zero end is a great, great way to approach things. And then consider again the value of the asset you're trying to protect and what the assets are going to be relative to your particular game. Because, again, that's going to change for every single title, every single system. So if you're protecting a, if you're protecting a banking application versus a, you know, a healthcare application versus a game, those three things are going to look very, very different even if they had a similar type of system design or use similar network protocols, things of that effect. And then the last thing that I want to mention here is that not everything needs to be 100% secure. So nothing is ever going to be 100% secure or else we all wouldn't be at this conference because everything would be perfect. And we know that's not the case, hence we're at the conference, trying to learn more about this stuff to make our industry more secure. So, you know, as security professionals working in this industry we don't want to, you know, it's going to work against us if we run into a business owner's office screaming as if the world is on fire at our organization and claiming that we need to make everything 100% secure from 100% of attackers because that's not the case. In all likelihood, unless you are running one hell of a game a nation state or a spy agency or some kind of really well funded, sophisticated attacker that's going to be attacking you know, the Lockheed Martins of the world is not going to attack World of Warcraft. It just doesn't add up. So, you know, you need to make sure that you understand the threat actors relative to your business, to your title, to your revenue model and then secure your assets, your critical assets against those threat actors. And everything needs to be relative. And I want to make sure that I left some time for questions at the end or that we can escape early for beer o'clock. So, any questions?