 This is track five. Securing MMOs. So full disclosure, you've probably read in your programs that I'm a bioarmythic employee that is no longer true. I've been advised to say only that in order to give a more complete version of this presentation, I've altered my employment status. So I've worked in the security industry for quite a while. And hey, I'm actually working awesome. For quite a few years, and have gotten some pretty useful experience, took that, parlayed that into a job application. And don't even start. A job application at Bioarmythic brought some unique skills to the table to a game industry type company that really was focused more on math and dying of fire. You know what? So a lot of these people right here are, I'll say acquaintances because they are not friends. But yeah. So I apologize as I interrupt my talk to flip people off back and that sort of thing. So I have been spending a couple years writing subversive softwares, the term I like to use. It's not malware. You don't have it on your computer. I'm not reading your email except for yours. I've written and I should say I've helped write static analysis tools at a product company, that sort of thing. So when I was at Bioar, I was a senior software engineer to give you scope on what a senior is there since that word gets overused is there were 10 to 15 of us at a 150 person company, about half of which was engineering when I started and a slightly smaller company when I departed. I've written some other tools for other games, Eve, Warcraft, that sort of thing. And most notably have been here for four years running capture the flag before DDTAC took over. So a lot of these jackasses up front helped out in that in one way or another. I don't know this guy but I'm sure he's okay. He's not. Okay. So what's in this talk? Some thoughts on what it's like to go from a job that most of you are probably pretty familiar with and what it's actually like in the game industry. Obviously I left so I couldn't have been that awesome. Stories on how bad security really is in pretty much every other field but especially in the games industry where it's exposed and a lot of people actually interact. You get a little bit more insight. Some examples of some hacks and other tools that people have written, including the ones I've written and some other pretty ridiculously terrible ones that we've encountered at Bioware. And then some thoughts on how it can get better and why there's no chance in hell they're actually going to do anything I say. Besides the fact that I'm a giant asshole. What's not in this talk? I did not bring any O-Day for any of the games that, enough. I did not bring any O-Day for any of the games currently being published. I do not have a release date for Star Wars, the old republic, don't ask. Nobody does. I didn't include any crappy clip art, just this one thing on the right here and there are no shout outs in this presentation. I think that's overdone. So I mean we can go and get started and talk about actually making the jump. Getting into the games industry worked out best because, like any other job, I knew a guy. Alternate slide title, how I learned to love the shorts. It was actually very disorienting to show up to an interview in a suit and be greeted by the CTO wearing flip flops and shorts. It's a very, very different mindset, that's for sure. Even when they were dealing with customers. In the security industry, you look especially bad when you write code that crashes or has vulnerabilities in it. That's not really an issue as anybody who's ever run any sort of desktop PC game and it constantly crashes underneath them. They're not suffering because their game crashed when you do the specific key sequence or whatever. So when we're constantly CY-ing and ensuring quality, I've moved into a world at this point where crashes are a matter of course. Things go really, really terrible. Somebody might actually say sorry. I think there was a time at which Blizzard would give free time for extended absences and that just isn't the case anymore. There's no recourse as a customer. And then from a world of kilobytes to gigabytes. When you're writing shell code, when you're writing small clandestine applications, you're worried about every byte, every kilobyte. And this quote down here was a realization that I had. No matter how much code you write, you could write for years, your binary will still be smaller than the intro video that took like three months to render. So it's very different considerations and a very different coding style results. A little bit more lackadaisical one. And another neat little statistic is after getting stood up the first day, it's like okay, let's build the project and go get lunch. Because it took about an hour to build. And the last link step. So every time you just made one little change, that's okay. It did incremental compilation. You still had to wait five minutes for it to link. So it's really a big jump and very disorienting for me. So what was expected out of some of the applications that were being written at Bioware and MMOs? 4,000 users per shard. So a shard is what you think of as like a realm or a world in Warcraft or something like that. And it was expected that the server would introduce no more than 100 milliseconds of latency. We could also give a whole talk on the network infrastructure behind a game like this, which is just unparalleled. It's fantastic. They could actually start there on ISP in their colos. It's crazy. So each user is spamming dozens of commands. They're moving around. They're issuing combat commands, that sort of thing. And all of these have to be very responsive because nobody actually wants to lose. And if they do lose, well, I suspect many people do want to blame it on the client. But we don't want them to continue to blame it on the client in a public forum, right? And 4 hertz. So that means every 250 milliseconds, the whole set of commands are batched and processed. So what this means is that any two people doing something within the same quarter second actually are effectively doing it simultaneously. And low speed. This isn't talking about the server. This is talking about the people. You've all, I'm sure if you're interested in video games at all, have heard of the crunch period where, you know, six months a year, sometimes 18 months of people working around the clock. It's my theory that happens because they're slacking off the rest of the time. There's a lot of YouTube in video game companies. There's a lot of Hulu now. And there's also a much greater tendency for formal education, especially CS and hard math. Whereas in our field, you're able to kind of get away with it because there's no such thing or there wasn't such a thing as a degree program for own age 10 years ago. So what makes security difficult in an MMO? Imagine, if you will, an Apache web server. We can assume that they implemented it based on RFP that was reviewed by dozens of people battling back and forth, defining how things are spelled and what tokens terminate and that sort of thing. And it takes months or years for revision of a formal protocol like HTTP to be revised and then finally accepted and then even longer for it to be implemented. Many of these users, you can actually have some sort of compensating control of authentication in front of it, VPN, that sort of thing. Who cares if there's a giant pre-auth vulnerability if nobody can even get to the system to exploit it? Insider attack, yeah, yeah. Some are open source and benefit from constant peer review. People are testing new static analysis tools on open source projects, that sort of thing. So there's a lot of attention being paid to security in open source tools. That's not to say open source tools are secure. We can talk about that later if you're interested. So in the games industry, we define us back as we go. Some producer or designer seemingly on random whims, I'm pretty sure they're like throwing darts or tossing dice somewhere. Decide what feature goes in this week, what feature comes out. Oh, we're not going to do that anymore. Never mind, got all the supporting code you did for that. Code ends up staying around. Things change scope, right? Oh, we need to be able to have a hot bar for 10 commands. Oh, also now it needs to be 40 commands and now it needs to be 140 commands. Actually, you know what, can you just make it user modifiable and scriptable? It's not the same. Things are constantly shifting, right? We'll give a connection out to anyone that asks. Anybody with a credit card can show up and connect to our server. That's a big deal. And then as games age and they begin doing free trials, we'll give them out for free. Sure, you can have 100 accounts, whatever. We closely guard our code. There's kind of a treatment of IP as the most precious thing because it really is what the whole company is built on. And so all of these combined make our lives a whole lot more difficult. Each one may not be so terrible if it wasn't for all of the others. So as we evolve as game players and as game developers, we intend to include more. We do everything that was already done up to that point and then a little bit more. And then a little bit more. So each successive online game or even video game just pushes the edges a little bit further or it tries to if it intends to be successful. So the more sophisticated games breed complexity, they breed more sophisticated hacks and more sophisticated hackers. Other people have cut their teeth on earlier tools and that sort of thing. I'm also fond of hackers. I'm glad we share something in common. This is great. I feel like we're connecting. This is good. So client side security is always a losing battle, right? But it's fun. Defense blows but when you start to turn it into a game of offense it can be a lot of fun. I have some pretty cool stories about that. And a market for in-game currency, gold sellers, means that there is incentive for fraud, for cheating, for doing things you shouldn't be able to do, including dooping items, that sort of thing. So why do people cheat? The most common is to win, right? Except that those people tend to be small scale, not terribly destructive, whereas RMT real money transfer, sale of in-game currency. There's real world money at stake here. People pull dirty tricks and try everything they can. And often they're in nation states that perhaps we don't have some sort of extradition treaty with or really don't give a shit about enforcing U.S. cyber crime laws. Other people prefer to cheat for griefing a captive audience. People love their video game and cry extra sweet tears when they've been griefed in that particular video game. Some people like to get an edge, like I said, they like to win. And then some people like myself, I'm sure some recreational type guys in here, you own it because it's fun. You're done playing the game, now you're going to beat it in a different way. So you're going to write automation tools, you're going to write cheats or whatever. Yeah, that doesn't tend to bother developers because they're mostly doing things for curiosity. It's only when you start selling or widely distributing a tool that it really starts to show up on their radar. Make no mistake though they actually know when you are developing one. Security in the game industry is extremely new. Just like I'm sure a lot of you have growing pains, the financial industry was the first one to jump on board years ago and now games are getting into it as they realize that it actually can affect their bottom line. It's not just an annoyance of cheaters. And sadly, everyone's heard the term buffer overflow, everyone's heard integer overrun. But few people actually know how to prevent it in practice. Oh, if I know what vulnerable code is, then certainly I can prevent it from being written, right? We don't write vulnerabilities, we're professionals. I write this for a living, I've been doing this for 15 years and to which I reply, yeah, I found the same bug 15 years ago. Also whoever's attacking me right now, that's dirty bullshit because I can't hit B. Dirk tangent, you know who you are. So knowledge of vulnerabilities isn't the same as being able to spot them in your own code and especially not in the code of others. Nobody wants to hear that they write code less secure than IE, right? Everyone trashes the browser and oh my God, that thing's a pile of shit. I can't believe it crashed when I went to Disney.com. Why are we going to Disney.com? That's not important, it crashed. And nobody wants to hear that if somebody was doing something equally malicious to your application that it would fall over. The one thing they have going for them is that blind exploitation is virtually impossible. So they really have to worry about some sort of insider. But taking things down is very trivial. And the one thing I will say for the games industry, at least they don't say cyber without meaning chat sex. That is bullshit, stop doing it. We can fight this together. Yeah. Down with cyber as a single word. Okay. So what are some of the techniques and types of things that people do to cheat? Blind scripting. This is things like writing a script to emit keystrokes. This is about as basic as it gets. You're just throwing input. Screen scraping and scripting. Auto it. If you're not familiar with auto it, go check it out. It's pretty cool. I've seen some amazing things for Eve that actually would account for the background color of the environment you were in to adjust the OCR to tell what the statistics were. And people have spent thousands of man hours trying to work around it in this way. And it's really the tool of the layman. You can get so much more done more quickly elsewhere. Obviously the most direct. Just reach in and touch memory. Go read state that's already there. Go modify state. That sort of thing. If you've ever had a game genie when you were a kid, same thing. It just locked a piece of memory. You can do the same sort of thing in modern PC games. So all of these are thwarted with a very simple are you there? Either by manually initiated by a game master or with some sort of automated query. Which of course can be thwarted if it's automated. But asking people to do something semi intelligent like basic math or name the color of something. That sort of thing. All of these make it very easy to detect when someone is automating. So the ways in which you tend to see things actually be exploited are actual takedowns in the form of like a classic exploitation of like a buffer of a Florida numeric condition. And more common in actual use is taking advantage of race conditions and state flaws. Specifically any kind of item dupe is just because you've managed to add an item to another character without having it removed from yourself. And then somehow ensuring that your character is not saved in that state where the item has been removed. We actually saw that some people tooling around with the Warhammer protocol were actually understood it better than some of the engineers in house. They were able to craft specific packets to ensure that such and such in zone three and such and such in zone four could transfer an item. And then zone three guy would be replaced with some guy that was actually already logged on in zone five and he actually booted him off. And now you've got two people wielding the you know plus two sort of ownage which then sells for a million gold in the auction house which they then sell to you for 999. So I don't actually know where you can get a million gold for 999 don't ask. So some other things that people like to do packet injection and sniffing. Sniffing way more popular. It's almost impossible to detect unless you're actually looking for someone running some sort of monitoring tool on the same box you can put on another box. Problem solved. So the way you thought that use SSL doesn't actually have to be you know incredibly strong it just makes it so that you have some way of verifying that contents originated with you. What that does is forces the attacker to come into the client where you actually have some program at a control and you can actually tell what's going on how they're developing their tool you can actually watch them as they do it. Also this poor camera guy have you been drinking sir are you just really bored. Oh fuck you too. So an account thefts people often fish spearfish whatever get a key log around your system they'll actually say hey this is a tool that will help you earn a million gold download it run it and then it doesn't work or maybe it does and next thing you know your account is empty and your purpose have been charted. That's just it's actually the most common attack that we saw actually run was key logged so what is Bioware currently field for MMOs? Ultima online. Yeah it's true they didn't write it but they maintain it. There is there is a full time engineering team still writing code for this 12 year old 2D game. No shit. Dark Age Camelot it's gonna be nine years old this month actually. Warhammer online three years old Star Wars old republic never actually gonna release not ever. Yes I know Duke Nukem Forever has been canceled that's what makes this especially delicious it's never coming out. In reality the guys working on this game in down in Austin are doing a kick ass job. They fucked up a lot of things but it's getting better and some of the guys at Bioware in Virginia are pitching in and at which is the office I worked at are pitching in and are helping even do design for the PVP stuff it's actually gonna be pretty good I know I trash talk it a lot I'm sorry. So this has actually happened to multiple game companies. We know this because other people have asked us how we dealt with it. Website got owned either it was like some sort of PHPBB type crap hosting forums or whatever forum software result in some sort of sequel injection was actually then used to escape the box got rooted with an LPE and now the damn thing is hosting viruses you click on a news link and you're owned. Defense in depth actually saved the day on this one while we did manage to infect thousands of customers with viruses. We did also protect all of the critical information including game database, credit card and billing information that sort of thing so they didn't actually really get anything useful until they got you to install the key logger on your system and then they got it from you and not from us. So yeah we actually it actually worked amazingly. So this one is my favorite like a lot of people you know I'm introduced to this monolithic decade old code base and told okay go ahead and implement this feature and so it took a little while to get familiar and like some people in this room I'm sure the way I got familiar with it was just with a source audit. Just read through look at it and I guess it didn't really intended to start that way but that's how it wound up and find a bug here, find a bug there, hey look if someone's playing around with the protocol they're going to be able to take it on the server. I've got a slide for that one too and until I found this one and it was actually you could type a command and crash the server and how this got by QA I do not know. I'd like to think that he was in one of the layoffs hopefully. But how did it get into the code? Some dipshit wrote it. No you have a follow up question I saw you put your hands up. Okay. Yes. So specifically it was just a signed unsigned comparison not checking that A to I you know actually return negative numbers it would use a signed index to index into an array, crash. Fortunately after the discovery we started tracking it down it turned it was only in a GM command so there was no immediate threat but there are there's also historically been pretty terrible control on who has GM accounts. So this one is pretty awesome. Imagine if you will a structure that has a length okay. What the fuck man he's supposed to be a goon like helping me out and he's attacking me up here. Okay. So imagine if you will a structure that contains a four bite length sitting at the front of a message and then the remainder is used to store data. That means the length is specified by the client on the wire. Why do we do that? I don't know. But to change it apparently would break everything ever. So we essentially worked around it ensuring that nothing would be received more than that and checking some length boundaries but why we still let the client specify how much data is coming in in that fashion for a copy I have no idea. And of course desk is a fixed size buffer. It's actually lots of function pointers hanging out right after that so it was pretty easy to actually do a quick POC. And yeah like I said no ode this has been fixed so don't try this at home. Okay. So trial accounts. I mentioned earlier that we give away accounts to people who just ask. They don't even have to ask nicely. They just ask. And the majority of trial accounts and that's not 51%, that's like 85%. The majority of trial accounts belong to spammers and gold sellers. 10% of all accounts of all trial accounts created in a three month period belong to one IP. We assume that this was some sort of cafe because a big surprise you reverse it. Oh look it's in China. Supplies. These accounts then get used to actually do the spamming and advertising for the gold that they're stealing from you with the key loggers that they got to install in your system when you went and clicked on the link in the form software. It's actually pretty sophisticated, pretty cool. Just really unfortunate that we were on the receiving end of it. So we suddenly started getting reports of players' accounts being emptied. And people are saying, well I haven't played in X and Y months. There's no way that they got it from me and such and such. I didn't empty my account. So what we actually saw was that that same IP that had registered 10% of the trial accounts would slowly over time log into a bunch of high level accounts and they would log in once. They'd be there for about a minute or two and then they'd log off. And then six, eight, nine months down the road, they would log on and transfer gold in one million increments to some random string of characters who would then in turn transfer it on to someone else. I don't know why they thought that was going to throw us off the trail when we have a fucking text log of what's going on. But they did. So what we suspect was that things were being harvested through key loggers since we specifically checked how many failed authentication attempts happened against these accounts that were emptied. Zero. By and large zero. Sometimes occasionally you'd find one but it was from the real IP. So we assume there was no brute forcing unless they were managing to do it in some incredibly stealthy way that evaded us knowing when they make a TCP connection. We suspect they were coming in and inventorying high level characters. And then once someone had made a request for gold they would log back on to the character who they already knew how much they had and they would transfer it out in a million at a time. And actually my favorite thing was when you ordered more than a million they would send you one million one, one million two, one million three so that you knew which one was which thing was a batch of how many millions you had received. Just seemed incredibly pragmatic and like they're clearly business minded about it. So again like I said same IP. Not really that amazing. So we suspect it was actually pretty large group. So how bad is it outside of where I actually got an inside peak? Senior producer over at CCP says we don't trust the client. Blizzard trusted their client and look at the mess they're in. And they're talking about wow glider actually having to go and engage in real litigation to deal with a game being hacked I mean really. I guess their paranoia is why I was able to just you know inject Python and make the whole goddamn thing dance. Clearly they're doing something right. So there are a bunch of tools in the wild that things are, things are actually being used. First one is auto eave. Anybody who happens to follow my twitter account? You've probably seen this. I may actually post up another link to it after I make sure it works in the latest version. It's just straight up Python injection. It does actually use vtrace right now but could just as easily use something like debug help. Injects Python code and has a semi sophisticated script for mining and automated travel and such. So my favorite story about this is that someone actually here in the crowd who if you've seen a guy running around with a thing two shirt on he went and interviewed at CCP and they looked at his resume and they said oh you're from Virginia okay you're in the security industry. Have you played around with any like you know hacking tools or game hacking before? And he replied kind of like a cautious yes. And they said do you know Metro? So remember how I said they know when you're working on something. I promise. Now of course they've never done anything to me or retaliated in any way after this talk that may change. Oh well. The tool is especially cool. It uses their own APIs. It's especially brittle because every time some programmer at CCP decides they want to change their API I have to go change and use it and I don't have their docs. CCP like a lot of other gaming companies is hiring security professionals. They have slots for people who are dedicated at like dedicated security personnel including code hardening and architectural approaches. It's changing. So if you're looking to get into games you know get your resume out there there's actually a good chance right now in the hiring blitz that you can get in. And of course like I said came up just even in casual conversation. This is one of the more depressing things. I'd like to think that the hackers out there are you know pretty sharp and pretty brilliant. This guy, this isn't the real name of the tool. I just didn't even want to give this guy the fucking hits from Google as you went and looked for it. You can find it if you're looking for it. It's just a simple scripting tool. And he bothered to exor the strings used in get proc address for read memory, write memory. Because that took me a whole extra 90 seconds in IDA rather than just looking at the import table to just start the damn thing up and watch where he calls read and write process memory from. He of course went for direct reading and writing of memory. It actually was a pretty sophisticated little tool. Did a good job. But if you're going to bother hardening it like at least make it fun for the guy who's going to be looking at it. This is nonsense. So undetectable again not the real tool name. This was someone who touted on his cheat and hack forum that he had come up with a way that he could manipulate the client in such a way that it would be 100% undetectable. There's no way that any of the engineers could ever figure out that you were running his hack. Except that all it did was register as a debugger. For anybody who's not familiar, there's a god damn API call to tell when you're attached to a debugger. So we, having fun, decided that we would let him continue development and let him continue to make this claim and actually watch him as he logged on and in the different and interesting ways he would crash his client. So once he had distributed the latest version of his tool and he said you're absolutely undetectable. I've been running it for weeks and months. They're not catching me at all. We would let him distribute it for a week and then we'd ban everybody that used it. And then this one guy is sitting there, I'm totally undetectable. I don't know how you got caught. No, we know. We just also know you're the tool developer. And so eventually you go through a couple cycles of this which says okay, okay, well I changed something. Don't worry. You're undetectable again. Let him distribute it. Everybody gets banned. People stop going to site and downloading his tool. It was actually pretty effective. So what are some of the mechanisms you're going to have to look out for when you have inevitably decided you want to go home and write a hack for your favorite online game? Memory checksums. If you've ever tried to poke memory in a, I'm sorry, a Warcraft client, you'll know that it won't even let you log on if you've modified it. I promise you the trick of logging on and then modifying it and it seems to be okay, they're watching. But who cares? Timing. When you're in a debugger, anything gets paused or stopped for a, did you bring enough to share, sir? No? Okay. In the future, please. Timing. So looking at the differential between any two API calls, if there's a debugger sitting in the middle, like somebody has enthried in the middle of that, you're going to be able to notice that it took a second and a half instead of the normal millisecond and a half between said API calls. These are pretty easily detectable when you're looking for them. Get tick count is not usually used for a whole lot of other things. Anti-debugging measures. Just crashing or closing in totally random nonsense ways when you detect that there is a debugger. Cheat detection shouldn't stop the cheat. If you want to actually be effective at catching the person that's doing it, you let them do it, you let them see, let them actually develop their tool, watch what they're doing and then figure out a new and interesting way to make their life extremely difficult. My favorite is the industry standard of you never store pointers to strings that appear on the screen directly on your data structures. And the obvious reason for that is when I am looking for a toehold in a data structure that's somewhere in this 700 megabyte heap, I look for things that I see on my screen. Maybe it's my character name, maybe it's the name of an item, that sort of thing. Well, they would actually store pointers to pointers or just pointer pointers and essentially causing it to be incredibly difficult to work backward from the string to the original data structure when you're just in this morass of nonsense. And then actually managing your bottom line as it regards cheats is just effective PR. Tell people what's going on. The players will understand. Well, some of them will, but I mean there's always the troll. And then be quick in your turnaround once you've announced that there's an issue. You need to be agile. The one thing gamers aren't is patient. So, you can take the blizzard took to mitigate things, write your own goddamn root kit. This is actually pretty effective at spotting apps that they know they're looking for. It's the warden if anybody's not familiar with it. Hoagland has actually done a pretty good write up on it a couple of years ago. It's changed since then, but it's, you know, still in the right area. Then there are gameplay workarounds for impossible problems. There is no way that you can write a client where you can update state when someone whips their view around to look behind them in a 180, send them that information on demand, have it arrive in a timely manner, and continue processing, and thus obfuscating things that they should not be able to see on their screen. People write tools that will snarf the data or all of the surrounding characters and objects and displayed on a radar. Well, how do you stop that if you have to send the data? Give everybody radar, level the playing field. Sure, it removes immersion, but it also means you don't have cheaters actually winning all the time. It doesn't become a mandatory thing to do to win at your favorite game. And then psychological warfare. Like I said before, leave the developers alone, ban their users. Their users will abandon them. So, how can the very stressed, already overworked, underscheduled industry improve? Power assist, source analysis. There are open source, there are academically provided, there are commercial tools for source analysis that would catch a whole bevy of really stupid shit that I found manually, and I'm sure I missed other stuff in my hurry to scroll things down on a piece of paper. Basic fuzzing, it's worth it. Internal red teaming, as you develop a security staff, they need stuff to do when they're not responding to incidents, be proactive. Educate the remainder of your developer base. At least get them in the habit of using secure APIs. Stir and copy where you provide a stir lane of the incoming string does not actually do anything for you, and it would be great if way more people understood that. But it says I can't use stir copy. And can I at least get somebody to write up a decent threat model for an online game? Like, no, they just don't even, they can't conceive of all of the things that have been going wrong for other people for the last decade, because they just haven't had to worry about it until very recently. And then balancing code quality and release time. Realize that when there is a large scale exploit, it actually will alienate your user base. And ultimately, your user base are the people that continue to decide to pay you $15 every month for making their monitor look the way they want it to look. Ultimately, it's entertainment dollars and people will spend them elsewhere if they're not having fun anymore. So why want to get better? Mostly because it's actually way cheaper to have CSRs fix the things that result from hacks than to actually close all the vectors. This remains true in other industries, I imagine, but it's definitely true in an industry where people will beg and practically work for free to be CSRs. And these are, you might be, you know, qualified at something else, but choose to work in a game position and so they'll take their $10 an hour pay. The bad news is that even if you beef up your introspection of your own code, you have people dedicated to red teaming and fuzzing, there may not actually be anything there to find. There probably is, but there may not be. And so you have no quantifiable results to pass on to a board that's reviewing your expenses. Why did I give you all this money to review this code? You didn't even find anything. I guess we're just really good at writing code. And then measuring the true cost of lost reputation is actually really difficult. And there's not a whole lot of awareness and there's no good formula for figuring out how much getting owned actually hurts you. And of course, if you have cheaters, that means you can ban them and then say you banned them and everyone rejoices. Because even if you were to tell everyone, hey, guys, there are no cheaters. We've stamped them all out. No one's going to believe you because the guy that just beat them is obviously cheating. So there's also the principle that there's no such thing as bad press. This is especially true. I don't know if anybody in here actually plays Warhammer. But if you do, you probably had your checking account or credit card billed to about zero a couple months ago. This was all over gaming news. And it turned out that this giant snafu with the credit card processor, it wasn't our fault, it was not actually our fault. It was the credit card processor. This giant snafu actually resulted in more subscribers as people heard about it, read it in their favorite gaming news websites. They said, hey, you know, I meant to try out that Warhammer gamer. Man, I should really resub. And our sub numbers went up when in fact we had seriously just billed people thousands of dollars and took four days to get it back to them. So yeah. So last slide, which is good because I was just told I have 10 minutes. I lied. There are shout outs. Yeah, camera guy is really happy that we're done. He wants to go home and drink. I just want to say you guys right here in the front row, the guys that couldn't be here, Conscioto, the guys who helped me run CTF, I should help me. We all ran it as a team. You're awesome. Yes, fuck you too. Nurse Betty, that's the hot blonde up here. She's put up with my shit all weekend. You'll see her on the way out, I promise. DD Tech, you guys have done an awesome job taking over for us. I don't even know if they're in here, if any of them are here, but they've done an awesome job. If you see them, congratulate them. They are finally seeing me. I'm really excited to be able to run CTF and I hope they stick through it. Don't break our record, but go another year or two. And then the goons, you guys have been really, really awesome with me this weekend, especially you guys running around at the last minute trying to find me adapters and getting me set up with all this shit. I'm an idiot. I fail logistics just like every other nerd. That's it for me, though. Does anybody have any questions before we have to give up the room for closing ceremonies? Not a one. Oh, the one right here. Oh, I'm sorry. Right here in the blonde. No, to be honest, I'm... What's that? I'm sorry. Oh, yeah, I'm sorry. The question was whether or not I was aware of RSA dump and whether I'd looked into it at all. Is that some of your questions? Oh, you say dongle? Yeah, well, so that would... Yeah, okay. So just like Blizzard is doing with their real ID program, they're sending out their dongles. And had we looked into them, the problem is the expense. Yeah, so they are getting cheaper, except for when you have a... Unless you're willing to immediately pass that upfront cost onto the customer, or you have some assurance that they're going to continue for a while, it's tough to make that argument with bean counters, because like I said, it's actually to them, it's cheaper. They're paying those CSRs to do their job anyway. And if we need one more, sure, let's just get to that. I agree, I think that's actually a good solution, and I bet customers would be happy for it, but I don't see it actually happening because it's an actual upfront cost, which is the one thing that game companies actually despise. If you can avoid upfront costs, it's the way to go. So I'm sorry, you had one right here? Nope, behind you. I'm sorry? So I was asked if I was affiliated with DFC. Yeah, right here. So I was just asked, all the extra time you spend on anti-cheat stuff, how does that equate to, or how does that compare to the work to make a Linux client? And would an open source kernel change that? So the answer to that is where I was, we did both. We used transgaming to do our ports and essentially what they do is they send direct X and some of the APIs if you're not familiar with it. So to be honest, in terms of actual effort, the port was way easier because transgaming, they essentially get a small cut and they do all the work and we just make sure it isn't broken. So we used RQA resources and their dev time. So anybody that's not making a Linux port just doesn't want to give up a cut to it. So the question was, how much does it affect developer workflow when actually working with these anti-cheat mechanisms? And the answer is in debug mode, none at all. When you actually have symbols emitted, you've got everything macroed out that's done a lot of this stuff. So in actual, like in the case where something only happens in release, it's a giant bitch. Yeah, that sort of stuff and you don't even notice it and it's macros used to access it or they'll be wrapped in C++ wrapper classes, that sort of thing. Anyone else? Oh, right here. So the question was, finally got it this time, the question was how many times do you see people actually attack the protocol versus the client and the answer to that is a lot. People would just seem to be able to catch it and throw it out and we'd note specific IPs of interest and then like I said, there were actually people that actually seemed to know the protocol better than in-house engineers. It was some incredibly sophisticated attacks against the state machine and realizing where characters were saved and how characters were saved and taking advantage of it for item duplication. So as a third, there was just like noise all the time and we've got a room for a couple more questions and that's it. We have four minutes. You know, I have not heard of that at Bioremythic. That's not to say it hasn't happened. I may have just not been involved. They may have been before my time as well. I was there for a whole seven months so they didn't really involve me in a whole lot of high level