 Thanks everybody for coming. I appreciate a good turnout. This is this is good stuff This is my second time talking at DEF CON and it was a good turnout both years. I am Jacob West I run security research at fortify software and Matthias is one of our researchers I'll let him introduce himself in a second, but I came to the security world because I worked on static analysis So I worked on a static analysis tool at Berkeley for looking for security vulnerabilities in C and C plus plus And I got really excited when on our last project with that we applied it to the entire Red Hat Linux distribution So it was like over a hundred million lines of code It was the biggest static analysis exercise to date and what it turned up was we could find a bunch of bugs and bugs that Were easy to report easy to fix easy to deal with for the most part And so that got me really excited and I joined this company called fortify and since then we've been building Static and and now some dynamic analysis tools to find security vulnerabilities I got to write a book a few years ago called secure programming with static analysis with a colleague of mine Brian Chess That's a great reference for any programmers out there who want to understand how sort of static analysis as a technology and Software security can come together and how static analysis, which is good for hammering all different kinds of nails It's particularly good at the software security problem As I mentioned my co-host today is Matthias Mado Matthias you want to introduce yourself? I'm principal security researcher at fortify software and at fortify software I focus mainly on developing new techniques for finding vulnerabilities this by doing static analysis on source code and dynamic analysis on running apps another project that I'm working on is Developing new ways to protect web application firewall So we have some sort of web application firewall, but it's inside the application So it's context sense context sensitive so we can do actually do a better job Last year I contributed to the b-sim Europe project I'm not sure if you're familiar with b-sim but b-sim is a maturity model. I'm sorry interrupt I'm curious about this audience who's heard of b-sim. Oh That's more than I expected So b-sim for for the other people b-sim is a maturity model that is based on on real-world data from leading Software enterprises so this model can help you actually shape and and plan your software security initiative And when I was a researcher at Ghent University, I developed a code obfuscation and bind leader writing tool So that's a little bit my past All right fantastic So I'm gonna get us started today with a little background on what we mean by the insider threat problem What we mean by an insider what kind of threat they might pose Matias is going to go through a series of examples and kind of a classification of the insider threats the code level that we've identified and then I'll talk some about what tools people trying to protect against insiders Have at their disposal both in terms of process and technology and then we'll conclude with the examples that Matias already explained Lied lined up against the defense techniques and the technology that I'm going to talk about and showing you In the real world what we can actually accomplish today in terms of identifying insiders so first off Last year 43 percent more than 43 percent of organizations that responded to a CSI survey Said that they attributed at least some of their losses to insiders. So people that were employed for their company Ostensibly working towards the same goals as the company, but that exacted some damage deliberately. I Like a definition that Bishop and Gates Gates as from CA labs and Matt Bishop, of course is at UC Davis Used to define the insider threat So there are really two kinds of threats that we're worried about here when we talk about an insider somebody inside the organization One is they have some existing set of permissions access to some resource something of value and they use that in An incorrect way so they have permissions legitimately to read all the users from the database because they work in support But when they take that information and send it to their buddy on the outside or use it to spam all the users Then they've used legitimate access for a nefarious goal some illegitimate use The other class we're interested in are insiders who gain some access that they're not supposed to have so They're supposed to have a user login that would let them query their own account in LDAP They hack into the system and gain access to an admin login that can get update all the records in LDAP That's gaining more access than they were intended to have So you can take legitimate access and do illegitimate things or you can gain ill-gotten access Illegitimate access and do more than you're supposed to be able to do with it Of course insiders can fill all kinds of roles. So we hear a lot about it people We hear about people on the support team We hear about all kinds of users who might be upset with the organization or want to do some damage At fortify and from our backgrounds. We're interested in software. We're software people So in particular, we're going to talk about Developers and what kind of an insider threat a developer with access to commit code to whatever your organization's code base might be What kind of damage they can do and how we can identify the kinds of problems that they introduce So what what motivates a malicious insider one of the most interesting stories I have from two or three years of working on this is a board of directors Contacted our senior management and said what can fortify do to help with? Insider threats we said the board of directors is asking about it insider thread Programmers, they don't know those people. What are they doing? Turned out they were getting ready. This was a big financial institution around 2008 and they were getting ready to lay off 6,000 programmers They were really worried about you know one of those programmers that they were about to lay off leaving something behind a time bomb Some sort of malicious logic to do some damage to the organization. So they had started pushing their internal security team What are you going to do to protect us against this? What's your process and technology in place to avoid these insiders doing damage and in this case these two? These two potential motives right revenge This is the first and foremost But also maybe monetary gain right if I'm going to screw the the big guy Maybe I can do a little good for myself on the way out These come up again and again, and these are the concerns Both that drive and motivate the insiders and the concerns that Organizations looking to stop insiders focus on and what really motivate effort in this area like what what the work that we have done So We're looking for developers. We're looking for developers might be trying to Bring monetary gain for themselves or do some damage get revenge against their company The main crowbar the main tool the developer has in contrast to an IT administrator or someone on the support team is Code they get to check in code. They get to control the software in some form or another now We'll talk about a lot of examples today and how Just checking in code to do something really nasty It's not quite as simple as just checking in code to do something really nasty you've got some concerns like not getting caught in the way to and What how how that functionality is actually going to execute as part of a greater system? But in the end what we're really looking for is bad code, and we don't necessarily mean code that is vulnerable We'll see a lot of examples today where the malicious insider code has a perfectly legitimate use Except that's not the way it's being used in the case that we're looking for it's being used illegitimately So where do you find examples of this stuff? If you want to find the newest way to write a sequel injection vulnerability or the latest and greatest JSP tag cross-ed scripting example There's plenty of open source out there. You can find examples of these vulnerabilities very readily If you can't find it there, then you can go talk to companies and you can say okay, you've got cross-ed scripting We all know you've got cross-ed scripting. Why don't you show us some of it? And we can help you detect it and avoid it in the future People don't want to talk about insider threats if you've been bitten by an insider insider Then it means you had some employee that you hired and paid a paycheck to every month And they tried to bring your company down or they tried to steal from you it's one of the most embarrassing problems in the software security field and for that reason a Doesn't crop up a lot in open source because who's the target who are you trying to get revenge against there exactly and companies that are actually Susceptible to these or have been bitten by these don't want to talk about them. They don't want to show you examples They don't want to even admit that it's happened. So coming up with examples is a real challenge There have been some notable examples in open source or public disclosures of closed source examples of this We'll show some of those later today Some of our customers have at least gotten close to showing us the real thing They're like well it looked kind of like this except we changed a few things around so that at least gives us an idea But one of the most useful Sources of examples that we found was a contest run at Stanford in 2004 Called the obfuscated voting contest and the goal there was to write an electronic voting system that contained malicious code basically an insider threat to Miscount votes and skew the results of the election subtly towards one of the candidates So in this case we had 40 or 50 Strong programmers come in and try to circumvent a review try to keep auditors from finding their their insider code From their time bombs or whatever it might be And we found that this was a great example of the techniques that programmers are likely to use to try to avoid detection So between these three sources we've come up with the examples that we'll talk about today We have to acknowledge some related work here. This is a relatively new field, but there are plenty of people that have made important contributions Why so Palin Aang published a great paper about using static analysis on binaries to detect backdoors and other insider threats? Jeff Williams has done work in this area Matt Bishop who I mentioned earlier and Various co-authors of his he's approaching it very much from the we need an academic underpinning for all this We need to define the terms and give formal definitions for what we mean by an insider and a threat in that context And then finally at CMU there's a research group called Sylab That's doing work on the insider threat problem more broadly and they've actually with a little coercion focused somewhat on Developers and programmers code level insiders. So this is a particularly interesting one for us Okay, so there's our background I'm gonna hand it over to Matias to go through some examples and classifications of different sorts of threats that we're concerned about and Then we'll come back and look at some of the techniques we use to defend against them Okay, so there are like numerous ways to classify insider threats and to give you an idea I'm gonna go through a couple of examples and I will try to classify them So I have three examples up here. First of all, it's the Metco case. So after a particular date and time We'll delete over 70s All the files on 70 servers. So first of all, we can classify this as a very destructive insider threat Another way to classify this is it's of high risk of the company itself for for the business If this piece of code is executed, well, it will delete 70 servers, which means downtime Which means no business. So it's like of a high risk for the company itself It's also trivial to write this piece of code. It's not complicated at all So you can classify this as a very trivial way to to implement Depending on where this code lands in the application itself It's like an auto trigger. You do not have to manually poke it You just have to wait until a certain date arrives and it will execute so a fourth category can be like it is an auto trigger The second one is from Linux and The functionality of the if statement should be like the following like two flags for a particular command can only be executed when a person is rude So the options you're gonna compare the options again clone and wall and you check if the if the user is rude or zero And then you can actually use that function use the if case, but something is wrong over here Can somebody tell me why my yeah Yeah, correct. So It is not trivial to see but over here. This is not a comparison It's setting the value of setting the user ID to zero not comparing the user ID to zero and this is a little bit blended into the code It's not like a separate statement that says hey, we are making the user ID rude No, this is in the if statement itself. So this is definitely trying to hide from from a From from a manual auditor Another way to classify this is This has to do with authorization So only the root person can execute this particular command over here We have a problem. So this can also land in another bucket, which is like authorization The third one was a Borland interbase case. So how at Borland they had like a software Software design problem and the way they fixed it. They had like a chicken and egg problem about Authentication and the way they fixed is what was by inserting this line of code So they hard-coded checked if the username was politically and the password was correct So this works if if the code is closed source But of course when you open up the source to everybody if you make that source If you make it open source everybody can spot this line of code and every guy everybody can grant himself access to the code So the way we can classify that one is in three ways. So we can classify it as an authentication modification We can also classify this as hard-coded sensitive data that's in your application should not be there Hard-coded username hard-coded password should not be in your application itself And a third way to classify this is it's actually abnormal control flow path creation So this normally should not be in your application So you're creating an additional control flow through your application that bypasses the authentication mechanism So The way we actually classify Insider threats is based on characteristics to spot the insider threat So later on in this presentation this classification will become clear So now it's a little bit odd to look at this But during the face-off I'm going to explain how we actually Find will find these insider threats in your in your application and then it will become clear while we chose these classes of Insider the classes classification of insider threats So what I'm going to do next is I will go through these classes and I'll give you some code examples So I'm gonna explain the class and they're gonna give you some code examples So I will not show you how you can spot it in in your code that's for later on in this presentation That's for during the face-off So first of all logic bomb and time bomb. I'm pretty sure that everybody is familiar with that So it's a piece of malicious code that is dormant in your code until something triggers it If that triggering mechanism is related to to time then we call it a time bomb There are like numerous numerous public disclosures over here And it's mainly about comparing hard-coded date and time against the current time What we've seen so far is that if you're really destructive if you have a destructive piece of code You're using a time bomb to to set it off You choose like a date like your birthday or one when you're a year laid off or something like that One interesting case over here is the logic bomb hacker gets eight years for failed stalker ing So UBS employee planted the logic bomb into the code and went short on the stock So the logic bomb went off it deleted like over files over a thousand of computers He got caught he was prosecuted was eight years in jail and unfortunately for him the stock didn't even go down So he was like completely screwed What we actually figured out of what we found in code was not that it was really destructive So what we found in code was that people used it for various reasons for instance in the first case What we figured out for with this Logic bomb was that they tried to bypass control mechanism. I'm not going into detail of what they exactly did But so there were like checks installed and after a particular date these checks were no longer valid So you could easily bypass these checks another one that we found is Instead of being truly malicious what you can do is is update queries at random change some data And this is really hard to track it down if you do not find this particular piece of code You're going to look into where you're updating your queries or whatever Is is related to to the fact that your career your database is changing and you will not look at this piece of code So this is really hard to spot or to track down if you do not find this inside the thread Backdoors and secret credentials This category is all about securing your future access to the machine and you can achieve that in like numerous ways What you can do is adding additional credentials adding a master password Some code that allows some that allows some remote access to the machine you're bypassing some Authentication mechanism or even just a way to execute commands on the machine is enough to grant yourself access later on Couple of examples over here. Well, I've explained already the Borland inter case Interbase example where you insert like a hard-coded username and password another one to secure your way afterwards was the WordPress backdoor, so You can still execute Commands on the server itself if you know the special parameter So knowing the special parameter means you can execute whatever you want on the server So later on you can grant yourself access you can execute whatever you want on the server The last one that we found in code was Inserting some special credentials at startup. So if you're starting up your application the first thing that's executed well after a while the credentials database is created and They inserted a line of code that added an additional pair of credentials to that credentials database So later on you could come back and use the username Insider and the password thread to grant yourself access to that particular machine and So backdoors and inside and secret credentials are not like limited to these quote-quote normal applications All applications can be backdoor. For instance Optics pro it's actually a backdoor program backdoor program containing a backdoor because it was like a master password to grant yourself access Sub 7 is a Trojan with it with the backdoor. So it's not limited to two normal applications nefarious communications, so nefarious communication is is about a fixed communication channel to transfer data outside of the Organization so we cannot execute whatever you want on the on the on the server But it contains some code in it and we can poke it and trigger it or it Connects back to us and it gives us some data So it's an excellent way to transfer sensitive data out of the organization. So a couple of examples over here So you can do that for instance by the server opening a socket or opening a network connection to our machine and transferring data to us or you can actually The server containing some mailing Application that mails once in a while some secret information to us and the way you can The way that code becomes active is sometimes it's time-based So on a certain date or on a regular basis it sends sends over some information or you can actually monitor the database some some Confidential database and if that database is updated it's going to transfer the whole database to our to our server Or you can actually poke it if you know how to poke it. It gives you back the information couple of a couple of examples over here, so This is about making a server sorry making a socket creating a new socket and They were actually monitoring one specific file I've changed it to confidential file So they were monitoring one specific file and if that file changed it opened it created a new socket on port 666 And it transferred the whole file out of the organization This is a very similar one. It's pretty much the same but instead of opening a socket over here It's posed to to a website so to an evil website Again, you can monitor the file and transfer it out or you can like manually poke it The email so the blackberry email spying case so in a particular country this update was Advertised as a performance update when you actually Decompiled the performance update you found the following piece of code So it contains like some forwarding mechanism that for each mail you receive on your blackberry. It was also forwarded to that particular address That was actually deliberately into the code and it was there to comply with regulations So Work of Jeff Williams. Well, sorry dynamic code injection and manipulation This category is all about changing adding compiling code on the fly So you have some some source code in your application and you compile it on the fly and execute it We got inspired by Jeff Williams work on on on this particular category So he enumerates like a lot of categories like abuse of reflection where you rewrite read-only variables Resource rewriting compilation on the fly abusing class loaders pretty neat stuff One example over here It comes out of his work out of Jeff Williams work. It's the abuse of reflection So what you're actually doing over here? So you have some final strings in your application. You're not able to modify it You're not able to change the data of these final strings. What you're doing is you change the string class So you change the string class in such a way that all final strings become writable So it does no matter it doesn't matter anymore. If there's final with the string you can actually rewrite all the final strings Obfuscation and camouflage over here. You're trying to hide from from different actors. For instance, you can try to hide from auditors These days we see that people try to hide from from tools too But at the beginning it was mainly about hiding from people looking at the code doing manual analysis on the code and The way they do it is they make like really subtle changes that and by doing that they achieve their goal But it's really hard to spot. I went through the first one That didn't check if it was rude, but it was actually setting the route another one is the x11 case. So the forgotten parenthesis There was like there are like two checks over here to get you ID and to get you ID so the effective user ID But with the second one as you can see they forgot the parenthesis. So This will make the statement always true, which means That it can use whatever module you want and normally that was exclusively for the route But now over here because of the forgotten parenthesis everybody can add a module And it's just a mix-up of names. So if you if you have like a variable with the same name you can do this stuff. Oh Yeah, another one over here is Assume you want to hide the following piece of code So you want to hide that you want to remove everything in a particular in the root directory So you want to rip out everything on a particular machine But this statement is obvious to spot for for a manual auditor The way you can achieve that is you're gonna encode it with base 64 for instance, you're gonna encode it You're gonna decode it on the fly and you're gonna execute it The good part over here is these three lines of code do not have to be In sequential order you can spread them out over your application Where it makes more sense to to put that piece of code. So Once you come to the execution of the string you have no idea what's actually executed It's not obvious. So you have to track back and try to figure out what's actually executed over here So that's like a need way to to hide Another way to hide from manual inspector is using very simple substitution ciphers like for instance rod 13 for square beef it and so on and so on Over here what they did is they wanted to insert something in a particular database But they want to hide what kind of database they were inserting to and they also wanted to hide what kind of values They were inserting to that database So what they did up front was they declared a couple of variables the database the data and data to they didn't give any Specific names and they encoded it with rod 13 rod 13 is is really useful because it only Requires like six lines of code to implement. So if it's not by default in your application You only need six lines of code and you have it over there So what this was actually doing it was inserting into the credentials database insider and thread But it was a little bit hidden from so I've shown that example before where it was just like you inserted in the credentials Database over here. It's much more harder to to track down by use of a for a manual auditor So I've introduced like a little bit the classes of insider threads and and a couple of examples Jacob is going to tell how what kind of techniques you can use to to spot these Militious code Thanks, Matias So I'm gonna talk about a couple of techniques first and foremost Maybe most popular peer review right manual code review if you're reading through code that somebody else in your organization wrote Static analysis is when it comes to security vulnerabilities a great way to Expand on and expedite the code review process. So we're going to talk about how static analysis can apply to finding insider threats a Couple of slides on runtime techniques for tracking these down and then a little bit about how to interpret the results It takes a little bit of a different mindset when you're looking for an insider as opposed to an accidentally introduced vulnerability so first off if You're doing a code review and you see those lines up at the top you're gonna ask What on earth is that? Why is that in the code? That would look strange no matter what code base it was in? I mean, maybe it's a crypto key or something interesting, but whatever it is there aren't many of them, right? The second line is a little suspicious, but there isn't anything completely out of the place there You're checking a request parameter, and then you're doing something based on the request parameter That happens in a lot of code bases usually not with a hard-coded name and a very specific case like this But that one might slip by you know if you came back after lunch, and you were reading through code You saw that one you might let it let it go What about the last one you know somebody over on the side of the room I think tracked down the problem with the last line of code there when we put one line of code on the Screen and asked what's wrong with it if that was one of a thousand or ten thousand lines of code That you've read this day or week you're much less likely to see it so the ability to track these things down with manual code review is Relatively low and we've seen this with the success of insiders writing code like this that is lane dormant in code bases That have been reviewed year after year because it's it's not a smoking gun It's a breadcrumb that kind of points towards maybe a possible problem Something a little bit fishy, but again, it's nothing egregious in and of itself So those are really the two challenges with using manual review for insider threats One is your reviewer has to know what you're looking for what he's looking for what she's looking for So they need to have a lot of domain expertise in terms of the kinds of techniques that insiders are going to use To do some sort of damage so a your reviewer has to be pretty well trained They've got to be thinking about this problem specifically to Where do you look? Some vulnerabilities. It's easy. I'm going to look where important stuff is happening Right the database queries that access the table with money in it I'm going to go code review those make sure there's no sequel injection the pages that deal with Authentication or other sensitive activity. I'm going to check to make sure XSS and CSRF aren't a problem insider threat code could be anywhere and a lot of it isn't terribly interesting right a lot of it is as Matias showed on the slide before Hiding the thing that you want to use decoding the thing that you want to use and then eventually may be using it and only the place Where you use it would get any sort of security attention because the other two can live in any arbitrary place some dark corner of the code base so static analysis can help find insider threats in code two ways one it can Represent all of that domain expertise. What are all the kinds of ways insiders might attack a code base in a set of rules? And granted those rules aren't the same as the rules we use to find normal vulnerabilities but they can be codified and Used systematically so the static analysis tool can apply those time and time again. The reviewer doesn't need to keep them all in their head Second they can tie those breadcrumbs together So I said a lot of these techniques that insiders are going to use aren't necessarily malicious There's a malicious use for them and then there are lots of legitimate uses too So static analysis can start to look at an entire code base as opposed to However much of it we can cover with manual review 1% 3% maybe it can look at an entire code base and contract down all these breadcrumbs and it can start to show you a heat map of the application it can start to show you where is they They're a particular particularly thick line of breadcrumbs. Where do things start to look like well? There's this suspicious thing and that's suspicious thing and oh there are three of them there that really warrants some further review They're not going to find smoking guns and we'll look at examples of that because these things aren't smoking guns They're again perfectly legitimate ways to do all the things that an insider would want to do They're just doing it in an illegitimate way So just to give you a little idea of what the inside of a static analysis tool looks like the piece We're talking about is just one aspect of the the tool overall So source code comes in the front end you build a model from that source code You run some various kinds of analysis on it and out the other end come results The piece that we're talking about changing are the rules these are the definitions that tell the analyzer what to look for what constitutes a problem and Here it's very different from vulnerability detection Vulnerability detection is useful too sometimes an insider leverages Something that is technically just vulnerable. This is an example that Matias talked a little bit about earlier Except he just had the first line of code. So the first line of code says This is PHP code. I'm going to check a request parameter And if it exists called IZ then I'm going to call this get theme M command function What the hell does get theme M command? I guess it's going to load a theme if I was doing a manual review and I wasn't looking too carefully at this I might just read right past it But the definition of get theme M command is to execute pastor which just executes a shell command or script on the server So this is a command ejection vulnerability and any static analysis tool worth its worth its price Would find this out of the box because you're taking a request parameter IZ and passing it to execute as a shell command That's just a bad idea no matter why you're doing it It doesn't matter that you've got that conditional over to the left that says only do it sometimes It's always a bad deal. So out of the box rules would find this problem Some other problems aren't as obvious. They aren't vulnerabilities in and of themselves So let's talk about a scenario that Matias already showed us code for a laid-off employee installs code that reads the entire database on a regular basis and sends the results over the network So there's a couple of bad things going on there And I'm going to talk about them one by one and then show you from a detection standpoint how we might go about identifying this broader insider threat So first off if you're reading the entire database an entire table for any reason It's pretty suspicious if you're doing migration. That's a possible legitimate use for it But you probably are looking at more than just one table at a time too So if you have a query in your code base that does a select star on a single table That is in and of itself suspicious It's not a vulnerability all by itself It's not a command injection like the slide we looked at last but it's pretty weird And if you're a manual reviewer looking for insider threats, this is a good place to start This is a bread crumb in in a pile second off you're reading something from the database and We'll assume you're in a web app here because a lot of people looking for these things are in web apps, right? It's it's your banking site. It's your e-commerce site. These are where insiders tend to target You're doing manual network connection management. You're opening sockets particularly sockets with hard-coded ports again Perfectly legitimate reasons to do that, but it's pretty uncommon Why not not not not something you'd regularly see in a web application? So we can create another rule that flags uses of sockets in web applications and in particular Hard-coded socket numbers just really strange. You're building in some special sauce there Whether it's for legitimate purposes or malicious purposes remains to be seen All right a regular basis Most applications don't depend on fixed points in time That's not to say there aren't legitimate reasons why they might but if you're checking the current time That's a breadcrumb. That's something interesting. You're not going to do that a million places in your code base So a manual reviewer looking for insider threats could afford to look at all the places where you read the current time and check on What you do with it? In particular an interesting thing you might do with it is compare it against some hard-coded value So if you access the current time That's a little suspicious if you access the current time and then compare it to a static string Compiled time string in your application. That's really bizarre. There are very few legitimate reasons for that So again the pile of breadcrumbs gets a little thicker. I Guess that's the last breadcrumb. This is a lot of what goes into the detection techniques that Matisse is going to show examples of later I'll talk about this a little more when you talk about interpreting results But again, you don't get to find a single smoking gun. You're looking for Things that indicate something suspicious and the more suspicious things you find and the more suspicious each of them is The more likely you are to have an insider threat, you know The goal for most organizations would be to run this rule pack against their code base and not find anything That wouldn't be terribly surprising. You might just not find anything Or you might find a few of these that are being used legitimately Or you might find a few more and see hey this this spot on the heat map is looking a little red This is actually a problem Okay, so aside from static analysis, how much how might you track down insider threats? Your existing functional testing in QA can be pretty useful here Here for the most part you're looking for code that isn't run very often and probably during a functional test It's code that's gonna look dead in fact if you have a mechanism for overlaying What's potentially security relevant so talking to the database talking to a directory server printing content out to the web? Executing shell commands right there. They're a handful of things that are security sensitive If you can overlay your ability to detect dead code during functional testing with things that might be security sensitive You find people opening sockets in dead code. That's a good thing to get rid of either way at the very least It's dead code that might introduce a bug or cause some problem later on but it's also quite possible that it's behind one of these logic bombs or hard-coded credentials time bomb something that's meant to not execute on a regular basis in Production runtime monitoring can do you some good too because the now This is a little a little late in the game But if you suddenly see a bunch of data being written out from your web app that isn't typically or a bunch of traffic on your admin Console that isn't typically there There are things at runtime that suggests something is out of the normal here And this comes down to for the most part anomaly detection looking for common behavior and looking for Things that deviate from that so watching our application both in QA and in production can help track these things down We're particularly partial to the static analysis approach because that's where we've put a lot of our energy But belt and suspenders does apply here, too Okay, so now you have a really big pile of breadcrumbs none of them are smoking guns Let's talk about this code for a second. So you get the current time. That's nasty You can pair the current time against some hard-coded string. That's nasty, too This might legitimately be update code. You might say we're a startup and once a month We just want to check to make sure there aren't some new bits that we should download And this was a hacky way to to clue G way to introduce that It's possible. You might be able to talk your way around it But it might also be nasty and the fact that we see the two breadcrumbs that we latched on to before right accessing the current time And doing a comparison based on it Give us a little more confidence So in the end what we can really do with these results is we can associate a Strength with them or a priority with them just like we would other vulnerabilities that we detected in an automated fashion so some problems like Accessing the current time might only give us a low fidelity feed that it's an insider threat There might be lots of reasons you access the current time But if you compare that with a hard-coded time, that's that's pretty strange Maybe that goes at the high end of the bucket So each one of these breadcrumbs has a strength of implication How likely is it to describe a real vulnerability and one of the most exciting things you can do with static analysis is Combine these so you have a low a low a medium and they're all in the same method Okay, that method definitely warrants manual review But it's important to see that eventually this does come back to a manual review process You need a smart person who understands what an insider might go after and how they might do it And that person needs to take the code base as one input and the feedback from the analysis tools as another input And maybe some runtime monitoring as another input and combine all these things to start looking for the real problems Of course if you've been doing this by hand This is gonna sound great because now you've got more inputs used to just be the code right and figure it out Now you've got other things to help start to develop that heat map if however You weren't doing it before if you weren't looking for insider threats and somebody says yep Now you got all these inputs and it's up to you to interpret them It's gonna sound like a hard problem. So a piece of this all boils back down into What's the motivation and what's the organization actually after finding right are they interested in finding these things if so This is gonna sound like a good story Okay, so I talked a little bit about how you might find these things Matias talked about what these things are classes of insider threats Now we're gonna line the two up and talk a little bit about how latching on to some of these specific attributes at analysis time We can point out problems at varying degrees of priority The face-off so what have we done? So where are we today? We have 17 insider threat categories and we've implemented them all for Java We found multiple real issues and enterprise code so we have confirmed issues that we found in enterprise code So the way I'm going to do this is I'm gonna rerun the examples and you're gonna describe what we flag so Just to have all up to have this all up. We have like 17 categories. They are listed here They are in the slides so you can if you want to talk about them afterwards feel free to do so So all these categories are like sub categories to these main classes of insider threats So I will tell you what we flag in the code what we flag and how that relates to the code that we've shown before so first thing first logic bomb and Logic and time bomb so what we flag is in the light and a low priority We if you get the current time it goes into the low priority bucket If you get the current time and you compare it against the get the current time and you compare it It goes in the medium priority bucket if you get the current time you compare it and you compare it to a hard Coded date then we drop it in the high priority bucket. So obviously this was found in the high priority bucket So it's ticked out. We found it immediately Backdoors and secret credentials the way we flag step goes as follow we flag insertions in the database if you Insert something in the credential database. It's a low issue because there is a there are good reasons to insert new credentials in the database If you insert something in the credential database, and it's a hard-coded credential. Well, that's pretty suspicion and that's that's at least Medium issue if you insert in the credential database a hard-coded credential and you do that at startup Which means that these credentials will always be there and you can always use them. Well, then it's definitely a high issue And and that's in the perfect way to look to secure yourself access to the machine What else do we report in this category? We report comparing hard-coded usernames and hard-coded passwords So all the all the comparisons of the username with a hard-coded username and or the password with a hard-coded password We flagged as a that as a high priority because that's most likely a threat Okay, so we have a couple of Username and password if the variable is username and password. It's obvious, but not in all applications username and password So you can also customize your we have some custom rules available for your application that you can simply Use your variables And and we will automatically trigger them Default command injection in WordPress. So for some cases we do not need any special rules So the command injection it will be found by regular static analysis if you really want to dig have this in your Inside a thread code base what you can do is you can take the command injection rules and port them over to to the inside a threat rule pack and we will flag that as as a Backdoor and secret credential On the various communication we flag hard-coded ports and sockets So if you use a socket and it's a if you create a new socket with a hard-coded port We flagged as that as a high issue what we also flag in general is server sockets and Most of our customer base. They are not using any server sockets So if you do we flag that as a medium issue We also flag accessing hard-coded files So we have a couple of default rules if you try to access ETC password or whatever file you do not want to expose We were gonna flag that we also have custom rules So if you know of some files that are really confidential to your organization, you can set them and and We we can track that Hard-coded email addresses so every hard-coded email address is flagged in in your applications There are again legitimate uses usages when you want to contact the administrator or whatever That's a legit usage, but normal case. It's a legit. We can also flag mailing APIs for instance We flag functions that change the read-only flag so for instance the set accessible On the field class we're gonna flag that because that's that you shouldn't use that in in your application We have similar rules for all the categories that are described in Jeff Williams paper But we broadened our view and we thought about more examples and how people do stuff So we have rules for all these things Obfuscation and camouflage we flag the usage of a single equals inside an if statement You should not use a single if equals in an if statement. So we flag that in the obfuscation and camouflage We also flag When you're having the same name of a for a variable as a common function If you're doing that, we will flag that in the obfuscation and camouflage And then the really useful one is you if you're trying to decode hard-coded strings We're gonna flag that as as an obfuscation camouflage. That's that turned out be a really useful one We found a lot of issues with this kind of rule If you decode a hard-coded string, it's very suspicious. It's implicitly saying like hey I'm hiding something so you do not want to have that in your code So I hope that makes sense and For the conclusion I want to hand off to Jacob again Thanks again Matias. All right. We're almost done guys first and particularly for the Defconn crowd How do you avoid getting caught? So companies are gonna start using technology like this to find insider threats How do you avoid detection? What's the technique for not getting caught? First off and one that we've seen used again and again is make the code look as legitimate as possible Make it look like real code and make it look like code that Fits with its surroundings If you have a five-line Encrypted string hard-coded at the top of your class definition It's gonna stand out no matter who's looking at it even if it's just a cursory review to make some other edit So make it look real Also, if you can make it benign Meaning it doesn't show up as a purely a pure vulnerability like that command injection that I talked about in WordPress That's better. All right. The more it looks like application logic as opposed to a vulnerability The less likely it is to turn up for people who aren't really looking for insider threats It's good to understand who you're up against so Understand what? Organizations and security teams are looking for and understand what tools they're using to look for those problems if you can Get access to one of those tools if you can run for example the static analysis tool against your code again and again You can tweak little attributes of it until it doesn't show up and this is of course a real concern for the defenders But finally There's a lot more attention on this problem today So one way not to get caught would be to just not do it in the first place I think getting a job on the support team and avoiding the source code it all together would probably be an easier route to wreak some insider damage So for the good guys What is this all bowled down to? It's really like looking for a needle in the haystack. So you're looking for a Huge variety of different tools and attackers that people might attacks that people might use It's hard to think with this big number three in front of my eyes. Sorry The attackers have a big arsenal of tools at their disposal because they've got the entire programming language or multiple programming languages If you think of a multi-tier application They might do some damage in JSP or Java or PL sequel or a native C procedure that no one's ever going to look at so they have a lot of tools at the disposal and What we've seen for the most part has been They put a lot of planning into this they think about the code for a long time They've got it in place the year before they get laid off. It's not a spur of the moment. Oh, man I hate my pointy-headed boss. I'm gonna go code up something really nasty put a lot of effort into these So they've got a lot of variety available to them and they have a Really large window of attack much bigger than the period between when the board decides to lay people off and they actually do Technology can help a lot here I hope with some of the examples that we showed today you guys see that static analysis Although is never going to smoke out point out smoking guns is a big help over manual review and can really Reduce a multi-million line application into a heat map of areas that a code review can can do more value in by focusing on those those hot spots and It's really key that Going into it. You don't just rely on technology, but that you have your auditors and the people looking for problems Apply the right mindset right the things coming out of the static or runtime analysis aren't smoking guns They aren't vulnerabilities. They're breadcrumbs. They're indicators of a potential problem and until you get a thick enough pile of breadcrumbs You're not guaranteed of finding a problem Thanks very much. I know software is not the usual def con def con topic, but I appreciate you humoring us And thanks a lot for coming out today. We'll be happy to take any questions