 All right. Next up we have Morgan Roman giving a talk on security, integrated security testing. Please welcome Morgan. Hello. My name is Morgan Roman and I'm going to tell you how to turn your testing department and your QA department into a huge problem. So the first thing I like to talk about problems is my favorite HR violation ever. And it's pretty simple. A tester likes to find usability bugs. And the way he does it is he puts in the HR tag, which in HTML shows up as a line, horizontal rule, and then he files a bug. Just shout it out. Anyone can tell me what's interesting about this. I take your silence as lack of knowledge. Now, what's interesting about this is essentially HTML injection, which can later be used for something called cross-site scripting. So I took this tester and I said, hey, that's pretty fun. What I like you to do is instead of using the HR tag, use script alert 1. And just tell me if something pops up. And of course, quite a few things popped up. And I said, oh, good job. And I gave him credit for it. And all of a sudden all his usability bugs, which no one wanted to fix and they just sort of ignored, and that really annoyed him, became security bugs and then they had to fix it. So another question for the audience. How is this different from what penetration testers do? How is this different from hackers? I like your response. Nothing. Basically nothing. I mean, we'll try to find maybe some escape sequences or something along those lines, but that's really all we're doing. So that's actually my entire speech. You guys can leave now. And that's the entire thing. So yeah, feel free to clap. No, after this presentation, I'll go into the details to actually do this and at scale. And you're basically going to make existing testing stuff that you have, which could be automated tests or manual tests, find security bugs with as little false positives as possible. So I really like that. And with as little bit of work as possible. So be lazy and get other people to do your work. What that means is you can find these crazy bugs and you can keep them fixed, because you have testers constantly looking for them for you. So because this is a testing conference, sorry, this is a hacking conference, not a testing conference. I want to introduce two things that you may already know, but just to, you know, let everyone know. And there is two of them, but I have an off by one error, as you can see right there. One is Selenium, which is essentially browser automation. So if I want to make sure that a web application works all the time, I can script the browser to do all these things. That's what this tool called Selenium is used for. It's pretty common for QA departments. The next thing is integration testing. Integration testing is essentially not a unit test where I'm checking does 2 plus 2 equal 4 on my internal logic. It actually tests the entire calculator, right? Or tests the entire flow. It's a little bit different. And anyway, if you guys have questions about anything or if there's something that goes over your head, just shout it out. I kind of like questions during my talk, a little weird. Another thing about me is I started originally as a tester. And then I, you know, continued working at DocuSign and I wrote a lot of integration tests for the API and web interface. And then eventually I switched to application security because I kind of had a little bit of talent in the area. And one of my main things I like to do is I like to find tools that aren't normally used for security but we're already using them wherever and somehow leverage them. And that's sort of the Archimedes principle right here where Archimedes says, let's get a long lever and using the moment arm, I can move a lot larger object than I normally could with a lot less force. And that's the whole idea. So, just going to throw this out there as a question. If you normally want to automatically find security bugs in a web application, what do you do? What do you use? Just shout out. Sorry? Pentesters. Pentesters, yeah. Zap. Zap's good. Releasing it publicly anyway. That works as well. What about dynamic scanners? What do you think of those? I'll take your silence as anger because I don't really like them either. So Burp and Zap, those are all powerful tools. Pentesters themselves, yeah, you know, we can just enslave them and they'd be pretty useful. But most of the time, oftentimes they'll say like certain dynamic scanning applications, right? And I don't really like them because this is how they work. You normally just log in, you program it somehow. It will just send a bunch of payloads. So I'll try to find SQL injection in every single cookie. Great. And then it'll interpret it somehow and give you a report with like a lot of interesting, definitely always true results, right? That always happens. Except when it sends in these payloads, it doesn't understand the business logic. So if you're doing like a banking application, it never creates like a bank account, right? It never will transfer money between two people. It never clicks on things. It doesn't understand the main workflows. And when you get the report, you end up with this result, which I really, really like. And I'm sure all of you like as well. Who here works in sort of like application security? Raise your hand. All right. Who here works just regular development or QA, right? So you should thank, well, hopefully, you should thank your application security people for dealing with all these for you because that's mainly our job. So what we can learn as either hackers or defenders or any sort of thing from regular testers is a key idea right here. And they sort of work like this, right? They find the most simplest workflow. Magic, find it in, log in, do what you're supposed to do. And then you create like an automated test for it. So that way you can test that. But then you start modifying it. You start saying, all right, I have this list of test cases in my test plan, and I'm going to make sure that the test that I have will hit all of them. But you start with the most basic one. And then usually this ends up with a few bugs. And then you can reuse that previously automated stuff to test that that bug was fixed. Pretty great. For those who are doubt that this is what's done, this is what I basically used to do. And what's really fascinating about this is you get something real, right? You don't get some imaginary workflow. So oftentimes, you know, penetration testers, what they'll do is they'll go in and they'll do things that nobody ever will ever do, right? So odds are, let's say you send in some payload, right? Well, it's unlikely for a user to render that, right? So for any cross-site scripting. Now that's important for edge case testing. But if you want some really important kind of the main areas, testers can help you out a lot with there. And they also do tons of stuff, right? They buy the rules, they have to test every single scenario they can, right? In order to sort of say the application will work in most situations. For hacking, you really just need to go for, you're just aiming for one target. You're not aiming for all these weird little areas, right? You're looking for just one way I can get in. Bam, I got it. And the other beautiful thing about this is oftentimes they want to run these multiple times to make sure it is fixed once and for all. That's pretty good. So once again, the entire crux of this idea is, and by the way, I'm using cross-site scripting because it's a simple thing to explain. We'll go into more bugs in the future. Basically, make all your strings a basic payload. You don't have to be very fancy with this. And then send it to all the scenarios that you have in your test cases. If it shows a failure or if it alerts or anything like that, or if it just breaks the test because of that HTML rendering, then voila, you have a fully working example. POC. That's amazing. And you can give this to anyone in the organization and they can use it to reproduce the bug. It's pretty great. So the basic idea is you send in the payload, you keep running the test and then while it's sort of running, you're checking if it worked. Because the whole idea is maybe on step one you could send in the payload, but on step 17, it'll actually fire, which is incredible. But most sort of automated scanners aren't intelligent enough to figure that out. So let's talk about the easy way. We create a payload. Has anyone here ever used Selenium? Can you tell me what's going to cause problems with this? Just shout out if you know. Yeah, alert boxes are a lot of fun in Selenium. They basically stop any automation unless you're really prepared for it, but even then that's a huge pain. This is what I like to do instead. In automation, no one's actually looking at it, so you can just look in the logs later for this. And if you do that, bam, it's pretty easy. So you go in, use it. Once again, I seem to have problem with numbers. And you then read the Selenium logs to find it later. And I usually have the third step just constantly running. It'll check after every action, which is very cool. Now before I move on to other types of bugs, I'd like to show you all a demo. So we're going to go through a couple of them. That shows up all right. And so guess what? I'm just going to show you some horrible things with my company, and that'll be a good way I can get fired, right? No. This is a toy application that works, it just is like taking notes and sharing notes with another user, right? So it's a good way to show it. So we're going to run some Selenium automation. I purposely slowed it down by a ton. So have that pop up. And you can see inside Chrome here, and what I'll show you as well is the console. So ignore that. And what, in this test what I'm doing is say, let's make the user have some script in there, and see what happens. And I like console error a lot too, because it shows up as red. Okay, nothing happens. It looks like it's encoded properly. We're going to try this again. So this is a test that runs through and we'll try the same test, but with different parts as the payload. So this one will make the note that it has in there. And you can see it'll automatically put in the payload and try it. Oh, oops. Sorry. It's going to restart before that. So it's going to go back in. It's going to log in again. And then it's going to put in, for the payload, an example attack. So here we have it. So then on save, hope it wasn't too late, it output right here. And I like to kind of use random numbers that are large, so I can be guaranteed that nothing in the logs ever collides. So that way to reduce false positives. So now it goes in and it reads the note as another user. So now you have an example of, hey, it's not just self-excess, I can do this with someone else. So we can, another thing I want to point out, and I think this might be really hard for you to read. So let's try to fix that. Is that easier to read now? Not really. Well, the main point I want to point out here is any error that you print out for testers should include something very descriptive and very clear, and maybe even like a contact email. Because if a tester finds this, they don't know what to do with it. And they just might say, oh, you broke the test, but actually it's a pretty serious issue. All right. Who here knows what angular injection is? Raise your hand. I'm guessing no one. So now angular is sort of losing steam due to a react, but angular allows you to evaluate a lot of, how do I put it, sort of templating information. So inside double curly braces, if you say give me variable A, it'll render it on the page. Well, if the attacker sends in double curly braces, do some action, it can be used to leverage and get executed on the other page. So let's try this real quick. Same test. It's going to be really exciting. So this one, you can see a basic payload. I like to take two large numbers and multiply them. And then you can see right here in the page, it disappeared. It will show it actually multiply. So if you see 144 in this case, voila, we have something exploited. So you have that all right there. And this test, and I'll show you in a minute, it essentially is just a test I would have wrote if I were testing this particular web app, right? It's not particularly fancy. There's nothing really going on. It's the kind of thing that your QA department will write anyway. And all you have to do is leverage what they're already doing. And you notice like I'm not doing anything right now. It's going all on. So let that go. So essentially all I did to make this happen is I wrote a utility that will just say all right, create some sort of payload based on the username. And all it does is get a random number, do script alert one, and append it to whatever string that I have. Now one thing that's really great to do is you can say here are a bunch of different escape sequences. Let's try them out and see which one works. In this case, I can sort of show you. This is what I like to do for SQL injection. SQL injection, I've never really had like a very clean way to verify it happened. I just know if the test breaks. So what that means is I got some SQL injection in there. It just caused me unable to read the data that I was intending to put in if I were a regular user or if the test. And so therefore I can just say all right, test breaks, what's going on here, right? So I'm just going to show you one. This is an API test and all I'm doing is just sending in like a single quote, right? Seeing if that will work. So there's not much to see on this one because it just runs, it's an API test. There's no UI. This is a case of hey, there's no Selenium, but you can kind of use this for Postman or whatever you're testing. To let you all see what's going on a little bit, I have one of my favorite tools on the planet, Burp. And what you can see right here is hey, we have a payload right here and the response, it might respond. This one, it sort of responds. But then you have other areas where it cannot get certain things. Trying to find one. So essentially right here, what it'll do is it would not return the expected value. So this one you kind of have to use, so this sort of methodology you have to use something to determine what's going on under hood and I'll sort of explain what's happening when we get back to the slide. So we talked about angle interjection. Basically I create a payload, two large numbers, fairly random, and then I multiply them and then I look through the DOM. And that's pretty simple. You kind of just set in some JavaScript to say return the document dot body. This one I like to use as large numbers as possible so you don't have any collisions inside the DOM because the number four can show up a lot, but 1-882-1744 is pretty rare. Alright, SQL injection. The thing that was quite hard to see. Essentially you create a payload, you send it in and you sort of expect you get some interesting error. You either have a time delay if you're testing for blind SQL injection, that sort of thing. But if it's a dynamic scanner, it oftentimes happens too early. So that's kind of what you would need a manual pen tester for. This way you can kind of send it in and when the test just breaks and it throws an error and it's kind of mysterious what's going on when you send in a single quote in this case. Alright, voila. You kind of have an example of SQL injection that requires some investigation. Alright, XXC. Who here knows what XXC is? Cool, three of you. Most XML parsers follow the XML spec. The XML spec is Turing Complete, which means you could basically, well, I'm not sure if it's Turing Complete, but it's basically its own programming language. And you can do a lot of nasty things. One is set variables and set external entities, right? So what this attack does is the parser will read it and then say, alright, I see some variable A or some entity A, and then I'm going to assign whatever you want. And it could be a file in the file system, so include some nodes in an XML file. It could be some part of the XML file or it can call out to the internet. So you send this into an application and it will read whatever you want, right? Or it's pretty powerful, right? So oftentimes, and I think this is the way Burp works under the hood, is essentially you send it in, right? I think the intruder does this. You send it in and it will check the endpoint to send. But oftentimes, with the application you're testing, somebody sets it up correctly, which is pretty rare, but they do do that occasionally and they will have it so you can't call out to the internet. Okay, fair enough. But they're still exit C, you could probably still read a file in the file system. Well, testers have an interesting game they get to play. They are oftentimes running it locally on the machine, whatever application. So you have a little bit extra ability right here. So yeah, you could go ahead and do this. But if you have file access, you can do this. Now, the third command on the bottom, ls.lu, you essentially just see if there's any file read, right? And then voila, you say, oh, there was a recent file read a couple seconds ago. Obviously something here worked. I quite like this one. It's pretty powerful. All right, CSV injection. Another fun bug, not kind of weird because you kind of have to say okay inside of Excel and Excel is a very secure application. And so essentially you create a payload and you're saying let's evaluate some function, right? So two plus two, but you try to get like a shallow command, something like that. So you send it in, right? And then you say okay, somewhere in this test blow, they're going to download a CSV. So you just write a function inside their testing framework to then say all right, just read the CSV, make sure anything that sort of looks like an equation right here will be prepended with a quote, right? So that will tell Excel not to evaluate the equation. This protects against that. All right, let's talk about this one. This is surprisingly powerful. So if you're just looking to make sure that people follow the correct cookie policy, you can manually test this and have it go through and all this stuff and create rules and training and all this stuff. Or you can just have the test look for it automatically. And that's going to be running in the background regardless. Every deploy, part of CI, CD, that sort of thing. And in this case, you can just get all the cookies on the browser inside Selenium and just check which ones don't have the secure flag. Pretty simple. If there are certain cookies that you don't want to be able to be accessed by the browser, so only sent via HTTP, not by the JavaScript inside the page, you can just send in some JavaScript to return all the cookies that JavaScript can see. Pretty simple. Let's make this more interesting. So this was an attack on a lot of U.S. and U.K. government sites, which are known to be very secure. And basically, they were relying on a CDN. And that CDN was hacked. And essentially, they were mining cryptocurrency for a lot of people. And the sheer number of sites that were attacked, essentially, is enormous, but as it turns out, the attackers did not get away with much money. They made like $24. We don't know who they are, but, you know, pretty good job for $24, I'd say. But here is another interesting thing. So recently, there was a Ticketmaster breach. And that breach was due to the fact they were relying on an external provider for some JavaScript on their page. And that was harvesting user credentials, I believe. Okay. One way you can test to make sure this doesn't happen is you check the resource, you take a hash, this is coming out, be a new thing, you take a hash of whatever JavaScript you load, and you make sure that never changes. Most CDNs do not support that right now. So the easiest way to do it is you just say, I'm going to load all my scripts internally. So it's all going to come from the same origin as originally. Cool. You make that policy, you send out the training, you do all that work. And some programmer decides, well, I'm going to dynamically load it somehow anyway. Because I need this version of jQuery in this weird area. And that's great. I really like it when, you know, programmers don't listen to what I say. So as it turns out, this little incantation here, window.performance.getEntries is a JavaScript command you can run inside of a Selenium instance that will tell you, get me the performance logs for everything, all XHR requests, all JavaScript requests, everything that it does. And then you can also use this to measure timing, all sorts of things. Pretty powerful. But what you can do with it is you say, all script-based requests, let's see where they're going. Is there just coming to me or just coming to maybe our private CDN or something that you have, you just have sort of a white list of what rules that you can follow. And if it goes outside of that, you can start throwing an error inside the test. Now, what's very beautiful about this is tests that go deep into the flow. So for some reason, in like one weird flow, they start loading external script. Voila, you can cover it. And because tests are regularly run all the time, you're checking all the time. Basically, you don't have to do work anymore. Let's talk about authorization bugs. These are the money bugs, because they're essentially much harder to find and require an actual proper testing. So imaginary application, Alice creates a resource, Bob reads it, that kind of thing. All right, maybe our attacker Eve can Eve's drop and start doing some bad things, provided they know the resource ID. Cool. So to do this, you can test this, but you have a lot of things you have to scramble. So one thing you can do is you say, all right, here are the preparation steps, making the resource. Then you attempt the other ones, so you try to get the resource, you try to update the resource, and you try to read it as someone else, that sort of thing. And to really do some great testing in testers, they kind of know this. They understand that, hey, we can do every angle. As you change, what is the preparation step and what is the attack step? So steps one and two are just preparation. But then, hey, Eve can share the resource with herself. That's pretty cool. Or maybe some steps have to happen before all of a sudden authorization fails. This gives you a lot of plausible deniability, but also a lot of coverage. I want to go why this works and what you can sort of do to make this work with your organization or let's say you're an external pen tester and you're trying to help whatever your client wants. Basically, any sort of functionality will outlive the feature. This is something that people don't really think about. Who here has refactored code? I'm assuming more of the view have refactored code before. But essentially, when you refactor code, you're changing, yeah, you're renaming things, you're moving things around, but you're ultimately actually changing how the internal thing works. The same feature has to exist. That's the whole idea. You're moving things around to make it clear to read or a little bit faster, but the feature has to exist. And what makes sure the feature exists are the tests. So oftentimes, the code for the test will outlive the code for the feature. Somebody wants to change things around, they want to change the color on something, that sort of thing. And testers automatically run these things along with any sort of deployment steps and also sort of any sort of CICD platform. And they don't need to know the internals. They don't need to quite understand, all right, this is why XXE works. They just need to know that's a bad thing. And the few testers that you find that are really interested, like you always should share the information, but sometimes they don't care. They're just trying to do their jobs. The ones that are really interested, you can share the information and find an amazing ally. And because of that ally, and this is a big deal, they're on the same side as us. Developers are always trying to ship stuff out the door as soon as possible, right? Testers are always trying to break it. We're also trying to always break it. So if you can leverage the already existing organizational work and already existing code that testers already have, voila, you can get a lot more done with a lot less work. And basically, it ends up with this. You can push left. And this is how you do it. So for those who are like, all right, this is nice and I don't really know what to do with this, this is probably the easiest way to get someone started, right? You make them a string generator function. What I want to point out here is don't try to hammer in some framework they aren't already using, right? This is why I'm not, I could give out sample code and all this stuff, I do have some stuff online in GitHub to show you, but everyone uses a different thing, right? There's test frameworks, I think there's one for Ruby called Gherkin, all sorts of .NET or Java or whatever. Everyone uses a different thing. Find the one that is already being used at wherever you're trying to work at or work with, and work with them and go with that flow, right? Use that to key off of to make any input. So user names, titles for things, that sort of thing. And then you can help them with the error messages because if they come to you, right, or if you don't help them, they're going to say, oh, you just broke our test and wow, you're mean, we don't like you, right? And that's bad. And also, they might not know how to help the developers fix the test, right? So this is how you can get that organizational change right there. Which goes back to my main point, right? The leverage you can get is the already existing test frameworks, organizational work, teams, code, tests that already exist. And then this, you can make those major changes to make anything you work on more secure. That's the main thing. And because my Twitter name is Hackimedes, I'll just say this. Give me a place to stand, I will hack the planet. Any questions? All right then. Anyone want to see another demo? Cool. I have a few. I have no guarantee that they'll work. So this is the one about the external scripts. And it will be like as if someone placed in a capture the flag game with this one. So you're going to see the same boring test that you saw before. Except it's not going to be checking, by the way, you can have like these check for multiple things at once, but for demonstration purposes, this helps out a lot. It's just going to do the same thing, type in a note, send it to someone else, right? And then a share step. It's going to show a very clear to see error. But it's going to say, all right, something loaded this bad script. And if you were to look in the code, right, and I want to show like why SAC analysis would have a rough time finding this. Let's see. Source views. So this is a sharing template that didn't load, right? But essentially it's not even loading, for some reason the developer here didn't even like write this in the HTML, right? What they did is they dynamically created through script. So if you're like looking for in the HTML script source, whatever, you're going to have a hard time. But this looking for the performance logs, for really, really nasty code like this. I have one more demo. And this one is going to be required to understand what's going on. So what I did is I took the AuthZ test, well, sorry, the API test, that just say, all right, go and create a resource, you know, modify it, send it to someone else, right? And see what happens, right? So the sharing aspect. And that is very clear. So kind of weird. We see these assertions, they fail. And then if we go into Burp, right, we actually have sort of a very basic CRUD application, right? Create, read, update, delete, right? And, you know, create an account, get the login token, you know, have all that created. But then we have the attacker, or sorry, the victim go in and make a note, right? But then one interesting thing is sort of, we have this step here. And from what I showed you earlier, you can have Eve sort of attempt to share the resource with themselves, right? Eve have it right here. So Eve shouldn't be able to do this. It isn't particularly clear that it's the attacker, but it is. And, you know, you get a response. All right, that worked. That shouldn't happen. And then, interesting, right? But then, all right, you create Bob, right? But as it turns out, we go in, and Eve ends up getting the note, even though it's sort of hard to see. Sorry. So, sorry, this one's a bit hard to see, but essentially what it's doing is it's testing for the authorization controls on not just the reads, but also the shares. Because sometimes you can find an application where you are not able to create a resource as someone else, but you can modify the resource, right, without being able to read it, that sort of thing. This is what this kind of testing can find, probably should have rethought of how to show it. Cool. All right, thank you so much. And thank you.