 I'm the CTO over at Immunio. I wanted to welcome everyone to Kansas City. I hope everyone's having a good time. It's the third day in the conference, so I'm going to try to keep references to we're not in Kansas anymore and the Wizard of Oz to a minimum. You've probably heard them all. RailsConf has always been a good place for us at Immunio. It's actually, if you remember us here last year, we had a booth in the Expo Hall. We actually launched our company at RailsConf. So our first supported platform and our product is Rails. And yeah, we love you guys. So thanks a lot for that. Speaking of launches, I want to sort of divert at the start here and say, did anyone stay up late last night and watch the launch SpaceX guys? Let's give them a round of applause. That was awesome. This actually isn't a talk on REST APIs. By the way, I used to talk on teamwork, collaboration, closeness. How many of you guys did any of these things today? How many of you did all of these things today? I think I did everything except to have a booth in the Uber E-Tier before, so it's... The last time I did this presentation, I sped through. I sped talk and we got there eight hours. I'll try to get it done in like seven to six hours for you guys. Sound good? I don't feel like there's, like, half a group is like, yes, maybe you're seven hours. The other half is like, why? Is that coming over our speakers? I don't know. So very quickly about me, the reason I'm doing this workshop, I'm an API fanatic. How many people here have used APIs? How many people have used APIs and don't work? Does anyone know what they're talking about? Is it better than my talk? Or poorly document? Or break? How many people like those APIs? Okay. No, it sucks. It makes my life suck, so... Can everyone hear me when I'm talking? Should I just talk over them? Is that fine with everyone? A way that's future-proof, a way it takes advantage of the best practices and modern technologies. Work with me here, we'll try to make it through and not, I wonder if they can hear me though. Interesting, okay. How many guys did this? All of you probably. How many of you guys maybe added some web banking in there? Maybe you arranged a date for the weekend. Maybe some of you right now with laptops are actually sort of like managing your company's network or touching your production servers. All this is happening online through websites. You guys write websites. Other people are out there writing these websites. This is all happening on the web. This is, everyone knows, 10 years ago, 20 years ago, it's just getting more and more stuff online. Government data, sensitive data, your date profile on Ashley Madison, everything's online these days. And the other people who are online are people like this. And of course, this is exactly what they look like. Or this one you've probably seen a lot. And how do we protect ourselves against people like these? Well, we go on the internet and we download padlock images and stick them on the screen. You guys have all seen these, right? Make your site more secure. Maybe another one, a green one in there, shield. And you know, stop. So hopefully, all of you guys are doing this. And all of these other websites that you're visiting and you're trusting to store your data are also doing these fancy techniques here. And that kind of brings up the question, who is actually protecting all my data on the internet? Probably some of you guys are managing sites that are holding my data. Certainly there are other developers who are developing these sites. Because fundamentally, over the past 20 years, say it's become easier and easier to make a website. We're all here this week because we love Rails. That workshop that we were listening in on, they're learning how to make websites. It's really easy. Rails makes it easy. Other frameworks make it easy. But there's still that question of who's actually protecting our data. Who is protecting that data? Is it people in this room? Probably it should be everyone. And are these people doing some of these things? Checking out our framework defaults, making sure things are up to date, maybe getting their code reviewed for security, pen tests, that sort of thing. Looking for new CVE vulnerabilities. Hopefully you're doing maybe some of these things. Maybe you should be doing all of them. Or maybe you're lucky and you've got a pair of magic. Ruby slippers. Sorry, that's the last one, I promise. Because security's hard. All those things I listed are hard. It takes a lot of time, it takes a lot of work. You guys, generally you're focusing on making your site work, making it do cool things. Building features for your customers. Security, everyone wants to be secure, but it's a hard thing to do well. It's difficult. But it can also be really, really interesting. And I hope some of you are here today to get into some of the interesting parts. So really I'm focusing today on three things. Three types of vulnerable code that might be in your applications that you've written or that you're going to write. This code that you write, your application itself, the view logic, that sort of thing, the controllers, the actual app. This is code that you've written and you're responsible for. There's also code written by other people. You use the Rails framework. Rails runs on Ruby. You probably run a ton of third-party gems to add various functionality to your app, to your app. Other people's code that you're now sort of responsible for their security posture. And then code not written at all. So maybe you should have written some additional functionality to protect you against attackers. But again, you don't always have time. So this code not written is also a gap in your apps. Kind of sounds a little bit like the ghost of Christmas past, the ghost of Christmas yet to come, but whatever. So let's look at an attack type that probably everyone's familiar with, I hope, at least you've heard of it, SQL injection. This is a very simple example up here. You've got an SQL query and you're building it by adding some strings together and putting in some user input there. And then a user comes along, one of those shadowy hacker guys with a mask, and says I'm gonna set my username to this in red. It's red because it's from a hacker, of course. Single quote or one equals one. So if you're not familiar with SQL injection, you might look at that and say, okay, what does that do? But what happens is when your query gets put together, it modifies the intent of your query instead of a where clause with one parameter, name equals username, and it has two conditions, name equals whatever or one equals one. And I don't know if your expert's on math, but one equals one is always true. So that's always gonna return true. So instead of giving you back one record that matches your username, it's now going to give back every record in your database. And this is the type of thing that introduces and allows these massive data breaches that you probably read about on the news, where there's millions of credentials floating around on the internet. Some of them might be yours, some of them might be mine. Now SQL was first sort of discussed publicly in 1998. It's really well understood. This is not a new attack vector. It's not an advanced attack vector. So pretty much this is fixed in every app on the internet. Right? It is? Question? Obviously not. Last year, Talk Talk, a big UK ISP or mobile carrier, 157,000 of its customers have their details stolen. They figured after the breach and all the handling of it that they lost around 100,000 customers and about 60 million pounds, probably about, what's that, $100 million US because of this breach. SQL injection, not advanced anything. Just that SQL injection that was first talked about almost two decades ago. Okay, but they're an exception, right? Right. VTEC, anyone with kids, this one's kind of scary. VTEC makes these little kid laptops for children and the children log in, they've got their name, they can put in their birth date and they got breached VTEC and they lost details on 200,000 children along with almost 5 million of their parents with home addresses, passwords, emails, names, et cetera. SQL injection, two decade old vulnerability. And this one, a bit more fun. Weatherspoon's Pub in the UK, they got hacked. Now this one wasn't awful. They only lost around 650,000 customer details and it was only phone numbers, dates of birth and email addresses. So no damage there. Oh, and beer vouchers, that's the one that really hurts me because these now shadowy hackers are drinking free beer on Weatherspoon's, so maybe they deserve it, I don't know. So this age old attack, almost two decades old, should be well solved. You're thinking probably, well, I'm a Rails Dev, I don't write SQL statements like that, I use Active Record, takes care of a lot of this for me. Well, after the talk, go take a look at this website here. If you haven't seen it before, railssqli.org. It's written by actually Justin Collins who's speaking right after me, so stick around to hear his talk. You would be shocked, I think, at some of the things that in Active Record look like they should be very safe. A function that takes a single parameter, like a column name or something like that in the case of pluck. And, but instead it actually internally really just kind of builds up these strings. So you can do, again, lots of arbitrary type of attacks like SQL injection. So it's still a thing after 20 years. This is code that you're writing. Now what about code that other people are writing? So this is a vulnerability that was announced at the start of this year. CV 2016, means it's 2016. There were, I think, four Rails vulnerabilities announced all in the same day with sort of a collection of things. This is credited to John Pullin from Invisium, company that does Ruby on Rails, consulting, security, they're really, really good guys. He's also got a blog where he describes this in depth. I've linked it on the bottom there. And I'm basically just showing you the highlights of his blog post, but it was reported as a possible information leak vulnerability in Rails. And it really, it's an issue if you have something written like this in your app, the render function, and you're rendering well anything where it's coming from like user-supplied input. So the idea here is I've got render params template so that I can have different templates for in this case, my users. Maybe this is the dashboard template. We're looking at the bottom, so you type slash dashboard and you get the dashboard template. If you do slash details, you'd get the details template. So it saves you some time, right? So if you look at the render documentation, it's not super clear what that first parameter is. Is it just the name of a template that then gets looked up along with all my other templates? Or is it a path to the template file within my app? Or as we'll learn, could it be an absolute path to my file system? And of course I'm talking about it because it's this one. So yeah, that same view, you add the percent encoded, but Etsy password, which is the database on my Unix systems that stores all of my user details, and dumps it right out. That's all the contents in there. Okay, so maybe an exception. What about something more Rails specific? What can you do with that sort of secret down there for session initialization? You can do a lot of stuff. You can basically forge any session data in Ruby because you can sign your own cookies. So basically, through this vulnerability, we can read any file on the file system that your web server would normally have access to. They call it a information disclosure, arbitrary file disclosure, or sometimes directory traversal, where we're sort of using these dot dot slash or absolute paths to get into things. And yeah, you can read any file on the system. SSL private keys? Probably something you don't wanna lose. What else do we have here? Secret tokens, secrets.yaml. These things on the bottom, not everyone's aware. Sometimes you put sensitive data into your app through environment variables so you don't need a file on your file system with data. Well, if you can read these proc file systems, you can actually read those environment variables just like a file. So you're not really safe no matter how you do it. If you've got this vulnerability, you're in trouble. Yeah. Okay, so bad. It's really bad, this vulnerability. If you happen to have code like that in your app, and I just did a quick search on GitHub for render params, give it a try sometime, there's something like 50,000 matches. It's not a rare occurrence, this thing, this stuff happens, but it gets worse. So there was another vulnerability in 2014, another CVE 0130, and Jeff Jarmak, I think this is his name, pronunciation, from Matasano, he wrote up an article and kind of dug into this, and it was a similar concept. Basically there was a way that you could convince Rails to load an arbitrary file on the file system. And he said, well, can I take this further and instead of just reading a file, can I actually execute arbitrary code? And of course, you can. And it really comes down to a helpful default behavior in Rails where you can, basically if you load any file with render and it doesn't recognize the extension, it defaults to treating it as an ERB template. So in the example before, when we were Etsy password, what we didn't really see is it was actually rendering that as an ERB template because it didn't know the extension. So for Etsy password is not a big deal, there's no ERB code in there. But if we can put ERB code into a file, like something simple like that, and just using the backtake operators, which executes shell commands, we can get that into a file on the computer and then get Rails to read from it. Then we can execute any arbitrary shell commands on that server that we want to. Who am I tells us the current user, but you can imagine we could put far worse things there in red. And of course again it's red because it's from one of those shadowy hackers. So the basics is we write code into a file and then we ask Rails to execute it for us. So how do we do that? How do we get code into a file in Rails? Well, thankfully that's built into Rails because every request gets logged to your log file, production.log. And I don't know if you guys have looked in it, but by default it's showing all your query parameters. So, great. My code, one, two, three, four. We could change that to something a bit more sneaky. Percent in code, there's our ERB data that we want to get in the file. Make a request and of course it dutifully gets written to production.log. So now if we put this exploit together, the first half, the new vulnerability plus this half the old exploit from 2014, we get the same idea. We get a single request. You feed in arbitrary shell code, send it to a vulnerable server and it executes. Yikes. So, okay, that's a vulnerability. What does that let an attacker do? So this is something that's kind of a bit new over the past maybe year, year and a half. It used to be all this stuff. If you could get files from a file system, you were trying to steal data, personal information. If you could execute arbitrary code, maybe you're trying to steal data as well or use the server to launch further attacks elsewhere or to further explore your infrastructure. But something new that started to come out is this idea of website ransomware. Now you've probably heard of ransomware, the normal kind. This is software that infects your computer and in the background, it goes around, looks at all your files and instead of deleting them like maybe malware used to do in the old days, it encrypts the files with a special key. And it's a key that you don't have. So now when you go onto your computer and try to use it, you can't access any of your files. And it pops up a helpful message that says, pay us $100 in Bitcoin or something and I'll decrypt your files for you. And it's really effective because the price is generally pretty low and people want their files back so they pay. So what started happening end of 2015 last year is the techniques started to evolve to websites. So primarily in WordPress type scenarios where it's the Wild West in terms of security but it's also affecting other apps as well. The ransomware, excuse me, gets in, executes arbitrary code, downloads a payload that encrypts all the files on your website. Maybe even the data in your database. And when you or your customers view your site, you get something kind of like this and it says if you want your website back, pay us the ransom. And like before, generally people want to pay. This is actually affecting hospitals in other types of server malware. It's a big deal, so just be aware that this isn't just people coming in and stealing data from servers. It's not just your customer's data, it's also potentially your revenue and your livelihood, your website. And now the other type of attack I'm gonna talk about is something called credential stuffing or that's what some people call it. And it's similar, basically the idea is some other site gets hacked. Maybe through some technique we just talked about or some other breach and the attacker steals a massive list of user names and passwords. Now chances are, and I think there was a study that said it's around 50% of user accounts are using shared passwords. So the password on your site might be the same as the password in the dump that that hacker just downloaded. And they wanna know which ones match your site. So what they're going to do is they're going to take that big dump, maybe it's a million, maybe it's 100,000 username passwords. And they're going to write a script that will go to your website and try to log in with those details. And when they find them, they know those accounts are active and vulnerable on your site. And if they fail, it's just a failed login, they just move to the next one. Excuse me. So I always like talking about this case because it's a bit interesting in terms of blame. Who's fault is it? Some other website got hacked. You might not be vulnerable at all to these SQL injections or arbitrary code execution. It was somebody else that got hacked. Their website got breached. Those are their user accounts that have been dumped. And it's kind of the faults of the users maybe, like maybe they shouldn't be reusing passwords. It's kind of their fault. They didn't choose strong passwords. Maybe they're that one, two, three, or five, six up there. But fundamentally, they're your users. The ones who these attackers are breaking into, these are your user accounts. So it kind of falls to you guys to protect your users as you're developing sites. How do you protect them? And this isn't just a small problem. It's not just common passwords like this. These are massive breaches with millions and millions of leaks. There was actually one just a few days ago. I really just pasted this for the headline here. This was a site that would actually help you find out if you were in one of these dumps. And then they got hacked. And now there's a consolidated list of what did they say? 866 million user credentials. So this site has hopefully consolidated them and organized them for whoever got them. So just be aware that this is not a problem that's being solved or going away. This was, was it two, three days ago? There's now a new, almost 900 million user names and passwords out there that might be coming to your site any day. And okay, maybe there's a bunch of passwords from this list that match users on your sites. And, you know, is it your problem or is it the user's problem? And maybe you could justify saying, well, we don't really mind if our user accounts get breached a little bit. There's not maybe a lot of sensitive data there or, you know, those users aren't admin. So maybe it's not that big a deal. So I wanted to talk to, through one use case or one attacker use case, I should say, where attackers are using these types of account takeover attacks to directly monetize the result. And it's something called warranty fraud. And this happened to Fitbit a lot in 2015. The basic idea was they used these techniques, the hackers, they got a bunch of accounts. And the first thing they did when they found an account that worked is they went in, they changed their email, they changed the phone number and they changed the address to the point at the attacker. And then they called up Fitbit and said, my Fitbit broke, can you send me a new one? And Fitbit wants to keep their customers happy. So they send out the Fitbit before they receive the broken one. So now the hackers are getting all these Fitbits. They turn around, sell them for half price or cheap on eBay or something. And they've now turned your vulnerable accounts into cash for them, which means they're highly motivated to do this. So there's always a way to monetize these things. So how do we protect ourselves against these attacks? Educating developers, that's probably why you're all here today. You're learning about security, learning more. There's great resources out there. The OWASP top 10 is sort of a quick intro to a lot of this stuff. It's the top 10 vulnerabilities that cause most of these breaches. And it kind of helps people stay up to date. They can see, they update the list every year or most years. So new attack types will bubble up as they become more prevalent and you can focus and learn more about them. There's static analysis. Actually the same guy, Justin Collins, writes Breakman and Breakman Pro. It will scan your Ruby app and tell you if you've got vulnerabilities. It detects the one I talk to right now. And it also detects the one from 2014. So use these tools to scan your code to try to stop these things before they get released. And imagine earlier the manual code review or maybe pen testing. Get other people to try to break your code. It's a good way to get a second opinion, to understand maybe things that you don't know about that someone else does. The big thing I don't like about this list is it's the same list I would be sending out 10 years ago if I gave this talk. Nothing's really changed here. There's no new defensive techniques that are helping people these days. So this is still good. You should still do all these things and they help you produce a secure app. But then once you deploy it, how do you kind of protect it against stuff happening in real time or against new threats that maybe you don't know about? There's a few tools that already exist for this kind of active defense. There's a thing called a web application firewall. You may have heard of it. It kind of became very popular when it became sort of a mandatory requirement for PCI compliance for handling credit cards. But basically it's a box that sits in front of your app and you can kind of see, sorry, it's really small, but you go through every single URL in your app and you tell it all the parameters that you expect to see. You tell it whether they're mandatory or not, how long they should be, what sort of characters you should expect. And you can kind of get from this that it's hard to configure. And the problem, the big problem is it's really easy to bypass because this is a box sitting in your network in front of your app. It doesn't really know what's going on. It's just applying these rules or signatures. And there are actual books about how to bypass web application firewalls. So it doesn't give us a lot of additional security. There's also kind of a problem with how they're deployed really. So you've got your apps down there on the servers. You've got the internet cloud classic over there. You've got the yellow line, bad stuff being rejected. You've got your web application firewall protecting your app. And that's all great if your deployment looks like this. But more and more these days, our deployments look a bit more like this. It's the app is the thing that you deploy. It's not the infrastructure around it so much anymore. You write your app, you push it to Heroku or maybe Docker if you're rolling something your own. And you want security to go with the app. And that's where we kind of get into one of the defenses that I'm gonna talk about today is this concept of rasp. Never been a big fan of the acronym, but it stands for runtime application self-protection. The idea that it's part of your app. It gives your application the ability to protect itself in real time. And it's kind of shifts the story a little bit. It's inside the application now. So we kind of get a bit more visibility on what's going on. So if we go back to our attack there, what was the actual exploit that happened in that CVE vulnerability? The vulnerability was that we could convince Rails to load an arbitrary file. But the actual exploit, the malicious intent, what they're doing is they're trying to read files that aren't normally read. Or they're trying to execute shell commands that aren't normally executed. So can we hook into that and instead try to stop that? Move that logic inside the app. And you can see these things happening directly. We're not sitting outside the box trying to guess what might happen when it hits the app. Here's some examples, what you can do. I mean, you shouldn't be reading anything from Etsy Password really. You should only be writing to Virolog and temp. It shouldn't be writing some arbitrary places on your file system. Your app should not be reading your SSH private keys. Shell code execution. Most apps don't actually need to execute system commands. Or if they do, it's usually in just a few spots. Like maybe you need to compile some CSS when your app first starts up. Or maybe you need to shell out to the command line to generate a PDF invoice or something like that. But generally you're not executing shell commands. You're definitely not downloading Perl scripts from Russian IP addresses and executing them. Like you should not be doing that in your app, right? So can we detect these things happening and stop that? So instead of tracing all the individual and trying to chase the vulnerabilities as they're announced, stop the malicious behavior. Same idea applies to those other attack types, SQL injection. We can see the whole SQL query. Can we stop that at the source? So instead of trying to guess at the bad input, look at the full query before it gets sent to the database. Stop that. Same for templates. If we're rendering the template, can we look for cross-site scripting then? Authentication, failed logins. Inside the app we get visibility of all that stuff. And as well, the application, it knows itself, right? It knows which lines of code are running these things. It knows which lines of code are running SQL statements. And if you get an SQL statement that looks abnormal, it knows where that came from in your code so it can work with the development team a lot more closely and help them understand where these vulnerabilities are coming from. This is sort of the general idea of how a rasp can work. This is not all of them work like that, but that's the idea, that you've got your app in blue. And then you add the security logic and it gets a chance to hook into all the various parts of your application. So it can protect against header-based stuff, it can protect against SQL queries, it can protect against authentication failures. And what that lets us do is if we take our old example of the SQL injection and we look at some possible inputs, first one is a good query. Most of the queries from that line of code will look like that. So we can learn that structure. Your where clause should have one condition. And then when the bad input comes in and all of a sudden that line of code is executing a query that has two conditions in the where clause, flag that up as an attack. And of course we know it's an attack because it's red and the other one's green. I actually had somebody tell me, why don't you just stop the red ones instead of all this logic, but that's the idea. And same for things like the authentication. Instead of just blocking failed login attempts and allowing successful ones, we can start to look at the rate of failed logins. Most of your users, if they forget their password, if they're like me, maybe they might have to try, four or five times before they get it right. But they shouldn't be trying a hundred times. And if they cross these thresholds, you should take action to stop them. You can do something like automatically serve a captcha or just block them, whatever you like. So back to these three types of vulnerabilities. Talk through a few examples. These are all, like this provides tools for you to address some of these issues because everybody's writing code, we're all using external code and nobody has time to write all the code that possibly they should be writing. If you can land a rocket on a barge in the middle of the Atlantic Ocean, surely we can make it easier to make websites more secure. Leaving with that, thank you very much.