 So without much ado, Chris Hanlon, everyone. I'm here to talk about Swiss cheese, holes in the foundation of modern security. This has been submitted to CERT under VU-919801. It's basically how I intercepted password reset tokens or requested the tokens, bypassed email encryption, intercepted the tokens, and exfiltrated confidential data for my own test accounts. If anyone wants to try this at home, I don't want you getting arrested, so please do it on your own test accounts. Before I get into the details of the attack, I'd like to cover a little bit about myself of being configuring and protecting desktops and servers since 1998. I've been self-employed as an IT consultant since 2004, and I have a few accepted vulnerabilities. Started with the Linux kernel vomph, then some proprietary software voms, and then a couple Google Apps voms. The second Google Apps vome got me into the Google Security Hall of Fame. The vulnerabilities that we're talking about today were reported to the international computer emergency response team, and there's actually five separate weaknesses involved. They're often referred to as CWEs or common weakness enumerations. Number 287, improper authentication. So the person sending the password reset token doesn't authenticate who's getting it. 291, reliance on an IP address for authentication. Is there anyone here who builds apps? Do any of you rely on IP addresses for authentication? But most of the systems reset passwords are effectively relying on IP addresses to authenticate who they're sending the token to. Missing encryption on sensitive data, in many cases it's both usernames and password reset tokens that are being sent without encryption, and both of those are sensitive. Most people don't want that broadcast over the internet unencrypted. Clear text transmission, I don't know why that's a separate CWE, but they've decided to separate it out, and weak password recovery mechanisms. I would say that sending a password reset token over an unencrypted connection is quite weak. So all of these tie into use of email to reset passwords. RFC3207 is the foundation of modern email security. The first draft was released in 1998, and then it became an official RFC in 2002. It has quite a few options on how the sending server implements TLS. They get to choose how picky they are about sending emails. In many cases, vendors use separate email servers for sending out reset links than their day-to-day email, but none of those seem to be configured. I tested over 30, none of them were configured to require TLS. So this isn't new that email can be intercepted. You've probably been hearing for as long as you can remember not to send credit card information via email, and researchers at Google back in 2015 covered four common ways that could be used to intercept emails. Strip TLS attacks are the most basic. It's a machine en route to the mail server that sends back an error saying, sorry, the server doesn't support encryption. And in every situation that I tested, the server sent the email anyway. They also typically don't check the security certificate. So I used, I tried testing again using the certificate for clonordivorceler.com instead of pandosi.ca. So they have absolutely nothing in common, but it happily delivered the emails to the server with the wrong certificate. The other two are DNS based, and they might look like the same thing. One of them is spoofing an MX record, and the other one's spoofing an A record. The big difference is that often the MX record will be through a different provider than the A record. So if anyone uses Office 365, it's your own name server for your domain that says send it to outlook.office365.com, whereas it's Microsoft's DNS server that says what IP address corresponds to that name. This is a real world problem. It's not just theoretical. Going back to 2014, we saw a US wireless provider where every single TLS connection was having TLS stripped. Similarly, we saw two large ISPs in Thailand around the same time. It's believed that both ISPs were compromised and that someone was primarily using this to steal credentials. That they, because people were connecting to their home SMTP servers, they would send a doesn't support encryption and hope to get the person's password. This is continuing today. We saw earlier this year, the AWS man in the middle attack that hijacked DNS and redirected my Ether wallet to a phishing site. This sold about $20 million of cryptocurrency in about four hours. And one of the other researchers at this conference, Chris Kubekka, her research showed about over 11,000 routers on the internet that were stripping TLS. Some of those were believed to be for espionage purposes, but there's a lot of other reasons why it happens. And that includes over 3,000 routers stripping TLS here in the USA. So one of the major factors that came up in Google's research was that many routers disable encrypted email for the purpose of scanning emails for malicious content. And one of the challenges with that is that a lot of the firewalls that you can use to scan emails, don't warn the users. When you set it up this way, it's going to turn off email encryption and all the email coming into your domain or all of your customers in the case of an ISP is going to end up coming through unencrypted. I believe that, all right, there's Chris Kubekka's work showed evidence of government espionage being a part of it. We see interception of confidential data because many people trust their email to send financial emails. And there's the password reset attacks. I expect that one of the best phishing attacks you could do would be to take a legitimate email and then modify the content to send someone somewhere else. The person is expecting a money transfer from you and you redirect them to a different server that looks just like where they're supposed to go. I haven't seen those in practice, but the weaknesses of how emails being handled definitely leave room for that to happen. So if you wanted to do this, you would need control over a router, a switch, a DNS server, or a mail server out on the internet. You need some kind of tool to intercept the emails. I believe, I'm gonna try adjusting my mic a little. I'm wondering if that, oh, that's worse. It's fine, okay, let's keep going. I think most attackers would probably use scriptable SMTP proxies. Those give them a lot of flexibility. They can pass through some of the emails, but not others. So typically, they'd wanna pass through all your normal emails to your mail server, but they wouldn't wanna pass through the emails where they had requested it and they would use filters in that script to do that. Those tools are quite commonly used by red teamers, the scriptable proxies. So if you know any red teamers, they can probably give you a more detailed look on that one. Another factor is that the term man in the middle can be a little confusing. First of all, it doesn't need to be carried out by a man, but secondly, you don't actually have to be in the middle. If you look over at the graph, the traditional route from Cisco servers that are actually in AWS to the Tuscany hotel where I'm staying goes all the way to Atlanta, then to San Diego, then through Los Angeles and here to Vegas. And most people don't consider Atlanta in the middle of Seattle and Vegas. Fortunately, I'm not running a mail server in the Tuscany hotel, but when you hear about the BGP redirects, like what happened with the Amazon attack, it's basically somewhat a router that successfully sends out a message, send that traffic here through Idaho or Oregon, it's gonna be faster, it's gonna be cheaper, and you'll get there the same way. And sometimes they're able to convince sending servers to send the traffic over a different route, and then that's part of how they do their attacks. So before I set everything up, I did a quick check to make a mail server was working properly. I wanted to make sure that there was valid TLS, had a valid certificate, and then it met modern security standards. If you're wanting to check your servers, you can do the same thing at checktls.com. So my upstream router was a Ubuntu Linux virtual machine running Ubuntu Linux 16.4, long-term stable. Took very little to get it to intercept emails that were passing through to machines after it. We installed Exum with one command, edited the config file, changed three lines in the config file. One of them was to have it run as an internet mail server. Basically means accept incoming email because the default configuration only accepts mail locally. To accept all host names and to listen on all interfaces, which if you were doing any kind of spying, you'd probably wanna listen on all interfaces and accept all host names. We then activate the config file and we use ifconfig to take over the destination IP address and restart Exum. Before I requested any of the password reset links, I checked to make sure that it really was being intercepted and this seems like a dead giveaway that we've got no TLS, no certificate and the vendor could have logged at the time of signup or at the time of a previous email. Hey, this server supports TLS but they still seem to ignore that and keep sending the links. So we got into a lot using this approach. Probably the most interesting one is Cisco Meraki firewalls and switches because once you get into that control panel, you can change their VLAN rules, you can forward ports in from the outside and it greatly expands your attack surface. The next interesting one was Google Nest security cameras. When we got in there, we were able to remotely turn on the microphone. We were also able to download the last 30 days of recordings and the list went on and on and on. Windows desktops now recommend you use a Microsoft account. They make it a little hard to not use a Microsoft account and even once you've set up the account as a local account they'll prompt you over and over again to use these Microsoft accounts. They let you, if you can man in the middle of the email, you can reset the password remotely. Even if the machine's BitLocker encrypted you can then log in whereas most other password reset attacks you can't log in to the machine. You can't use like a bootable password reset if you have BitLocker encrypted. It got worse because the machine started prompting for people to upload their BitLocker key to OneDrive. So even if the machine was totally dead you could remotely download the BitLocker key and then decrypt their volumes. Through that same Microsoft account we got into their Skype message history. We were able to impersonate them and send messages to their contacts, get into their Hotmail, their OneDrive. The same type of approach worked on Dropbox, Amazon Web Services, Oracle Cloud, my own online medical records. So I'd gone to an online medical clinic that referred me out to a specialist. I had someone else set the password and then I reset it. I was able to log in and check my sleep study. For me it's not a big deal. I mean anyone breaks in they can find out that I snore pretty loudly and that I believe it was about 30 times an hour I was struggling to breathe in my sleep which isn't very good but I think you can imagine more sensitive information. Many people if they had an STD or something they wouldn't want that out in public and it's something where there's a history of people being blackmailed over their medical records. So it needs to meet a higher standard. Similarly we were able to get into LogMeIn and the list, oh we got into three major online backup providers and those ones were awesome because once you get into the online backup provider you're able to download their password lists because multiple web browsers and email clients stored the passwords in a place that was being backed up. You download that and then you have a couple options. A, if you're only man in the middle in certain connections to their mail server you can get into all their email and get it even more of their password reset links. The next trick is you get into password reuse attacks. How many other sites are those passwords used on? So there's a guy at Microsoft who came up with the quote, attackers think in graphs, defenders think in lists. And when we look at it, once you are able to reset someone's password you can get into other systems and it just expands and expands and expands. This is a small version of the graph. The graph would fill up the whole room if we looked at how many places you could get in from man in the middle in a single mail server. Then, even more fun. I left the interception server online and when it was online I wasn't expecting this but I kept getting emails from the vendors. So even if you didn't know which domains were on the other side of the router you had broken into you'd start getting these emails telling you about accounts and domains that email was going to. And as those came in, especially if you were using one of those scriptable proxies you would be able to reset those passwords as you found out they existed. We had some fun ones like LinkedIn. LinkedIn encouraged my test account to add me as a contact probably cause we were on the same IP range. Similarly, Adam Bennett works with me. So I'm guessing that's also tied to the IP range and I don't know who the third person is they recommended but it's someone in our area who is in the same industry. A single compromised host shouldn't expose services on hundreds of systems. And that's a big problem right now because it doesn't matter if it's an admin's email account or a router that the client does not control cause most people only control at most one upstream router from them. So there were three services that we can take some inspiration from. They blocked the attacks at least to some extent. Apple right away as soon as you try and reset a password it puts a push notification on the person's phone. If their phone is offline, you can, if the phone's offline and you can't get it back online quickly Apple will give you an option to send you an email at a random time within the next 24 hours. And that approach is popular because it means even if you borrowed someone's phone you couldn't reset their password which is a little bit of a risk. You ask someone, hey can I use your phone to take a picture and if you can keep their phone a little longer you can start getting into their other accounts. Same with a single compromised computer. If someone's desktop is compromised and it accesses an email account that links go to you can request those links and get at them via the computer. Azure had some great approaches. Well, a very simple one. They have mandatory two factor authentication. If you try to reset a password on an account that doesn't have two factor you will have, it just tells you sorry you need a second recovery technique. They don't even tell you contact support you're just out of luck. But they do push very hard for adding that second recovery technique. Gmail had good security questions but only when you were logging in from a new machine on a new network. So at first it looked like it was really easy to get into Gmail but as soon as you go to a different network and a different machine they ask you when you created the account which if you're off by a little it gives you an option to talk to support. But you also have to say the last password you remember. So if, I don't know how many passwords they keep back but I'm thinking it's in the neighborhood of your last 10 passwords. So even it's very hard to reset these ones. There are some dumped passwords so I'd be curious how well people can do it getting past this but it really slows people down and I'd like to see more providers use one or all of these approaches. Now log me in was a really scary one because people think they're setting up two factor authentication that goes to their phone. But down at the bottom if you don't get the email or if you don't get the two factor via the phone you can just click get the code via email which means it's not really two factor authentication it's more a one factor authentication because you can reset the password via email and you can get the token via email. So if you're a developer I hope that you're tracking what's going on with password reset attempts if you have a bunch of reset attempts in the same IP address or IP range it'd be suspicious. If you have a bump in the number of password reset attempts it's worth responding to. All resets should have some form of verification question or a second form of authentication and the secure email standard has been out for 17 years. It's time to require TLS especially on sensitive emails like password reset links. That seemed to go a lot faster than I expected. Does anyone have any questions? Yeah. Can you talk a little bit more about how you actually experimented in that portion? I know you said you used to have a virtual host. So I contacted quite a few virtual machine vendors and one of them was willing to configure it so I'd have one router or one server that was upstream from the others. Linux has built-in routing functionality so I just turned that on and there's kind of a switch on the second interface. A virtual switch. So it's just a simulated attack. Yeah. Yeah. Yeah. So Meraki, I'm not aware of any Meraki devices that use iOS. Okay, the Meraki, we got into the Meraki dashboard. At which point you control it through the web. I didn't check, but I got into the dashboard. The dashboard had multiple routers and it multiple switches. Yes, because that's the purpose of the dashboard. What was that? Okay. Are you with Cisco? Okay. Then I'm happy to talk. Are there any other questions? It doesn't look like there's any other questions. Oh, shoot. There was, yep. And it looks like I had my slides a little bit out of order and that's why it was finished a little sooner than I expected. So we've got what you can do to protect yourself. So the EFF, after I submitted they added a option to register your domain with them as one that requires TLS. If you register right now, I don't think it helps all that much, but as more and more vendors start looking to that database they'll be able to see, hey, this is a domain that's supposed to have TLS. Why isn't it getting here? I would say, just like the CIS top 20 talks about inventory in your software, I think it's a good idea to inventory your service providers. Keep track of which ones have confidential information on them and what practices they're using to reset passwords. There's probably a few people here who use single sign-on services where everything ties into their active directory or another service. When you do that, as long as your active directory doesn't support these resets you're making your system a lot more secure. And I don't think, I don't think that's been advertised right now by the vendors that support the single sign-on but I think it's a huge benefit at this time. And if you're requiring two factor, I've even seen some of the physical token two factors that will downgrade to email. So it's really important to know whether both the email and the token or both the password and the token can be accessed via email. I am looking for new projects. I'm open to projects as short as a day or as long as 40 years. Thank you.