 This is Dr. Jeremy Galula. He's the Tech Projects Director of the Electronic Frontier Foundation. He's going to talk with us about kind of the cool encryption internet kind of thingies. So I'm going to turn it over to him. Let's give him a great big round of applause. Thanks very much. So yes, apologies if you were looking forward to seeing Shahid Bitar. He is elsewhere at DEF CON, but couldn't be here to give this particular talk. So I'm not going to be talking about NSA spying. I am going to be talking, however, about NSA spying, but just to have a totally different angle. So first of all, thank you everyone for showing up here. It's bright and early this morning. I'm amazed that there are more than like two of you in the audience at 10 a.m. on Saturday. So I will try not to put you back to sleep. So most people see the EFF like this, like a bunch of lawyers. So these are a team of lawyers. They're outside the United States Court of Appeals. And it's true. We have a lot of lawyers. They're awesome. But EFF also does a lot of other awesome stuff. Obviously, with lawyers, we petition the government. This is some comments we wrote to the FCC during the whole net neutrality debacle. This was, we actually put together comments from internet engineers, pioneers, and technologists about the technical flaws in the FCC's proposal. And so we bring technologists together. Obviously, we also do a lot of grassroots support, rallying grassroots support. Some of you might have been at some of these protests. But nearest and dearest to my heart as tech projects director is that we actually write code. We have a bunch of technologists on staff at EFF who not only advise the lawyers, we also write code. We create software that people can use to enhance their privacy and security. And I'm going to be talking a little bit about that sort of stuff today. So some of our software projects some of you may or may not have heard of. There's HBS Everywhere is probably our most well-known. There's also a browser extension you can install to enhance your privacy, or it's sorry, enhance your security online. I'll be talking a little bit more about that one later. There's also Privacy Badger. That's another browser extension. That's a privacy focus as opposed to security focus. And then CERTBOT, which some of you may have even used to get yourself a certificate. And I'll be talking a little bit about these two today. So EFF has these three different theories of change. We've got lawyers, activists, and technologists. So we get this wonderful sweet spot of being able to pressure the world to make the world a better place from three different angles. So let's take a trip down memory lane. And this is sort of how all of this sort of really tech-focused part of EFF started. So back in 2006, this is way pre-snowed in. A technician from AT&T literally walked in EFF's front door one day and said, there's some shady stuff going on at AT&T between the AT&T and the NSA. And I think you ought to know about it. And so this was a guy named Mark Klein, who was an AT&T technician. And he came to EFF and told us about, essentially, how AT&T was letting the NSA tap the internet backbone. There was this secret room in an AT&T facility in San Francisco. And this was, again, long before snowed in. And so we said, all right, we've got to do something about this. This is warrantless mass surveillance, not cool. Let's try and fix this. So we tried the courts. Our first lawsuit was hepting versus AT&T. Hepting was an AT&T customer we found who we had reason to believe that their communications had been sucked up by the NSA via AT&T. Does anyone know how this one turned out? So the fun fact is it disappeared because Congress, as soon as AT&T got sued, AT&T ran to Congress and said, Congress, you have to protect us. We were doing this because the DOJ told us to. We were doing this because the NSA told us to. How could we have possibly known that warrantless wiretapping was illegal and we should have challenged it? And so Congress passed a law granting all of the telcos retroactive immunity. And so Congress basically said, AT&T, you've got to get out of jail free pass. And it goes backward in time. So that went away. So then we're like, fine. AT&T, we can't sue you, so we'll sue the NSA. So Jewel versus NSA. Jewel was another customer whose, again, communications we had reason to believe had been sucked up by the NSA. Fun fact, Carolyn Jewel is actually a romance author who lives in Northern California. She's really awesome. If you're into romance fiction, you should check out her books. Anyway, so does anyone know how this one turned out? Didn't disappear. Good guess. Excellent guess. She did not disappear. She's still around. And in fact, so is the damn case. It's been 10 years and the wheels of justice grind inexorably slowly. It is still ongoing just because legal lawsuits take forever and they take especially longer. They take a much longer infinity if you're suing the federal government. And so this is still ongoing. We still hope that we will succeed at some point, but it's just taking forever. And so from our perspective, we're like, well, we don't want to wait for infinity time to get a resolution. We think we can do something a little better. So we tried some activism. How many of you have seen this before? A couple. So then I'm happy to show it to you. So this was a blimp that we flew in partnership with Greenpeace, I think the First Amendment Coalition. There might have been a couple other groups. I apologize for not remembering all of them. And we flew it over the NSA's Utah data center. So this is out in the middle of Utah where they've got this giant data center, which is presumably storing all of the data they're sucking up. And we thought people should know that there's some illegal spying going on beneath that. Of course, that didn't really pressure the NSA to stop. As you might imagine, they didn't really care about a blimp flying overhead. So what about technology? As I said, we've got lawyers, we've got activists, we've got technologists. So what is the fundamental problem? To solve it with technology, we have to define what the problem is we're trying to solve. So this is a slide later released via the Snowden leaks, but it shows you the sort of thing that the NSA was sucking up. And in particular, what you're seeing here is the headers from a connection and you notice that it's over HTTP. This whole thing is unencrypted. So the reason the NSA can suck all of this mass surveillance up is because it's unencrypted. If it were encrypted, the NSA would not be able to look at everyone's communications. Even the NSA, as far as I'm concerned, does not have the capacity to break everyone's encryption, assuming the encryption is configured correctly and is up to date, et cetera. They may still be able to target individuals, but that's no longer mass surveillance, and so then that's going to be a different problem. So we said, fine, the solution is to get everyone to use HTTPS. Well, 10 years ago, none of this was HTTPS. It's kind of hard to believe that just 10 years ago, Google, Facebook, Wikipedia, Twitter, none of them were using HTTPS by default. And so we said, well, let's pressure them because what we want to do is get as much traffic on the web encrypted as possible. So let's start with the most popular sites and then we'll work our way down. So IdeaKitty has a bright idea. We can fix this. So we created the HTTPS Everywhere Firefox extension. So what it is, it's a browser extension to Firefox. Now also Chrome, you can also install it in an Opera. I'm pretty sure it's compatible with Brave too, although I don't think you need it in Brave because Brave actually incorporates the HTTPS Everywhere list into the browser itself. And what it does is when a website supports HTTPS, but it is not by default, HTTPS Everywhere will upgrade your connection automatically. You don't even have to remember anything. It will prevent any sort of man in the middle style attacks, any SSL script attacks because it says, if that website is on my list of websites that support HTTPS, then I'm only going to let the browser connect over HTTPS. So we launched that. It was in June of 2010. At launch, it was about 1,000 websites. It's now tens of thousands of websites. It also blocked mixed content before the browsers did. Now I just want to double check and get a sense of the room. How many people here know what mixed content is? Okay, so about half. So basically mixed content is when you load a web page over HTTPS, but then it pulls in other resources over HTTP, so like images or JavaScript, and when it's pulling the other stuff over HTTP, that gives an attacker an opportunity to inject malicious scripts, basically, and bypass the fact that the original web page was loaded over HTTPS. So we did that before the browsers did, which was kind of awesome. And now, of course, blocking mixed content is the default for browsers. So shortly after we launched, this news, well, basically someone else launched another tool, which gave us great motivation and pleasure that we had just launched HTTPS everywhere. How many people remember something called Firesheep? So, okay, so a couple of people. So Firesheep was yet another browser extension, but what it was designed to do was to listen if you were on an unsecured wireless network or if you're just connected to a wired LAN, and it was just broadcasting packets. It would listen to the HTTP connections that other clients, other browsers were making, other computers, and it would say, hey, look, that one looks like a cookie, for a authentication cookie for Facebook. And then it would pull that in, and it would let you log in as the other person. And again, it was taking advantage of the fact that at the time, even login connections were not encrypted. So I could get on Firesheep, I could sit at a coffee shop, and I could basically become a login as any person who logged into Facebook at that coffee shop, because it wasn't over HTTPS. And so, HTTPS everywhere protects against that, which was great. So we were like, look, this is an active thing, this is an attack that can really happen, and we're protecting against it, awesome. So great, wonderful, we've solved the problem, we can go home, the NSA will stop surveilling. Sadly, no, that is not quite how life works. Everything was not awesome. Although we might have been able to protect many sites, not everyone's using HTTPS everywhere for one thing. We've got anywhere from three to 10 million users, depending on how you count. Fun fact, as a privacy organization, counting how many users you have is really difficult, given that we're not collecting that information. But regardless, that is not the entire web. Lots of people aren't using it. And there's still no default HTTPS support. So we said, all right, the real way to protect everyone is to get the actual companies to change. You know, we can put out this browser extension, and smart people can use it, they can tell their friends, they can tell their family, they can tell their neighbors, but it's still not going to reach everybody. So we launched a HTTPS Now campaign, basically pressuring the companies, trying to shame them and say, why the hell aren't you using HTTPS by default? It's not that freaking hard. So we made some progress, and it looks like I don't have all of the fonts on my computer, because those were supposed to be green checkboxes. What happens when you upgrade your operating system and don't check your slides to make sure that you've got the same fonts? But Facebook and Wikipedia still know default HTTPS, which is kind of ridiculous. So then along came Edward Snowden, and I'm pretty sure everyone in here knows who Edward Snowden is. And so back in 2013, we got the Snowden leaks, which told us even more of, frankly, what EFF already knew, that the NSA was wiretapping the backbone of the Internet and sucking up innocent Americans' data without a warrant. Not only that, they were collecting phone records, they had this X-key score program, which they would use to look through the data, and this is not good. This came out, and finally, the whole world started to realize, holy shit, this is really awful. And so we decided shortly after that to really turn up the pressure on encrypting the web report. That would just in plain, I was gonna say black and white, but I guess red and green, show the world this is what the companies are doing. And if the company's getting undetermined or red Xs, that means they're not paying attention to your security. And so we put that out. This is actually obviously not up to date. This was back when we launched it, because obviously today most of this would be supports HTTPS, encrypts data center links, uses HP strict, et cetera. As we're going down memory lane, it's fun for me to note that some of these companies don't exist anymore, but at the time they were important to rank. They were important to put on this chart. So we did that, and we actually made a lot of progress. This pushed a lot of these big companies to adopt HTTPS. So great, we're done again. That was a bit of technology plus activism, plus the original technology. We don't have to wait for the courts. I mean it's good to get that precedent, but in the meantime, we're set. We've now thwarted NSA mass surveillance. Well, unfortunately again the answer is no, our job is not yet done. Although we might have gotten the major websites encrypted, overall the internet was still not encrypted. So this is a chart from, this is actually courtesy of Mozilla, showing the percentage of page loads over a 14-day moving average that are used HTTPS. And you can see if it's tough to read the text in the back, the dark blue line is all users, and then more recently Mozilla broke out USA versus Japan users. USA is the one on top, Japan is the one on the bottom. But as you can see, back in 2013 this is like 30%. This is ridiculous. This is less than a third of page loads are encrypted. As you can also see, the story is going to get better, but we'll get to that. So at the time, back in 2013, TLS was not ubiquitous. You know, you log into a site like Quora, and you know, you can just look at the packets. And I know that's really tiny text, but what that's showing is the username and password. You know, you just get on Wireshark, you sniff it, and obviously the NSA is doing Wireshark on, you know, infinite steroids. So okay, sorry, this is even 2013. This is June 2014. It wasn't just like sites like Quora. It was even the Googles. So, oh man, I am missing all of the fonts. That's supposed to be the table flipping emoji. Man, all right, anyway. In particular to notice here, this is an HTTP page with a link to a sign-in page. And so an attacker could inject and basically redirect this sign-in to wherever they want. Because they could have intercepted this page while it was en route to you, said I want to redirect this to some, you know, URL that's using some sort of homoglyphs and some, you know, shady, it'll be some domain that looks kind of like Google, but it's actually ours. And so we'll get you to click that. You'll go to that domain, and you'll enter your Google credentials. And so even though Google's actual login is over HBS, we're going to trick you into signing in on our page. This is 2015. So why was this such a big deal? Why was this so hard? So the first reason that HBS wasn't ubiquitous was setting up TLS was tedious. Like, this is a page from Dreamhost about how to set up HBS on Dreamhost if you were hosted on their service. And it is just, even for an actual sysadmin, it's just a tedious ton of stupid work. Like, it's just a bunch of manual steps. It's also really hard. There's all this, you know, every time a new exploit or new vulnerability in some part, you know, some Cypher comes out or something like that, some configuration of TLS, unless you're a crypto expert, you're not going to understand this stuff. And not everyone who runs a website is a crypto expert, nor should we expect everyone who runs a website to be a crypto expert. We want to make the web easy to use, both to access, but also to put content up there. So, I mean, there are entire books written on how to set up TLS properly. And if you're like a mom and pop shop, you just want to put up your website so that people can see that you've got something or something, like you don't have to read a fricking book to know how to set up your website securely. So, TLS wasn't ubiquitous because the one thing I also missed is it costs money most of the time. You have to pay a service, a certificate authority to get a certificate that you would then post on your website. You have to pay for that HPF certificate. It was also tedious. It was also very hard. So, this makes me sad. But, what if we told you we could fix every single one of those problems? So, back in, I guess, around 2016, EFF started brainstorming. Well, how can we fix this? Like, this is a fundamental infrastructure problem. And so, the answer is to make the infrastructure better. So, we decided to fix that by creating a, I'm sorry, it wasn't even 2016, it was earlier, it was 2014 when we started thinking about it. We decided to create a certificate authority. So, the certificate authority is the entity that issues certificates, which you can then put on your website, that's how the browser, or that's how the browser verifies the HPF connection. And so, the goal of Let's Encrypt wasn't just to make sure that we could give everyone certificates, but it was to eliminate all of the obstacles in getting HPF set up properly. Whoops, that was wrong button. So, the first problem, TLS is not ubiquitous. The answer, let's automate certificate issuance and make it free, so that everyone can get a certificate and you don't have to follow a bunch of manual steps. So, you can type in one or two commands in the command line and just get your certificate, it will be set up. And the solution to this was we created an open protocol for requesting and getting a certificate called ACME. I can no longer remember what it stands for. It's something certificate management something. But it doesn't matter. The point is, it is a protocol that is open. So, any server, any certificate authority can use this protocol to issue certificates. And in particular, fun fact, Let's Encrypt is no longer the only certificate authority using ACME. I believe, man, I'm forgetting which one. There's another major certificate authority that's now also using ACME. I forget if it's Komodo or the result of Komodo or something else, but the open part of it worked, which is exciting. So, how ACME works. So, first there's an ACME client. Like, this is your web server. So, the client is the thing that wants to get the certificate so that they can put it on a web server. And then there's an ACME server, the certificate authority. And so, a lot of you are probably thinking, well, wait a second, we can't just give out certificates because then I could request the certificate for Google.com and then I could, you know, spoof Google and scoop up everyone's Gmail passwords. And so, the way it works is the ACME client tells the server, I want to cert for example.com and here's the public key I want to put in that certificate. And then the server, again, this is the certificate authority, says, okay, that's great. Here's a nonce. Here's some random string and I want you to post it at example.com slash nonce and sign it with the corresponding private key. And so, what that does is first, that verifies that if I can post it at example.com slash nonce, that pretty well establishes that I have control of that domain and the server that is serving content from that domain. Because what will happen is I'll say, great, it's posted and then the ACME server is going to just try to get that nonce. And if it can get that nonce over, and of course this is, there are a couple of other issues with this. So like if there was DNS hijacking, that would defeat this. So we are relying on DNS. Another thing that we're looking at implementing, and if this is going over, don't worry about it, is doing this from multiple perspectives on the internet so you can't just hijack one route. But basically, if the server can get that nonce, then it knows this machine I was talking to, controls this domain, and also can put stuff on the web, on port 80 for that domain. And so if that's enough, if it can do that, then it can do whatever once on that domain. So that's good enough, it obviously has control. And so the server will get that nonce. Of course the web server will respond with the nonce and the signature. And then now the ACME server knows this web server controls the domain can put content there, controls the public key because it signed that nonce with the private key. And so of course the ACME server can verify that. And so then it says signature checks out. So you control the public key, you control the server, you control the DNS. So here's your certificate. And so now in an automated way, this is all happening over the ACME protocol, you can get a certificate and prove that you control the domain. So that's how the protocol works. How do we actually get that certificate on the other side? So what we do is we created a client or an agent that basically gets it right. We created a piece of software, you can use over the command line, very simple. That will not only use the ACME protocol to talk to the ACME server, get the certificate. But also if you're using Apache or Nginx, it will install the certificate and configure it for you. So you don't even have to think about like, where do I put it, how do I set this up, what cipher suite should I be using. It will use sensible defaults and do it for you. It will, again, it will tweak your existing web server to pass those challenges. So what I just showed you that back and forth was called a challenge. It'll install the cert, it'll tweak the security options. Plus it's going to automate renewal. Certificates expire. In particular, since we're doing this in an automated fashion, what we decided was we wanted a very, a very quick expiration just in case something went wrong. And so let's encrypt certificates expire after 90 days. And nobody is going to always remember to renew their certificate, I guarantee you. So because computers are good at doing things on a schedule, we'll have the computer do it for you. We'll have your server automate the renewal and some of these security response tasks. So what kinds of automation happened? Well, like I said, easy ones, cipher tuning. If you're doing OCSP, OCSP stapling, you could also set up upgrade insecure for content security policies. There are some features where if you want to do redirect from HTTP to HTTPS by default, then you can tell this software, which is called Certbot. You can tell it, I set that up on my server too, so I don't even have to think about it. So install the cert and set up automatic redirects to HTTPS. What we haven't quite gotten to yet is full rewrites. So if you want to actually automatically rewrite all of the content on your web server to use HTTPS instead of HTTP, like if you've got hard coded links, Certbot doesn't quite do that yet. That's a little bit harder. But it does support HSTS, which is a protocol that a web server can use to say, when you connect to me, always use HTTPS. It's kind of like the server's way of saying use HTTPS everywhere, essentially. And then there's even harder stuff, like mixed content, auditing and correction, and HP public key pinning, which we're probably never actually going to implement. But this is a software. It's called Certbot. It'll do all of those things. Not only that, it supports plugins. So if you want it to work on a different web server, you can write a plugin for that. There are different challenges. So the challenge I showed you is just one challenge. There's also other challenges. Say you want a wildcard certificate. So you don't want a certificate just for example.com. You want one for star.example.com. That's a different challenge. You have to actually do the challenge over DNS. And thanks to various plugins, Certbot will automate that for various DNS providers too. It works on most Unix-based systems. And pretty soon it's also going to work on Windows, which we're really excited about. So actually this is not current events. This is, God, eight months ago events. So this is just giving you an example. As some of you probably remember, the US government shut down about eight months ago. And all of a sudden a bunch of websites went down, US government websites, because their certificates weren't renewed. They should have been using Certbot. Obviously that would have been solution one, automating renewal. You might as well automate this task. It's easy. Maybe the other solution is don't shut down the government over stupid budget disputes. But this is not a political talk. EFF is a nonpartisan organization. No, I'm just kidding. Encryption is inherently political. Encryption may not be partisan, but it is political in that it does equalize power. It allows people who normally wouldn't have the power to keep their information, their communication secret secret. And that's part of why EFF does its work. We want to give everyone this ability. So anyway, HDS for everybody. We've got Let's Encrypt. We've got Certbot. We've got security. It's usable. We've got transparency with Let's Encrypt now being part of Certificate Transparency. We're great. We're set, right? And the answer is actually, yeah, we did pretty awesome. So this is a chart of how many fully qualified domains are using the Let's Encrypt Certificate Authority. And so you can see back when we launched, when we actually went into a public beta, and God, this thing's actually a year old, and it's at 140 million certificates. So this blue line is fully qualified domains, and that's 140 million. The dotted line in the middle is certificates active. So there could be multiple domains per certificate, which is why that number is lower. And then this is the number of registered domains that are active. So no matter how you slice it, basically Let's Encrypt is now the largest certificate authority in the world. And so we are basically a fundamental part of encrypting the internet, which I think is pretty awesome. And if you look, I think we had something to do with this upward trend. Now, to be clear, Let's Encrypt didn't single-handedly make this trend go from, you know, 30% encryption to at the beginning of 2019, almost 80% encrypted page loads. There were other things. The browsers definitely helped. When Google and Firefox decided to say, we're going to mark pages as insecure instead of secure so that people start feeling creeped out when they're using an HTTP page, that definitely helped. Because now Sysadmins are like, well, crap, I got an upgrade just so my website doesn't look shady. Other things that look good, HTTP2 is going to be encrypted by default, and that is the only way it will be supported by all of the major browsers. So if you upgrade to HTTP2, the answer is everything's going to be encrypted. But there's more. DFF is not satisfied with just the web. The web is not enough. There are other things that we want to encrypt. And so for the last 20 minutes of this talk, I'm going to be talking about a couple of other things that we really want to see get encrypted. So in order, they are TLS, SNI, and I'll explain what all of these are. Encrypted DNS and Start TLS. So SNI. So this is a weird quirk and all credit to the Cloudflare for making this awesome image, which I am using as fair use because I work at EFF, but I know what that means. When you go to an HTTPS page, there's still information being leaked. And in particular, I know it's really tiny to see, but so this is the website you're going to visit. When you send the initial connection to that web server, you have to tell it, this is the domain name I'm trying to visit, and that has to be sent in the clear at the moment. And so that gets leaked. The actual page you're visiting, so if you're visiting example.com.index.html, the index.html gets obfuscated, but example.com gets leaked. And in particular, if this is sitting on a CDN, then that's telling the adversary which domain you're visiting because it could have been example.com, but now the adversary knows. And so they can see that. And then, again, after that, everything sent encrypted, but not until the adversary, someone listening on the wire can know what domain you visited. So the solution, encrypt the damn thing. So encrypted SNI is a, oh, I should mention, SNI stands for server name indication. It indicates to the server the domain name you're trying to visit. So the solution is to encrypt it. And so there is now a RFC out there. It's still a bit in flux, but you can now essentially send that server name in an encrypted way by getting a certificate ahead of time. And so now the adversary cannot tell what domain you're visiting. So great. But what about this side over here? I've been pointing over this side. I've conveniently ignored the DNS side of things. That is still going to leak the domain name you're visiting, because DNS, by default, is sent in the clear. It's an unencrypted protocol. And again, if an adversary is listening in, they can see that. So let's just encrypt all of it. Let's encrypt the DNS too. So there are currently two different protocols that are competing for encrypted DNS. One is called DNS over HTTPS, and it's exactly what you think it sets up an HTTPS connection and then makes a DNS query over it. And the other is DNS over TLS, which also is exactly what you think. It sets up a TLS connection, but doesn't build the stack all the way up to HTTPS. It just sets the DNS request over TLS. And they have some different pros and cons. DNS over HTTPS, it's harder for censorship-prone regimes to censor. And the reason is it looks like HTTPS traffic. Unless you start doing a serious traffic analysis, it just looks like something going over port 443. And so the only way to censor it is to block all of the HTTPS traffic. The downside, it's harder for network operators to monitor for malicious activity because it looks just like all other HTTPS traffic. And so if there's malware in your network and it's making DNS queries over HTTPS, it's going to be very difficult for you to tell that versus other HTTPS traffic. And it's kind of the... Basically the inverse for DNS over TLS. EFF does not have a strong position on either of these. We just want to get DNS encrypted. So if you have a strong position on either of these, come tell me. Tell me, I won't promise I will be fully convinced, but I am open to hearing good arguments for either side. Either way, this is the future. Encrypted DNS is the future. And we're going to see... We believe ISPs and also other providers, DNS providers, move towards encrypted DNS. And in fact, EFF is going to do another Encrypting the Net scorecard before too long and both of these things are going to be on it. Does your website support ESNI? And if you're a DNS provider, do you support DOH or DOT? Don't ask me when. It will happen in the impending future. But certainly if you work for a company, you should be thinking about these sorts of things. Because certainly if you work for a company that will get on EFF's Encrypting the Net scorecard, it's coming. What about email? So far I've been talking still mostly about Web stuff. I mean DNS supports the Web. Email is the cockroach that will not die. When the singularity has come and the hive mind, as we are all part of the hive mind, the hive mind will still communicate with itself via email because email will not die. So we need to fix email. So the way email works is you're on some client, either you're on Gmail's Webmail client or you've got Thunderbird or something. And when you're communicating to your email server, it's probably encrypted. And then when your friend is reading their email again, they're talking their email server, it's probably encrypted. But there's this thing in the middle between email servers that may or may not be encrypted and that's an issue because that is a point where an adversary could listen in on email. And so there's this protocol called Start TLS which is meant to solve this. And the way it works is essentially what happens is when these two mail servers are talking to each other, the first one will say, hey, do you support Start TLS? Do you support TLS basically? And then the other one will say, sure. And then they'll say, okay, great, I'm gonna set up a TLS connection to you and everything will be encrypted. The problem is that first message is sent in the clear. And so if there's a man in the middle attack, when this first server tries to connect, Google will say, yeah, I support TLS, let's do this encrypted. But the man in the middle will just strip that right out and the first mail server has no way of knowing that that's happened because all of this was sent in the clear, no authentication, there's nothing. And so it's trivially simple to strip that out. So that's a problem. Additionally, so the other issue is that many mail servers, even if they do support Start TLS, they're using self-signed certificates for that TLS connection. Which means that nobody on this end is ever checking the authenticity and verifying the certificate. They're not trying to say does this chain up to a valid certificate authority, to a root certificate. And so again, an attacker can sit in the middle and provide their own self-signed certificate and the originating mail server sending the mail is gonna be none the wiser. And actually, since on this side they're not checking the certificate either, Google isn't gonna be any the wiser. Both of these mail servers will think, oh everything was fine, the connection was encrypted. But in fact there was an attacker in the middle who was encrypting traffic to Google, decrypting it, re-encrypting it to EFF, EFF would send traffic, and then they'd decrypt it, re-encrypt it, et cetera. So it's again a really big deal. And this isn't theoretical. 20% of the Alexa top million domains that support connections, incoming email connections, don't use Start TLS. 40% of them present invalid certificates and I don't know what the hell is going on in Tunisia but 96% of the traffic is stripped of those Start TLS headers. So someone is actually going in there and removing the Start TLS header and preventing the encryption. Also, what the hell's going on like, maybe I understand that a lot of these places are in Uzbekistan, maybe I don't expect Uzbekistan to have the best security on the network. What the hell's going on in Denmark? Like 4% of traffic of the TLS is being stripped. Also this was in 2015, I should note. So this is, it can be a little bit out of date but it's definitely happening. So, trivial downgrade attacks, trivial impersonation attacks. The solution is a new, it's a new specification called MTA STS which stands for Mail Transport Authority or Mail Transport Agent, Strict Transport Security which is basically a, well, let's get into it. It's a way for a mail server to say always use TLS to connect to me. So the way it works is say I've got, I use EFF and I'm trying to send mail to somebody in Google. So first, my mail goes to the EFF mail server and then it's trying to figure out what do I do with this. So it goes to the DNS provider and says, mail.com have a particular text record in its DNS corresponding to this MTA STS field. And then presumably Google has already set this up ahead of time and it says yes, here's this record. And the record points to an HTTPS address. The HTTPS address hosts Gmail's MTA STS policy. The policy is basically just a text document which says these are the servers that you should be, if you're trying to send mail to gmail.com these are the actual MX addresses that will answer for that. Also, you can be in testing or in force mode if you want to make sure your configuration is working before you start rejecting email and mass you can put it in testing mode. And that way, even if things don't check out with MTA STS, in testing mode the sending mail server is still supposed to send the email. It will then log, it'll post a log to you saying there were these issues but you will start getting mail rejected. So this web server says okay, here's the MTA STS policy. And then I go to Gmail and say let's start a TLS connection. And then Gmail says great. And then you get your TLS connection. And so what would have happened if there was this attacker in the middle well let's say that this destination was in Gmail.com's MTA STS policy. So I'm sending, let's just assume it's Gmail.com. Then essentially what will happen is I will try to start the TLS connection the attacker will strip out that start TLS header and then I'll say holy crap something's going on according to Gmail's policy it's supposed to be using TLS it's supposed to be encrypted. So I'm not going to send the email I'll still send the email if it's in testing mode but either way the other part of the policy is a basically a reporting mechanism you can either send the report over email or post it to a specific URL but I can essentially communicate to Gmail that something shady went on so either you didn't get the email or I still sent it to you but somebody was trying to strip away this encryption. So that's great how do we actually get this out in the wild how do we actually make this happen? Well we started a project called start TLS everywhere and the goal of the start TLS everywhere project is to promote this spec get mail servers to use it it will also maintain a start TLS policy list so as you probably noticed there is this potential still for a downgrade attack if an attacker can intercept your DNS requests and basically say you know we go back to this point right here and say does Gmail have a MTSTS record and if they can intercept that first request and say no then I will think oh Gmail doesn't support MTSTS and so what the start TLS policy list is is it is a preloaded list that you can download from a different channel it's basically an out of band way to get a list of websites that support MTSTS and so now an attacker would not only have to intercept your DNS they would also have to basically block your ability to connect to EFF's website and if that's happening then probably something shady is going on if you can't connect to EFF's website we also want to lower the barriers to entry it's really hard to run a secure mail server in the same way it was hard to run a secure web server so in terms of promoting MTSTS adoption there are a couple steps first we have to make sure that mail servers actually support start TLS then we have to make sure mail servers can get certificates we have to make it easy for system minns to receive those failure reports because no system min is going to set this up and be like man I don't care if email isn't getting through anymore because if the email isn't getting through anymore you get fired and then you don't have a job and then it doesn't matter what great security things you did so we want to make it easy for system minns not only to receive those failure reports but also to post them to other mail servers when you are sending the mail and also make sure the MTA software finally knows how to check for and use MTSTS policies so there's actually quite a few steps to make MTSTS work so how are we going to do this? well, step one is the good news is mail servers already all support start TLS maybe they're not configured to use it but they all support it step two, getting a certificate well the good news is we happen to already have some software you can use to get a certificate so you can use certbot and we have guides for how to use certbot to get a certificate for postfix, for xm, for dovecock, for sendmail you can check out all of these if you run a mail server and it will walk you through this is how you get a certificate this is how you install it it is not quite yet just one click and it installs it for you and that's what we do to a lot of feedback we got from systemins who told us my mail system setup is so convoluted I don't want you touching it just tell me where to put the certificate I'll take over from there so that's what we did here's how you'll get the certificate you put it wherever you want if you're using a pretty default install here's where you'd put it that was supposed to be a greed check mark again sorry about the fonts so step three, receiving failure reports so this is the RFC for MTSTF and in particular it's a complimentary RFC for the reporting part and in particular the report may be delivered by email so the good news is if you've got a mail server it can send email and it can also receive email so all we have to do is basically set it up so that when your mail server encounters some sort of error trying to connect we just have to set up so it can send a report over email we don't have to create a new web server for you to receive reports via post or anything like that we're just going to say do it over email, it's fine posting MTSTF records and policies so this is like how do I actually tell the world that my server supports MTSTF so to do this, this is something we're still working on basically on the Start 2.0 everywhere website we are going to have basically a form you can fill out where you put in the details of like this is my my MX domain this is the actual domain for emails this is where I'm going to post my policy and then it will turn all of that into a valid MTSTF policy and tell you what DNS records to put it in and so we're working on that it's basically a web tool you can use to generate the policies and guide you through how to set it up it's not quite ready yet but it is something we're working on and then last but not least is getting MTAs to actually use MTA STS so this is actually getting the actual mail transfer software this is getting Dovcott or XM or Sendmail to actually pay attention to request the policy now that you've got it posted and then to know how to use it so what are the options here? well one, we could submit patches directly to Sendmail and XM etc but SysAdmin's upgrade their MTA about once every lifetime there's tons of really obsolete MTA software out there running and so even if we got patches in it would take forever for people to finally upgrade and actually get that in the wild we could write a middleware application that sits in front of MTAs and so the MTA is oblivious and then it's connecting to this middleware application that is then paying attention to all the MTSTF stuff and otherwise routing things back through that MTA it's one more thing to administer it adds some complexity but it also reduces other complexity the MTA doesn't have to know about start TLS or the MTA STS maybe there's another option if you have an idea come talk to me this is a problem we are still trying to solve we're still trying to figure this one out so we have our work cut out for us but we think the future looks bright we've come a long way in ten years from just a little browser extension to an encrypting the web report to actually launching a fricking certificate authority and being the biggest certificate authority in the world launching the start TLS everywhere project and we're really excited to see where we go from here before I finish up credit where credit is due this is the work of a tremendous number of people who have worked at EFF but also at other places it's just everywhere when we launched was a joint project with Tor a bunch of people have worked on let's encrypt and certbot let's encrypt I should say it wasn't just an EFF thing we worked with Mozilla and the University of Michigan to initially get it off the ground and now there's a whole host of sponsors that keep it alive if you don't donate to EFF you should donate to let's encrypt I think you should donate to EFF you should donate to EFF but you should also donate to let's encrypt so credit where credit is due a bunch of people have contributed to this work and the reason we've had been able to do all this amazing work is thanks to members like you your public support directly as almost all of you know EFF is a member supported nonprofit directly translates into this sort of technical work we couldn't have a team of 15 or so technologists on staff to do this work without your support and to be clear this is a I don't know if our 2018 chart is up yet this shows you where our money comes from and in particular these three things right here are all individual donations whether it's individual or like individual but it's employee matching that's basically individual it's not like Google gets to decide where their employees choose to donate they just match it this part right here this is sheer luck this is when we get an award in the courts and this is not when we're in a lawsuit sorry about that this is when another entity has filed a lawsuit and the answer is instead of giving a dollar to every person in a class let's give it to an entity that actually is going to do some good for privacy or security and that's totally like we can't control that so we can't rely on that and this is us bugging organizations for money because even though we have foundation in the name we don't actually give out money we take in money and then we turn it into awesome projects so as you can see the majority of it is from individual members so on behalf of everyone at EFF thank you DEF CON for your tremendous support over the years thank you for making work like this possible and I'm happy to take any questions question we don't EFF doesn't personally or like EFF work closely with Linux foundation but as you said Linux foundation does essentially do the administration and overhead for ISRG the organization that actually runs Let's Encrypt so which is pretty awesome because that way they don't have to like hire an HR person or accountants or stuff like that they can focus on Let's Encrypt yep yeah not yet are you talking about so I am pretty sure ESNI is a different thing I think I'd have to go double check we have seen that that sort of argument from our perspective the we feel like the right tradeoff is that for uh given that I guess the way I'd put it is normal users don't have the same resources to invest in making sure that their traffic is encrypted as a corporation does and so we feel like the right tradeoff is to get basically get as much encryption as possible and then if a corporation does need to do that sort of TLS termination the answer is it's kind of a side effect of making the web secure more secure for everyone I guess like I definitely get it yeah but it's true yes it does yep yep absolutely yeah so in particular uh this is that's an interesting one so uh I can think of two cases one Cloudflare actually rolled out ESNI in South Korea I think it was either early this year or late last year and basically ISPs in South Korea I believe in the direction of the government just started blocking ESNI just straight up because they could no longer look at the domains that you were trying to connect to uh similarly uh related uh about a month ago now I think it was an organization an industry organization of ISPs in the United Kingdom nominated of all organizations Mozilla as their internet villain of the year because Mozilla wants to push forward encrypted DNS now to be fair what Mozilla because like ISPs don't yet support encrypted DNS what Mozilla is thinking of doing is is Cloudflare has a centralized DNS server and they're supporting encrypted DNS uh and so it hasn't rolled out yet and from my understanding I think what Mozilla is thinking about doing is if you're in the US where the privacy laws are crap then they will turn that on by default because frankly Cloudflare has a better privacy policy than most most not all most US ISPs however that's not true in Europe or the United Kingdom where they're actually privacy laws controlling what your ISP can do with your private data and so they're thinking that it will be an opt-in there but regardless in the UK there is an obligation by the ISPs from the government to censor for all sorts of silly stuff adult content terrorist content whatever and because the ISPs if you're using Cloudflare's DNS in an encrypted way could no longer do that censoring really up in arms and they said oh my god this is terrible they later rolled it back once they realized that not everyone was against Mozilla doing this encrypted DNS thing but certainly in you know countries where there are I would say is democratic rule of law we are still seeing pushes for censorship or the ability to censor and so that's not even touching places like China or Iran or something else like that yeah so I think the from my perspective the way to do that that's the destination you have to get there slowly because if you do that immediately people will just stop using your software and it's just a sad we won't stop using that software because we'll be like what the fuck's going on someone's trying to intercept my traffic but you know my nieces and nephews my friends they're just gonna stop using that software to do something that works and so the trick is I feel like to raise the bar slowly and by raising it basically it's kind of like this boiling frog approach till you get to the point where there is no longer anything else to switch to because everything is encrypted yeah ACME protocol so there's a couple things one of the things that they're working on I don't know if it's rolled out yet is basically doing multi vantage point verification you would have to man in the middle not just let's encrypt servers they're gonna do it via Tor so you would have to man in the middle a bunch of unpredictable Tor nodes as well and so if they do like three or four of those from different exit nodes then that raises the barrier it's still possible eventually someone who controls all the internet traffic man in the middle of it yeah so that's I was gonna say I think there was another thing but that's the big one that they're working on it's not there yet the other thing is there's a thing called certificate transparency I'm sure many of you have heard of it but now let's encrypt logs all of its certificates to certificate transparency servers and so if you run a domain you can essentially log into these certificate transparency servers and say did somebody issue a certificate or did let's encrypt issue a certificate from my domain and was that me if I'm not using let's encrypt and that happened or I am but I go look in my server logs and bot doesn't say recently renewed my certificate something shady is going on so that's detection not mitigation but detection is still helpful while we're working for mitigation alright thanks very much everybody