 All right. Welcome back, everybody. I know it's getting a little late in the day, so thank you all for turning out. I'd like to introduce our next speaker, Lee Brothersston. He'll be talking about, as the slide deck suggests, Blue Team TLS Hugs. Let's all give him a warm welcome. Hello. Okay. Hi. So I'm going to take the standard format of state a problem, say you're all doing it wrong, and what we should do to fix it. A quick sort of upfront disclaimer. I mentioned HTTPS a lot. It's not a talk about HTTPS, but it's the TLS implementation that most people know, so it's an easy example. And, you know, it's everywhere. It's pretty well understood, and TLS has been used everywhere. So a super quick primer just in case anyone doesn't know what TLS does and not SSL. It's not a cryptographic algorithm. It's a protocol for determining what kind of crypto you're going to use between two endpoints that don't know what each other supports. So it's a negotiation protocol for encryption rather than the encryption itself, although the encryption naturally follows in the payload afterwards. So here's how it works, or kind of how it works. This is really more how it works. So the client sends a client a hello packet, and that packet contains a bunch of information. That's things like what cryptographic protocols it supports, what hashing algorithms it supports, what elliptic curves it has, if any. And then the server applies with, from that list, I also accept these things, and they can determine what they're going to do. They then exchange keying data, and then encrypted data, and then renegotiate, and that carries on in a cycle until they're done. So that said, the current state of the art, we should look at that to sort of see what's wrong and where we can go and do other things in the future. There's really three things that defense people can do with TLS. You can ignore it, you can break it, so that's man in the middle, and that sort of thing, or you can embrace it. In the real world, we don't bother with embracing it, and we either ignore or break it. The closest to doing it right is actually the domestic situation in a lot of cases, not in a corporate environment. So I'll just explain a little about how, what I mean by each of these. So ignoring TLS, if you've ever run something like an IDS or something, and you see TLS encrypted traffic on the network, it's just garbled up encrypted mess. I mean, that's the point, you don't see the payload. If you look at the snort community rules as they stood at the time, this is the clear text and TLS-enabled versions of each of the protocols, and that's the number of rules. You can see there's significantly more for the clear text, because, you know, if you've got your IDS alerting on any request that's like a dot, dot, slash, dot, slash, dot, dot, slash, that will alert fine in clear text, but not in TLS. The TLS rules are just harplead and things like that. So you're largely blind to a whole sea of exploits, and so effectively it's just ignored. So that leads to this. Does anyone know what this is? I will give you a further hint. It is the universal firewall bypass port. Because if we don't understand crypto, we just let it through, right? Because you've seen this, I'm guessing. You can get out of a lot of corporate environments just by SSHing on port 443, because what are you going to do with port 443? Well, we'll come to what you can do, but right now we're talking, it's often ignored and just left open. You can also often connect even when they have a proxy and just use a connect command on port 443, and it will let you straight through. So what do we do in the situation where we're not looking at TLS on the network? Most of the time that comes down to using endpoint security solutions. So antivirus, anti-malware, you know, various memory protection techniques and that sort of thing to capture all the bad on the endpoint as it's coming in rather than looking at it on the network. We also look at blacklists. So you've got a list of blacklisted IP addresses, for example. You don't know what the data is going to them, but you know they're part of a command and control network or something. So you know that there's bad data going there. You avoid it, run away, and you know hope that that was a good blacklist. But there are problems with blacklists too. Very often they're not up to date or are lacking information, but even if they have the information, there's a lag between it coming from the real world to the blacklist, from the blacklist to your environment and your environment to you actioning it, by which time half the addresses are out of date. So some of the time we break TLS. So I'm talking things like where you have these proxies that run their own CA and they man in the middle of the traffic to explain how it works. You have a client, a proxy and the web server you're connecting to, or any other server, sorry. Your client makes a TLS handshake with the proxy and then fires a request. So sends the URL that it would like. The proxy then does a TLS handshake with the server. The request gets forwarded, goes to the server and it gets a response, which then gets forwarded and sent back to the client. The problem is that this stuff is in the clear. That's because the proxy has its own SSL certificate. And what this means is that you have the actual CA on the right that uses all the proper signatures and all the magic that's associated. And you've got your shonky, my first work CA happening on the left-hand side. And this brings like a whole ton of issues other than it's a really horrible thing to have to try and manage. So, to explain how that works, it breaks the CA model in a bunch of ways because although the CA model is kind of broken to start with, this breaks it even more because it is not one of the certificate authorities trusted by browsers, for example, and that causes a bunch of issues. And I'm just going to go through a few and then we can get onto what you can actually do to fix it. So there's simple compromised appliance. If that appliance in the middle got hacked, data's going through in the clear. It could be manipulated. It can be sniffed and logged and all sorts of stuff. So, you know, think of online banking or logging in to look at medical records or something like that, just compromising the device that has the certificate is enough. If the key that's on there gets compromised, that's bad too because if the key is lost or goes to someone else, they will be able to take that key and impersonate any website to your devices because when you set up that sort of system, you set the CA as trusted in all the endpoint devices. And also, key management is hard. It's really, really hard to get right. If you want the closest I can get to an empirical study on key management is hard, if you look at ISO 27002, and I apologize for compliance framework type stuff, but key management alone has a higher word count than the whole of the rest of the cryptographic control section. And in the cryptographic control section, six of those are words key management referring to go look at the key management section. There's also poor certificate validation. So, when you connect directly to something with a web browser or any other TLS-enabled client, it does a bunch of checks. It does things like does the host name on the certificate match what I was expecting? It checks the certificate chains, it looks for revoked certs, it looks to see if it's a trusted CA, all these things. None of these you see when you man in the middle because all you see is the one valid cert for everything that you just installed on that device. That means that you are trusting that device to do all that validation for you. If any of you have ever run those devices, you'll know that you can't trust them to do that. If nothing else, their list of trusted CAs is horrifically out of date, typically. And they fail a bunch of the other validation checks. So let's talk about that trusted CA list. If you look in your browsers or your OS, in fact, as many browsers use the default OS12, you will see that there's the list of trusted CAs. So that is authorities who are able to sign certificates as valid. But these things get revoked. You trust what the appliance trusts, right? But a few months ago, a few months ago, WoSign got pulled by a bunch of browsers and pulled pretty quickly because of some of the practices they were employing. But if you are not updating that appliance in real time, you can go to one of those sites that the certificates have been pulled from and your browser will be none the wiser. It will think they're all valid because it's just getting that cert from the proxy. Anyway, there's also certificate unpinning. If anyone knows what certificate pinning is, this is where your client is preconfigured to say, when I connect to such and such a service, I expect to see this certificate or this certificate which has been signed by this CA or some combination it matches this fingerprint or it's got this intermediate one, but whatever you can pin to one degree or another what you're expecting to see. Certificates around the issue of things like, say a rogue government with not many scruples somewhere, convinces a CA to issue a certificate, a valid certificate for a domain that's really hosted with another CA, and then they can use that to man in the middle and without flagging up warnings because it comes from a valid CA. When you install a local CA, as is the case when you're having the man in the middle proxy, most browsers disable pinning. This is because if you're running one of these proxies, people like Google use pinning and they don't want Gmail to just fail everywhere on your corporate network. Disabling pinning is bad in a number of ways. And finally there's the malicious insider, just the member of staff who now, because they have access to the appliance or the certificate or both, has access to do all sorts of bad on your network and to your devices. So that is my brief TLS, we're doing it wrong. So I'm going to talk about trying to embrace it a little more and do the right thing. So let's embrace it. I'm going to set some goals. So if we look at all the different devices that intercept the traffic and do things to your traffic, they generally do this list. So it's things like content filtering, logging that content filtering, checking certificates, malware detection, et cetera, et cetera. In my opinion, which is largely unscientific, but I'm going to give it anyway. Go team. This is the stuff they approximately do right. This is the stuff I don't think they do particularly well. This is the stuff they sort of do well and not so well. And this is what we actually want. Not many people seem to want cryptographic checks. And by that, I mean checking which algorithms are used and not used. They're pretty happy to trust their browsers to do that. And granular content filtering, I have an argument about this, that people don't really need that. And by granular, I mean full URL content filtering instead of host name. For example, most people don't want to block one category of porn and not another category of porn. They just want to block Pornhub or not block Pornhub. That's pretty much where they're at. So I feel free to disagree, but that's what I'm saying. Anyway, so I'm going to try and get those goals sorted. So I'm a believer that perfection is the enemy of good enough. People boil the ocean trying to get RFPs done for 100% complete solution. So I'm trying to offer a, gets you a good chunk of the way there, but quickly and easily and without hassle. We have many cases in the real world where sort of half-assed things do work and get the job done. So we'll work with that. So I had a thought. So there's first, there's packet, there's a bunch of packet sniffing we can do. I did some studying a little while ago on TLS fingerprinting and you can use that to determine all sorts of bad stuff. The long and the short of that, there's another talk on that if you want to learn about the in-depth stuff, but TLS fingerprinting is basically you can go through and monitor the TLS handshake and determine what client it is. That means that you can treat this like an IDS and say, oh great, this host is connecting on Google Drive, this one's on Skype, this one's on Tor, this is using Chrome, this is using IE. If anyone uses a corporate environment, you'll know that seeing Chrome and IE is fine, seeing Tor and things is not. And depending on your environment, Google Drive or Dropbox or something could be potentially someone losing data. But what I'm getting at really is that it gives you the opportunity to understand the encrypted connections in your environment to a good way, a good degree, sorry. You can also use it to spot bad things in general. So you can spot rogue clients. You're in an all-MAC environment and you see IE, that's probably someone plugging their own laptop in. You can also spot malware. For example, Drydex use TLS. You can monitor the TLS connections that it makes and you can see that not only which clients you have are infected, you can map out the command and control network real-time as they talk back to it. This is Cisco saying the same thing because I can't push the slight advance button in a sensible order. But this is Cisco's blog that was saying, hey, we can't see stuff because malware uses TLS. But they do use different TLS fingerprints to a lot of other apps that you can often spot malware using it. You can also do semi-automated fingerprinting. So it's so easy to make fingerprints that you can automate creating a fingerprint for anything you see but have not seen before. And then that can be reused when you see that fingerprint coming again. So if it's seen once and associated with something bad and then you see it come back again, you can often tell that it's the same something bad. Prime examples are botnets have their own. Metasploit has its own. So you don't have to know what it is in advance. If you tie it back to a previous incident, you can still determine that it's bad. And you can do pseudo-anomaly-based detection as well, which is largely the same thing, but it's based on past behaviors. And finally, you've got fingerprint canaries. So if you think of being in the situation of being a generic ride-sharing app, a bankrupting blockbuster as a service, or one of the closed-API social media services, they all create their own API and their own client, and they don't open up to third parties. If you were to one of these companies, you could monitor all connections to your API endpoint to see if it matches the fingerprint of your client. If it doesn't match your official client, you just drop it on the floor and you've then removed Curl, Burp Suite, and all the other things that you potentially don't want touching your API endpoints, unless someone goes to the hassle of actually masquerading the TLS stack of that app. To give you an example of a similar technique, I mean, Mozilla released this blog about checking for man in the middle, which is exactly what I'm talking about here, based on the connections coming in from the browsers. You can also sort of do incident response and attribution, although attribution is hard and that's like a whole area I shouldn't venture into. But if you have an exploit that comes into a network over TLS, the tools that are used to develop it, if they have their own signatures, you can re-watch that signature propagating across a network and you can watch the progress of an incident as people pivot horizontally across a network. And I'm going to just quickly demo what this actually means before I stop talking about fingerprinting and drive you all insane on that one topic. Just for reference, though, this is the best German word ever. This means it doesn't work when someone's watching, so I've recorded it. So this is just a tool I wrote running in the terminal and this is a standard Windows VM. It's from a few months ago, so I apologize the versions are a little old, but you'll get the idea. So you go visit some site and it's just a packet sniffer at the end and it's picked up that you've got Firefox and Signal, because why not? Connect into a bunch of things and then at the same time, you can go, I think it's Google Drive or Dropbox I used. My memory's terrible on this. Anyway, yeah, so these all use either APIs or their HTTPS under the hood or whatever. So as the file syncs up, you'll see that Dropbox comes up there as having connected. And finally, the thing that most people like to see on their network is Tor. So I'll just let it do the Tor thing. So this is Tor browser, which is just a bundled Tor and Firefox all in one tool. And as that connects to the Tor network without doing any web browsing, it starts screaming all over the network, I'm Tor. If you were being a bit more active about that, you could use that for alerting and you could pick up on malicious network traffic. Anyway, that's that bit. So, oh, thank you. So enough on fingerprinting. There's server responses too. So the server hello packet also has a bunch of information that's really useful for you. First of all, it lets you know what has been negotiated. So you can tell if they're using old TLS versions, weak Cypher suites and that sort of thing. I'm running a long time, so I'm going to go super fast. You can harvest all the certificates. I'm going to refer back to something I mentioned earlier. You can log all the certificates that are associated with the website at the time. So in the future, you can go back in time and see what certificates you saw. Why is this useful? Well, this again. We'll sign. And Google dropped China's root cert. And I think other people may have done too. This lets, oh, sorry. And SuperFish, who had its horrible certificate situation. This is super easy to do. You can use a plug-in in bro and that will do logging of certificates. And then when someone says, hey, this certificate authority has been acting bad, you can go back retrospectively, see which sites use that cert, and you know if your clients could have been compromised in the past. Finally, I'm going to talk about inline TLS shenanigans, which is actually my new thing. So you can mess with the handshake. It's in the clear. It's not cryptographically signed, so you can modify what's inside it. I suggest this. No. What I suggest as an alternative is a proxy that works at the socket layer and passes data backwards and forwards. So it doesn't mess with the encryption. It doesn't re-sign things and fall foul of all those CA issues I mentioned. It can still see the server hello stuff. So it knows what's been negotiated. It knows what cryptographic protocols are being used. And it can see that stuff. Most importantly, it can fingerprint stuff and because anything vaguely current uses SNI, you can pull the host name in the clear that it's connecting to. That gives you enough for coarse-grained content filtering, and you can also tie this together with clients. So you can say that Skype can access Skype servers, but Skype servers alone, i.e. you can go to all sorts of other servers, whatever. This is also safe because it's subtractive. I can't put in a crappy cipher suite and trick browsers into using degrading themselves to that terrible cipher suite. They never offered it in the first place. I can, however, pull crappy cipher suites out and force them to only negotiate the stronger cipher suites and other aspects. And then you've got the compliance diseases. Most of the compliance standards including PCI will say things like, you can't use old versions of SSL and TLS. Well, if you're monitoring that handshake and have a proxy in the middle, you can look for old versions of TLS, drop them on the floor, and ensure you maintain compliance on your network irrespective of what the applications are doing. So there's a remaining problem. If you're not man-in-the-middling, it's super hard to put that little pop-up up that goes, you've been blocked, you were a bad person. So I tried a few things. I put a bad certificate in. Shockingly, Chrome and Firefox were super not happy with that plan. There is a response of server hello done, which is just, I'm not sent a certificate, but I'm done. It also doesn't sound super good to the user going, can't provide a secure connection, and this is Chrome, sorry, and in Firefox saying, it says secure connection failed, also sounds bad, and if it's the sub-component of another page you've allowed through, it throws those SSL errors about mixed content. So it's not very good. But I came up with a solution, which is super crude but works. So I will just demonstrate how this works really quickly, and then I will show you how that looks in browser sphere reference. So demo again. So this is my tool at the top. This is a proxy, it's just written in Go. I'm gonna curl, I've said it as a proxy, but it can be a transparent proxy too. This is just for ease of demo purposes. So if I try and curl the Crypto Village website, it logs that I used curl, but I went to Crypto Village, it's not on the block list, and it went through at the bottom. If I curl something that I'm less interested in, in this case one of our favorite ad networks, what you'll find is that the server-reported SSL handshake, that's actually not as bad as it sounds, that's literally just drop the TCP connection on the floor and it got the drop on the proxy. This looks crude in curl, because I'm just showing you and it has that nasty error, but it actually has the valid cert come back every time and the browser gets to do all their thing, but more importantly, it just looks like the server is down to the browser when you go direct, and when it's a sub-component on a page, like an image or a script, it just looks like the missing little square in all the browsers. You don't get any SSL errors, you don't get big buffs or anything like that. So gonna go as fast as I can. I say we did course content filtering fine and application detection we got, course logging we got, granular logging not really, I'll come back to that but I probably won't have time so I won't, certificate checks we did, malware detection we sort of did. We didn't do it in the AV sense, but you can have TLS signatures on malware so you can pick up a certain amount of it in page exploits, totally didn't do that, but I said I was only doing 90%. We don't detect documents going out but we can detect things like Google Drive and Dropbox and things that may hint a data exfiltration and data exfiltration falls under that banner. One more thing, do I have time? I'm gonna try and do it quick. Course train login, this is an encrypted payload. It's 227 bytes long. There is a set ratio of algorithm, clear text to in cipher text size. You can take it so, if you know the size of something you can guess certain amounts of the content. You know the host name that it connected to so you've got 206 left. You know the user agent because I fingerprinted the OS so you can roughly guess what the user agent is so now you're down to 151. You know what that user agent accepts so you're down to 95. You know how it encodes so now you're down to 58 and you've got the IP address so you can probably guess that you've got 26 left. You know that you're gonna have the words get an HTTP so you can guess that the remainder of the bytes are probably the URL and narrow down to a degree where you might be. Unless you pad, so just like pad everything. Conclusion, I'm running low on time so this is stuff related to this. Probably the main thing to do is just look at the slide deck one speakerdeck.com, my name then you can find this. I was gonna say any questions I've got maybe 10, no time for questions. I will be in a corridor or a bar or something and you're more than welcome to grab me and I'll do this again.