 I would like to introduce you, Hubert Cario from Red Hat, who is going to present about TLS Test Framework. Thank you. Thank you for coming for the presentation. As Niko said, I'll be talking about TLS testing and verifying that it's actually correct and properly configured. So SSL is dead. Let's start calling it TLS. SSL was the first version that was actually published, but now both of those versions are deprecated, are insecure, and you shouldn't use SSL. The only versions that are secure are TLS 1.0 and up. So what is TLS? TLS is Transport Liars Security. It's known from HTTPS. So basically every time you go to Gmail, you use TLS. It's also used for email security, for Tor, for Internet of Things communication, voice over IP, VPNs. So basically any kind of security, it's most likely will be TLS. So why we use TLS? First is confidentiality, so that the contents of the communication are secret. The integrity, so we make sure that the communication is as it was sent by the other side, so it wasn't modified in transit. And authenticity, so that the data comes from the person we think it comes from also in subsequent messages. So like I said, TLS is a very old protocol. Currently we are pushing nearly 20 years of age if we are looking at the actual on the wire data, since TLS version 1.2 is very similar to SSL version 3, so over 20 years now. Over those years, we had a lot of changes. Nearly 70 RFCs were published that either extend, update, or deprecate functionality in SSL or TLS. We have also six wildly deployed drafts that weren't actually standardized, accepted by the Internet Engineering Task Force. We have something like 27 extensions, and there were a lot of fixes for protocol and implementation bugs, like for secure negotiation for fallback SCSV, signaling cipher suite value, client hello padding, extended master secret that's protection against triple handshake. That made the TLS very complex. Currently we have over 300 cipher suites, just the official ones. There are also unofficial ones used by a very small minority of clients and servers. We commonly use about 15 different key sizes and types. That includes basically four different crypto systems. Commonly they used something like 37 different DV helman groups, 16 signature and hash algorithm pairs. It's a complex protocol, and most of it is a mess. So if we want to test this stuff using TLS libraries that are already available to us, it's rather hard. Most of the TLS implementations really don't want to send invalid data for some reason. So also there's a problem that they want to deprecate old stuff, that they don't want to already ship with SSL version 2 added or supported in the code. So if we take the newest open SSL, you won't be able to test if the server on the other side actually supports or not SSL version 2. Because TLS 1.1 doesn't know how to generate SSL version 2 client hello, so it won't get the response from the other side. Now, we could also use existing protocol fuzzers for testing TLS. Unfortunately, TLS is a complex protocol. There is a lot of state information that it needs to be maintained between the messages. And there's also the question of encryption. So you need to check. You need to be able to encrypt the messages and know which ones were already sent, which weren't sent, and stuff like that. So most of the protocol fuzzers actually are able to keep very simple or basically no state. So they are aimed at testing stuff like HTTP, which doesn't have state, or SMTP, which has very limited state. So that's why we started working on TLS Fuzzer, which is the dedicated network fuzzing or testing library. What we basically TLS Fuzzer runs on Python 2.6 and up. So basically on everything you can imagine. So all distributions shouldn't be much of a problem. It has no native dependencies. So as long as Python runs on this thing, you can run a TLS fuzzer. We currently have something like 60 test scripts that test different features or fields of TLS protocol that translates to over 18,000 individual test cases. It's also relatively fast. On local host, I'm able to test something like 300 test cases per second per core. It focuses on the new crypto and TLS features. We're using it internally at Red Hat for testing new features that are being introduced to TLS libraries. So this is the primary focus for the development. It's also unit tested. Currently, we have over 95% statement coverage. And it allowed us to find something like 20 issues in OpenSSL, GNU TLS, and SS, including one CVE. While I'm calling it Fuzzer now, it's not yet a fuzzer. For now, it's just a framework that will allow to build a fuzzer. And also, while it will be a fuzzer then, it will be a TLS protocol fuzzer. So it will not work on the level of bytes sent on the network. It will mostly focus on modifying the messages. So not sending invalid messages, but sending invalid fields in the messages. So why should you use TLS fuzzer? While we will be testing TLS implementations and hopefully most of the libraries that we don't ship will start using TLS fuzzer, the problem is that some things are configuration specific. And some behavior, some insecure behaviors may be caused by changes in configuration. There are callbacks in the libraries. So if you have callbacks, those callbacks may cause your server to be insecure, despite the underlying TLS library is secure. So that helps with integration testing. Also, you may have some kind of black box like UPS or a firewall or some kind of appliance. And you want to check if that appliance will be able to connect to that appliance with some future fuzzer if it won't have TLS 1.3 incompatibilities or TLS 1.4 incompatibilities. Because TLS as a protocol has a way to negotiate the version that is commonly supported between the client and servers. But some servers implemented it wrongly. So that was one of the reasons why we needed the fallback CSV. So that's also the forward compatibility. And also, you may want to test, use TLS fuzzer for testing your server, your server configuration against obscure clients. Since TLS fuzzer can generate basically any kind of messages, it can generate messages which are sent by uncommonly used clients. And you may still want to keep the configuration that is compatible with them. So you can simulate basically if that client and see if the server will respond and negotiate the connection. So if we want to do that, we basically need to tell the fuzzer first that where to connect, which host which port, which kind of ciphers to advertise and in which order, what kind of extensions to send. So we can send the SNI. So this is the virtual host support. We can do also support groups. So this will be for Elliptic or Divi Helman. Signature algorithms that are for TLS 1.2. And finally, we tell it to generate the client hello with those values and 3.3, which means TLS 1.2 on the wire. And if we, as a response, get a server hello, that means that the connection will proceed and the server actually understood our client hello, said, OK, that sounds like a sensible set of values. We will be able to continue. It sends them the server hello, and we know that we are compatible with it. If it isn't the case, then servers either will send an alert or just close the connection. Both of those situations will be detected by the fuzzer and the test case will fail. OK, so if we want to make a complete connection, this will look something like that. So we, again, have the client hello generator. Then we expect the server hello, then some more messages which are typical for the communication. And at this point, we send to the server some kind of data. Here it could be like a HTTP get or something like that. And then we can expect application data from the server. So we can, on the TLS protocol fuzzer level, we can detect where, well, basically we can do HTTPS requests and stuff like that. And then we have a regular connection close. So we tell that we want to close the connection. And for the close of the connection, the server should reply with close notify also. And then we expect that TCP connection to be closed too. Now, we can also test for vulnerabilities. In OpenSSL, we had a problem where the server would accept the changeCypherSpec message too early in communication. That would cause the connection to establish predictable keys for the encryption and integrity checks. So if you want to see what happens, we basically have to remove all those red messages. That's what we end up with. So we send the changeCypherSpec too early. And if we send it too early, we expect the unexpected message from the other side. We can also create malformed messages. So we can create a regular client. Hello. And then tell it truncate that message, that answering message, by four bytes. And as a result, we should get a decode error, which indicates that the server has noticed that the message is malformed because it's missing those four last bytes. And then you should close the connection. I also added recently some automation for that. So you can set up how to basically start a server and what kind of tests to run against it. So here we have an example for OpenSSL. It sets it up with some test certificates and tells it to accept connections on localhost on 4433 port. And against that server, we'll run those two test cases. This can also tell it that you can exclude some test cases. You can tell it that, yeah, for that version of OpenSSL, let's say, that the test script should fail and stuff like that. So you can make sure that you don't have false positives and false negatives while running it. So if we run this example, we'll get basically this kind of result that will tell us that it ran that test. There were no errors, so it doesn't print anything. It's finished. Then another test, and again, finished. Then killed the server. The server was killed by the SIGterm signal and it ran in the overall five test cases, non-failed. So two scripts passed and zero failed. Now, for the features, we support the SSL version 2 and up till version 3 less 1.2. Most of the ciphers, so AS, CBC, GCM, Chacha 20, this is all supported. Most of the, basically all of the HMAX are supported. Key exchange, both with RSI keys, Divi Helman, Key Exchange, and Elliptic Curve Divi Helman, some new extensions. So encrypt then MAC, secure a negotiation, foldback SCSV, extended master secret. And also we support the draft next protocol negotiation and the actually standardized application layer protocol negotiation. This is necessary for HTTP2 negotiation. Finite field Divi Helman group negotiation. This is actually part of the support for TLS 1.3. It will depend on this functionality and also the server name indication as I showed before. Unfortunately, as most projects, most of the stuff is in progress, we are missing support for TLS 1.3. We have just parts of it. We also are working on RSAPSS and X25519. This will probably come in the next two months or something like that. We don't have support for PSK, Camelia ciphers. And one of the big things is that we don't have support for ECDSA keys, unfortunately. And the upside is that currently something like 90% of the internet uses RSI keys. So it's not such a big, such a hurry to support it. We also don't have hard bit protocols. Unfortunately, I cannot show you the hard bit example and Kerberos from more obscure things. If you want to contribute, unfortunately, we cannot see. But if you want to contribute, the whole code is and script examples and stuff like that is on GitHub, on the TLS Fuzzer repo. TLS Fuzzer internally uses TLS Lite NG implementation. This is a pure Python TLS implementation that I used for the crypto stuff and basically like ASA library to code against and test against. The Fuzzer is under GPL version 2. And TLS Lite NG is under all GPL version 2. Unfortunately, you cannot read them. But tags review request and up for grabs are for people that want to contribute to stuff that I would like and I think that are easiest to contribute to. If you want to take a look and see if you don't find anything fancy, go ahead. And I'll gladly review that pull request. Questions? Any questions? Why is GNU-PG missing feature in TLS? You had it on your slides. So the GNU-PG is basically supported now only by GNU-TLS, nothing else. And actually, most likely, it will go out from that too. There's no plans to add support for GNU-PG to TLS 1.3. And it's incompatible with it. So it's unlikely we'll see it. And most likely, it will be simply removed from GNU-TLS. So you mentioned that you can truncate the packets and stuff. Can you also modify individual fields, for example? So you have a length field, maybe some client or some library is incompatible. If you have more data in extension or less data, then specific length. So the question was, if I can do any kind of other modifications? Yes, I can. There is a lot of modifiers, or like mutators, let's call them, to the packets. I can inject changes on the handshake protocol level. So I can modify bytes. I can insert bytes. I can XOR them with certain values. I can, of course, set any kind of values I want in the array. So if I want a longer array, I can just add stuff to it. And also, I can inject changes while the encryption is taking place. So in default TLS, the encryption goes like this. You first have data, then you calculate MAC over it so that there's integrity checks. And I can modify that MAC value. Then you have added padding. And again, I can modify that padding using those kind of mutators. And only after that it is encrypted. And that encrypted data I can again modify at arbitrary places. We have time for a few more questions. So what are your testing experience with existing implementations? Especially also TLS hardware accelerators. I mean, you don't have to mention names, but how often does a TLS server crash when being accessed with your library? So the overall experience is that if you can imagine some kind of error and bug, you will find it on the internet. And I haven't tested much servers, like typical accelerators and stuff like that. We are focusing on the stuff we ship in Rails. So that's NSS, GNU TLS, and OpenSSL. From those, there are differences. In NSS, it's usually quite good. With OpenSSL, it's not exactly this stellar, and GNU TLS is similar. As far as RFC compliance goes, I wasn't able to crash either of them. Mostly the problems are of the, like, they accept malformed messages when they shouldn't. But they interpret the data quite sensibly. So in 99% of the cases, this is not exploitable. But better safe than sorry. That's my opinion with the link of that kind of problems. If I understood correctly, you're not generating any, like, a fuzzing payload yourself, is it? Like, it's not that you would generate random test cases, but rather you pre-define your tests, right? Have you looked into, say, escapee, for example, what these guys are doing? And I don't know, is there any plan to convert to escapee, say? So the payloads are generated using the TLS fuzzer. So the problem is that you have the encryption in place. So you have to make sure that, like, you have to fuzz the data before it is encrypted, usually. And Scapi was actually one of the libraries I was looking at before writing it, before starting to work on TLS fuzzer. I noticed that it is very hard to keep that state that is necessary for encryption in Scapi to be able to generate the finished message, because finished messages needs to be encrypted. So basically, that was one of the libraries I excluded from using, because I couldn't use it. I couldn't basically test using it the first bug I was working on. OK, thank you, Hubert. And we have a five minutes break.