 as an error. So this talk is pretty tightly packed. Um, so I have to run through some of the slides. I may have some time for question and answer at the end but I'm not sure yet. So let's go. Story line. Well, I think that doesn't work. Hey, so the idea is maybe you're an engineer, you're just on vacation or like here, that's the venue of PyCon Russia. I've been in the last couple of days. Enjoying your day at a pool, relaxing, maybe having a cocktail. Super great. Yeah, fantastic. Final vacation. And then your pager goes off. You see, oh no. SSL error. You run to your room, already night. Check the locks and you see something like that. Totally gibberish. Wow. What's going on here? So I will give you a bit of pointers what may go wrong and on the line also explain how TLS SSL works on a high level including the handshake and certificates. So my name is Christian Heimus. I'm from Hamburg, Germany. I've been working with Linux and Python for a pretty long time. I'm also a core developer working on the security team, maintainer of the SSL model at Red Hat. My professional life were actually making running. Uh, work also in security engineering on identity management stack, single son on solutions called free IPA and related. So the stock, our first would give you short history of SSL, a very high level view, a quick way how the connection works including short internet so crypto one on one. Uh, after we have the handshake done, I will explain certificates a bit. Uh, how the root CAs work, hosting verification. Now at the end, I give you some pointers like resources, links to books and tools you can use. And hopefully maybe some questions. But before we start, should you actually lose, uh, deploy and use TLS? Um, there's one very short answer to that. It's yes. The bit longer answer is, uh, yes, you should. Because TLS has just one big issue. It's actually not performance, but it's just not used widely enough yet. So there's a website, if you consider performance, is TLS fast yet? Explain to you why TLS is actually fast enough? Usually it's sometimes faster. Try hunt, make this nice graph. So it's downloading a big website, lots of resource over HTTP. Takes almost six seconds. If you do that in the modern browser with HTTP 2, it looks like that. Bam. Done. One or two seconds. Much, much, much faster. Another reason is, if you happen to update Chrome today, wow, very lucky, you will see that. Starting, uh, like yesterday night, or today's morning, depending when you update, uh, any kind of non-HTPS link will give you a very big red warning box, not secure. Uh, several websites, including Google, also, uh, give you much better page ranking when you use HTTPS. So there's lots of reasons. And these days it's very easy to get a certificate for no money, use let's encrypt. Fantastic tool. So history, um, SSL is sometimes confusing because you have like SSL, you have TLS, and actually one is greater than three. It's the first thing that people confuse when I talk to them about SSL and TLS. That one is bigger than three, and one or two is actually hexed 303. Yeah. SSL, TLS, sometimes start TLS. I'm not getting into that, it's actually not that complicated. Um, but yeah. The history. Back in the old ages, Netscape 1993 and 1994, they wanted to start e-commerce, sending like credit card information over the internet, and they figured out some way to encrypt that data. So they came up with SSL 1.0, which was never released. Bit later, they came up with SSL 2.0, 1995, the first protocol that allowed encryption on the internet as a standard and back in the Netscape navigator, which was not that good. So they figured out, oh, now we have to redesign the whole protocol to make SSL version three. But at the same time, uh, the ITF, a standardization body, had also wanted to make that an official standard that works across multiple platforms and browsers. So they came up with the idea, let's standardize that, but let's give it a different name and restart the version numbers. It's only a very minor improvement a couple years later, but they came up with a new name. And actually, TLS 1.0 is SSL 3.1 internally. Um, couple of years later, TLS 1.1, which, uh, mitigated some attacks. Uh, 1.2 was a big step, uh, 10 years ago, they added, um, AED and better a PRF. So the way how certain encryption works is now much better than TLS 1.2 than the older ones. And eventually, soonish, hopefully, for four years now in standardization, is TLS 1.3, which hopefully comes out in a couple of months from now. Uh, the standard is, I think, finalized now, but they still had, like, the last two years to travel with certain hardware boxes. There's several TLS SSL libraries, you may encounter. For us, it's mostly important the Open SSL library. The other libraries are used, like, in Windows and Apple products, but, uh, Pi's usable mostly interact with Open SSL throughout the standard library as a model or to Pi Open SSL. Um, yeah, these are the most important libraries. There's also, written by Cory Benfield, a secure transport for YOLO 3 used in requests on macOS to use the Apple, um, uh, third database internally. And just a quick look how it looks when you actually create a connection. And as with the other model, you first need to create some configuration space that holds information, which kind of particles you want to do. That's the context. You can do that that way or you can do it manually. Uh, the first way is the preferred way, which gives you always a nicely secured configuration. We will also update that in case there's some kind of attacks. And then you have to create first a connection, later wrap the connection and actually do the handshake and then you can start, uh, to send data over the wire. And now I will run you through what happens at the different stages here throughout the whole, uh, rest of the presentation. So, high level view. What happens if you do any connection? So, uh, how does a protocol work? The first thing that most people will see if, if you connect, you get this nice shiny green lock symbol. That means connection is secure. And people think often that TLS SSL is mostly about encryption, but actually it's much more than encryption. Um, it's a very long presentation, a definition by Wikipedia, uh, have added some of the keywords. It's communication security, privacy, and also data integrity. So TLS SSL will not give you encryption, but also integrity of the data. So nobody can modify and temper with the information and also give you strong authentication and, um, of the peer. So the core features of TLS SSL are, um, encryption, what we, most people already figured probably out. It's also interesting to know that protocol agnostic, you can run much more than just HTTP. You can run like email, you can IMAP, you can run your database connection over TLS SSL. And sometimes you use, instead of TLS SSL, something called start TLS, that just means you first create the connection, do some communication, then upgrade your connection to encrypted one. So let's start TLS. You also get the integrity check so that nobody can change your data. You get something called replay attack protection and take an account like record your bank transaction and send that a couple of time again later on to the bank to get more money from you. It's also protected. Uh, so authentication of the server, that's the one thing you get with the certificates and this also shine greenie bar. And for enterprise users, if you like smart cut authentication or, uh, server to server authentication or for application backends, you can also use client certificates on the, on the client side. So the client can authenticate via the same kind of certificate as server certificate. Yeah. And it's extensible, which came very in handy for TLS 103 because it builds a partly new protocol on, um, the actual old protocol. We got lots of different standards, just a couple of buzzwords, uh, you will see throughout the next slides. And all this combination of different protocols and standards and network layers makes it rather hard to debug and understand what's going wrong. So it can go wrong on multiple layers, starting from the network, uh, crypto can go wrong, the certificates can go wrong. As in one is a rather complicated standard that the format certificates I encoded in and yeah, it's a bit of pain, but hopefully I will able to explain some of the issues I see both in daily life with maintenance as a model and at my work and help you to understand. So first of all, connection. First thing we do, we, uh, have to talk to the servers connected to the server somehow. And during that connection, the initial handshake, we do three vital steps. First, we have to agree on some kind of parameters to like which protocol version we talk, which Cypher's used to talk and we have to authenticate the server so nobody can do a manual middle attack. So there's no attacker that can just pretend to be like your bank or your email provider. And the last thing we need to do the actual encryption later on is agree on something called the pre-master secret. That's a shared secret that's then used to derive more secrets to do encryption and protection on the wire. So, first thing, we have to look up the name, do a, uh, GCP handshake to establish a connection. Then the client sends something kind of, hello. The server answers with another hello and some other fields I will fill in later on. And finally then the client sends something back, they do a bit of handshake. So you see it's like ping pong, ping pong and only at the very end, we start the actual encrypted communication. Um, so let's make Tila's a bit slow if you use that way. It's also optimized until 203 or if you use session resumption, which is a bit advanced topic, I can't talk during my talk. So first step, just establishing this initial, um, TCP handshake, looking out the name and sending the initial client hello, that can go wrong multiple ways already. So just get like a name, service unknown error usually means your host name is unknown. So just maybe some kind of DNS issues made a typo. There's also connection refuse, maybe the service down, maybe the network is down. So that happens on the TCP level. Sometimes you get the error message wrong version number. Um, I'll come back to that later. Sometimes just, you send something to the server and just kills the connection error number zero, which is rather annoying. Or in some cases, uh, it's just blocks hangs and does nothing. It can be a whole different list of issues from firewall issues to network connectivity to some servers. Even if you send something, they don't understand like new protocol version. They're just block, they're all playing nice. To get the hang of it, to see what's going on and to diagnose the first kind of issues, I always start to first like look up the host name, connect to resolve it, maybe send a ping to the server. So ping just sends a ICMP package to the LAPA replies, uh, protocol comes from submarines. So the tent is ping and hopefully it's over replies for there. It also helps to see the basic network connection works. Uh, you can like trace routing, uh, that pings to different steps. So the routers and servers, the request routed through that may help. And if nothing else works, you can still use a tool called TCP dump to look on the raw network layer, which is kind of a lot of gibberish and hard to read. I can't read that myself very well, but there's all the tools I'll show you later that is very, uh, makes it much more approachable. So one tool you can use to diagnose the kind of issues when you run on a server that I discovered just like a year ago is who has the heart of the tool curl, the downloader tool? Uh, it's just a, uh, common tool, uh, tool to download resources also has two options, um, dash s dash V, which gives you a enormous amount of information about the handshake. You see all the steps in the handshake on the console. So, so this tool is just to summarize them up. Um, Dina's look up ping, wire shark, corollate W get. There's also NMAP pod scanner. You can figure out which pods are open or the firewall is blocked or, uh, very, uh, if you have access to the server, just read the firewall locks, read the server locks. If you see any kind of errors or if you have a HTTP server, well, just use your browser. Just try to connect to your servers with your web browser and see if that works. Okay, we have the TCP connection out to the server. He sent the initial hello. And let's see how the handshake works internally. So again, we have the DNS look up. We sent the client hello. The client hello, this package we sent as a first tells the server which kind of versions we can talk, which ciphers we could support, um, and some random value. And the server, when it kind of handled the information, um, sends back the server hello, um, it sends back the information which ciphers selected, which version is selected and also random value. And from there on, we go on. And we had that a couple minutes ago, uh, either wrong version number connection hangs. If, uh, the server doesn't reply with a server hello or replies with this information wrong version number, that often means you may not actually talk to something that's an SSL or TLS server. Maybe you have misconfigured your Apache or an engine web server to not actually respond to HTTPS. It's something that is, uh, rather common if you start to deploy a new server and something that's not very obvious. Several other error messages you may get from OpenSSL, um, fail to negotiate acceptable TLS parameters, handshake failure, that can mean that any kind of this information what you support as a client is not understandable by the server. And there's time for policy, the server tells you, I don't like to talk to you with these parameters. There's no way to get more information directly from the handshake because it just sends this generic error message back, which is a bit of pickle, but there are other ways to get more information. Um, another unsupported protocol that may mean that the server doesn't understand the TLS protocol, but sometimes when it doesn't understand the TLS protocol version also send the first error message. Uh, again, it was error. It doesn't know what you're doing, maybe just kills you, uh, kills a connection. And, um, if you use your version of Python, do you have different kinds of protocols on the client side? Maybe you selected the wrong one. So, handshake failure. That's internally version um, uh, error number 40 means, for example, you can't agree on a common ciphersuit. The list of ciphers you can speak happens rather often if you update OpenSSL to a new version or install a security update, it disables old ciphersuits. Unsupported, that can also mean sometimes, uh, a couple minutes ago, a minute ago, that the server doesn't know your TLS version or that several of the other parameters you sent to the server is unknown, not understandable. So, you got the error number 40 in a whole bunch of possible ways it can go wrong. And the only way to figure out what goes wrong here if you get that error is, is actually use a tool that just tries different combination of parameters over and over again to give you a hint which kind of ciphersuits and TLS versions your server supports. One is a common line tool called SSL scan. So, every line you see here, like this reject, fail, any time you get this line it means it creates a new network connection, it configures just one version of TLS SSL once, once ciphersuit and then gives you the kind of error message. So, and that usually creates like a couple of hundred connections which looks a lot of ugly on the website, on the web server locks, but it's the only way to figure out what's going on. Or, if you can have a public service, I totally live this website, it's SSL Labs, so make small screen recording, it gives you very nice diagnostics and so here, testepython.org gives you some grading, tells me how the certificate chain looks like, which protocol version I support and it even gives me not only a list of all the ciphersuits, but also different version of Android, Windows, macOS, different browsers and tells me what they were able to negotiate as best selection of ciphersuits. Also, it tests for several known attacks and gives you a list and which kind of vulnerabilities your web server has or which countermeasures you have to take. Okay, we talked a bit about the ciphersuits. I'll give you a very, very short introduction, like three minutes in crypto to give you some of the steps we have in ciphersuits to explain how the ciphersuits are built up. And you also need that later on to understand how certificates work because both for the handshake, verification of certificates and the chaining, the signature of certificates, we need public key crypto. It's a small, very nice key way to explain it. So public key crypto means you have two different parts. You have a public key, something you can just hand out to everybody, you have a private key. That's why it's also called asymmetric crypto. Basically, three ways to do asymmetric crypto, the two ways, one way is you can use a private key to sign a document or actually the fingerprint, the hash of your document, and people with a public key are able to verify that you have signed the document, not some attacker that's on your right side. Yes. Or on the left side is if somebody can use your public key to encrypt some data and only the owner of the private key is able to revert that encryption, decrypt the actual encrypted data. So asymmetric encryption, there are a couple of algorithms, the red ones are the bad ones you shouldn't use. There are also, yeah. Thing is about asymmetric encryption is it's extremely slow. It's a very complex, very lengthy mathematical operation that only can handle a couple of bytes and it only takes lots of CPU cycles. If you don't like, like large images, videos over the internet, computer games, you don't want to spend that amount of CPU time, so we need another way to encrypt other information that's called bulk encryption. Probably heard of them, so these CPUs either AES or Chacha 20, again the RC4 triple desk or the desk one are the old ones, and these typically take like a couple of CPU cycles, provide or even CPUs these days like if you have modern Intel AMD or a ship you have dedicated CPU instructions to make AES even faster. So we also need to sign documents or sign certificates. You can use AES for that or there are several kind of new elliptic curve algorithms. Elliptic curves operate with different kind of math. This is usually faster to operate and make smaller keys but are secured too. One very important thing, since signing is also very slow, you have to first hash your document, make like a fingerprint and then you can sign the hash. Dash is very small, usually like 16 bytes or so, and you can hash like very lengthy documents and then sign this very small hash. And there are also several hashing algorithms, MD5 and Chawan are very insecure don't use them these days. Mostly if you have TLS as an L or certificates it's a chart too. So and the third way to do asymmetric crypto is something called a key agreement protocol or a so-called Diffie-Hellman operation where you can use some tricks to agree on a common secret without sending the secret over the wire with this kind of operation. So the key piece explains that it was mixing colors. You have your private color, you have a common public color, you mix your private color with a common color, send it over the wire. The other side adds the same, its own private color to the mix and they both have the same color at the end. And all these bits and pieces build up this so-called ciphersewed. So if the key agreement to agree on a common pre-master secret, you have something to authenticate the peer, you have the fast block encryption which also sometimes needs a mode of operation for block ciphers and finally the hash functions you use to verify some of the parameters. And that's the cipherseeds. So you can see all that, so just go through the first line to keep in time. The first one is a IPTIC curve Diffie-Hellman agreement. Use also IPTIC curves to authenticate the server. AES with Gullau's counter mode as block cipher and SHA-384 to verify the connection. That's the official standardized name from the IANA. But if you ever used OpenSSL or configured cipherseeds in Python, they look totally different because, well, OpenSSL decided, why use standard names? Let's count with our own rules and all names. They map rather funny, so the cipherseed maps to that very short name because they figured out they can just omit lots of bits and pieces for common things. Some map to slightly longer names, you can actually recognize the different bits and pieces. Yeah. You can use the new version of Python, you think Python 3.6, you can use the get cipher tool to configure a cipherseed and then get a list of supported ones. If you work with some kind of governmental bodies, you may also run additional restrictions with cipherseeds called either FIPS, GOST, or the Chinese one. These are standards you should not actually use unless you have to work either for the US government or for the Russian government or the Chinese government or have restrictions on the market you sell. So typically, Python developer or engineers ignore these, especially don't look into FIPS mode, FIPS mode is a mode to run OpenSSL, you don't need that. So you can essentially part two just to grab up what he has learned so far and add that to the mix. So we sent the client hello, the server sent back, the server hello also sends back the certificate chain, which will be the next part of my presentation. And in the RSA handshake, the old one we used, the client would just select a secret encrypted with the public key from the server as part of the certificate chain and then send it over to the server. The server can then use its private key to decrypt the pre-master secret. And finally, they changed CipherSpec, that means they switch over to encrypted communication. This variant had one very big issue. Since the client selects the private, the shared master key and send over the wire, if an attacker with lots of hard disks would just record all this information, like for years and years and years, and then somehow get the private key, the tech would be able to decrypt all traffic for the last decade or even longer. And there are some entities who have big, big space somewhere in the desert of wherever. So that's why we these days use a different way. We use a Diffie Helman key exchange. That means the server sends back some parameters and sends back his public key to the client and uses his private key of the certificate chain to sign that. And the client can then verify this signature with the public key. And it also generates on Diffie Helman private public key, or sends the public key as the final message. And both have then agreed on a private key. So that's a formal Diffie Helman key exchange with the very important step. It's perfect forward secrecy. So it's, since they throw away the keys of each operation, these handshake, I can't record that. Although, actually in TLS one or two, it's not yet secure. There are actually ways to crack that with session resumption. But that's fixed in TLS one or three. That we'll talk for next year. In Python, you can't get that many information. So I'm now used the open as a command line tool to make a connection with the Cypher suite, the modern one. And that's the certificate chain you'll get. I'll come to that in a minute. Some other information you will get to see which kind of connection it made. And it's a bit hard to read. You don't understand what's going on on the wire. But there's one tool you can look on the wire, what's going on. It's called a wire shock. And I'll show you how cool it is. Another recording. I made a rule to look for port 443, HTTPS Python, made a connection with that command line tool. We're now filtering out. Just interested in the TLS communication. We created a connection, TLS 1.2. The wire from our TLS 1.0. But that's not that important. We sent also the host name we're interested to talk to. And the server applies. With the server hello. We, yeah, select the Cypher suite. We select some other parameters, elliptic curve. We got a certificate chain back. So this is a certificate we got. We sent two back. The entity search, an intermediate search, which is lots of additional informations. We also now have the key exchange. So you see the Diffie-Hulman key exchange, elliptic curve Diffie-Hulman. That's a public key. There's the signature with the private key of the server. And then the client verified the information. Also the client key exchange sent its public key. And at that moment we switched to encrypted communication. That's the handshake. And this is one of my favorite tools to look into communication. But you need a GUI program for gravity user interface. Which makes it not suitable for servers. But in the beginning I talked about TCP dump. You can use TCP dump to dump network communication in your server to a file as a root user. Download that file to your desktop computer and use that tool to diagnose that. Okay. You have like 12 minutes left. It's enough for certificates. So we have the handshake. We have the pre-master secret agreed. Now we need some way to verify that we talked to the right server. Not to some kind of attacker. Mail in the middle attack. And the whole verification process is done with something called certificates. Or public keyings for PKI. The standard behind PKI, the other very old standard from the 80s. It's called X5 online certificates. X5 online are encoded in all the old standard called ASN1. With different representation. Most of you have seen, if you have looked at a certificate or private key, something that looks like base 64 encoded data was the begin and head up lock. That's just actually base 64 encoded raw ASN1. And these certificates always come in a pair of certificates and private keys. One way to look at certificates. Oh, there's also additional information I run through in a minute. To look at a certificate, there's a tool for text-based consults and OpenSLL2. So OpenSLLX509. We use the dash text option. You get the whole human readable text. We use no out. Then you don't get the certificate again. So it's a bit confusing, but yeah. And you get information like that. Certificates. So you get some data. First of all, they have a version number. These days, you will only see version three ones, except if you create your own one without certain extensions. They have a serial number. That's a randomly selected number. They used to be in serial, but there were attacks. And with random selected serial numbers, it's a bit more secure for the CA. Certificates are always signed by somebody. So if you sign a name in there, they're well for a certain period. With Let's Encrypt, you have three months. Yeah, give or take three months. There's also the subject. That's the name of the entity that requested the certificate. That's the certificate for the Euro Python website, which actually has not the common name. So CN stands for common name, EP 2018. Well, it's later embedded in the certificate. And finally, we have the public key with some additional information. That first block I showed you, that's actually the version one information. We have additional extensions, the X5.9 version three extension, then the version three certificates, which all certificates these days have. They have like a key usage, how to use the certificate. The signature means you can use the Diffie-Hellman key exchange and sign the secret. Key and assignment means somebody can use the certificate to encrypt the key. So this certificate you can use both kinds of encryption and signing. Without these, you wouldn't be allowed to sign. Other fields you can have on that is like signing CAs, signing revocation lists, etc. You also have an extended key usage, which typically is a web server or something like a web client. You can also have extended key usage for signing code. So that one is not a CAs certificate. It's an entity certificate. Identify us to build up the chain and some ways to get the certificate. OCSP is online certificate revocation protocol, status protocol. And finally, a list of names the certificate's valid for. And at the end, that one is signed by a CAs. That's a signature. So we have three different major types you see in web browsers. You have, first of all, the trust anchors, or trust anchors, or so-called root CAs. These are self-signed certificates that come pre- installed with a web browser or with your operating system. And these are fully trusted certificates. So then you have intermediate CAs. These are also CAs that are already signed by other one. You can have one, two, or just a chain of them. We use intermediate CAs to keep the root CAs locked away securely. So the intermediate CAs are typically in special hardware devices. And the root CAs, the keys printed out on paper and stored in a wall securely and not even on a hard disk for security reasons. And finally, the server certificate or the more technical terms and entity certificates. These form a chain. So the root CAs signed the immediate one, the next one signed the next one, and finally the entity certificates, the one you have on your machine. And you have to post the web server to send the whole chain back. So I just skipped through that. It's a different field you can have. It's going to be a bit more too much time to explain that. So the chain. As you can see here, another output of the L tool from other conference I presented the slides at. You first have your entity certificate for your website. Then you have the next intermediate one. And the root one will stay so on your browser installed. And just in case, just in case you wonder, sometimes you see this big green bar. That's called a standard validation certificate. This is just some additional fields on the certificate. They use an extra process to validate you, but these are not special certificates, other than they have this additional field. And one thing I want to point out, please keep your private keys secured and don't throw them like in the bin. So one of the most common things you can go wrong with verification is you get that kind of error message. The most annoying one. And too bad, if you still use Python 3.6 or Python 2.7, you won't get any more information back from Python. 3.7 is improved. I added some extra code, but you just get something went wrong. So you have to use an external tool called from OpenSSL, for example, to diagnose that. So it's SClient. You connect to the SClient. So you use bad SSL, which is a nice collection of error-nosed certificates or something is wrong. And for that case, I also have to send the host I want to connect to and I'll get error messages back. For example, here, certificate has expired. And I'm going through several of the error messages you make, come back, and give you some hints what can go wrong. So one thing is, obviously, cert has expired. So you have to get a new certificate somehow. Sometimes, more rarely, you get something like it's not valid yet that often means that your clock is wrong. So if you have learned like Raspberry Pi, we don't have clocks on the ship, on the die. You have to use an external clock so if something goes wrong, then you get that error message. If you generate your own ones, you may get some kind of very funky error message with handshake failure. These don't happen if you get official certificates. On the client side, if you connect to a server and have some kind of issues with the actual chain, there can be multiple issues you can have. The one if you run test certificates and you get self-signed certificate as error message, then you have enabled SL verification but didn't add the certificate to your trust store or had some issue with your test certificates. Unable to verify the first certificate in the chain is in almost all cases, I've witnessed that the server doesn't send you back the whole chain, only the entity certificate. So it's slightly misconfigured. Browsers can work around that. You can use information from the certificate to look up the chain and build that on their own. It's called AA Chasing. Tools like Python and any kind of command line tools really don't do that so you have to fix your server. Unable to get local issues certificate is a sign that you don't have the root CA loaded. Perhaps it's not installed, perhaps you're missing the way you get the root certificates. So you have to load them somehow and their multiple place certificate can be stored. Or sometimes you get the also self signed error message which is often a sign that the server sent all the root CA back but you don't trust the root CA. So the server can optionally send as part of the chain also root CA but if you don't trust it then it looks like this self signed error again. And if you update to Python 307 well you get actually a much better error message. Actually get the same information you would see in OpenSSL so here for example unable to get local issues certificate. You're welcome. I was annoyed too. I was very annoyed and it wasn't that hard to add that. I want to figure out how to do that. So root CA's. Root CA's can be stored in different places depending on operating system or be not available at all. So Linux or BSD is typically some file that's compiled into OpenSSL. So if you compile your own OpenSSL it may not work. Requests also have to certify package which just packages certificates which get rid of the error but have other issues. On Windows edit a hack to add certificates from the Windows search star which does not work properly if you have a newly installed machine. I didn't know that before. I want to edit that and on MacOS you can only have to install certify if you use the official installers because internally Mac, Apple use patch OpenSSL and we can't use that trick with Python.org installers. On Linux they may be, so vendors have never agreed on a specific location so you can have a long list of places you can look. We don't have that list in the SSL model in Python but you can use this small tool, small helper functions to get a list and also ways to override that. Can set environment variables. So I mentioned that I'm not a big fan of certify because you have to keep your certify package updated. If you don't do that you may run into some issues I found in New Relic a while ago that a very ancient version of certify and requests with very outdated invalid certificates. So just the final one, host name verification because I have just one middle left. So we also have to verify the host name matches the certificate that uses this subject alternative name extension and I have multiple issues to get that right in Python, the engineer, there's so many ways this thing can go wrong. We have like six major bugs, several CVs, several major bugs and now we use OpenSL for that, so OpenSL added this new feature. Also LibreSL added that until, yeah, I found a bug LibreSL was a work in Python 3.7 and that CV actually paid for some fancy medicine for my then sick cat. So thank you LibreSL, you paid the vet. And final remark is if you do HTTP you have to be sure that you use the same host name for both the server name indicator and the actual get request. So deals one or three out of time, yeah, out of time. So skip that. Summary, so some of the tools you can use and I recommend to get one of these books, especially the Bulletproof TLS-SSL and I'm right on time.