 for one more second. Okay, welcome. Let's welcome our last speaker in this dev room, Jakub Jalan with the talk, Consistent KSS11 in Operating Systems. Hello, do hear me. Looks like it's working. So welcome to the last talk of today, of this year for them in Security Dev Room. I appreciate that so many of you stayed here for so long for the end of, today. So my name is Jakub Jalan, I'm software engineer in Red Hat, and what I work on is mostly open SC and smart cath related stuff in operating system, REL and Fedora. My talk is named very widely about Consistent KSS11 in operating system, but I will focus mostly on REL and Fedora, which is the thing where we implemented this thing. So before we go to the details, what we did, I will ask a few questions. Do you know what are private keys? Is there anyone who doesn't know it? So, do you know what they can be used for? Many things. They can be used for mail signing, mail decryption. Many people probably use SSH, so they can use for authentication. Many people are developers, so they use it for Git. It can be used also for signing Git commits, text, it can be used for TLS client authentication in HTTPS. And basically it's just more secure password replacement that cannot be, that doesn't travel through network and that cannot be copied on the way. But anyone of you do know where are your private key stored? Are they on your hard drive? Are they in your computer memory when you are using them or do you have some backup in cloud? Encrypted, unencrypted? Do you know it? I guess everyone saw these pictures somewhere. So, software and hardware has bugs and if some of them show, they might, they might make your keys available to any attacker and this is not something that you want, so that's good thing to update. But if there are some zero-day exploits, you don't know which zero-day exploits are around here and how can you protect against them? Yes, the answer is dedicated hardware which can store your private keys and that doesn't have any simple way how to retrieve the private keys from such smart card from Ubiqui or from HSM or other separated hardware unless there is back in there. But the API should be pretty strict to avoid it. Last year I talked, I had talked here of a similar beginning. I showed up some examples how PKCSL can be used in various applications but it didn't look like very easy for anyone, I think. So, this was one of the things that we tried to improve. This is like a large image of all the tools that are somehow related to smart cards in operating system in Linux. This is, this looks very complicated but what you should care about is the top boxes which show the applications that want to use smart cards and on bottom there is smart cards. And something between it should be like implementation that I'll detail for user, for system administrator, for application developer. So, the user wants to have it somehow like this. They talk to some magic and that magic works. So, as I said already we are dealing with PKCS11. I will try to explain what is PKCS11, how we try to improve its usage in operating system. Then I will go through applications, how they use PKCS11 and what we try to improve and what we improved. And if you are developer or you would like to use it in your application, I will show you some ideas that you might check. And let's go for it. So, PKCS11. PKCS11 is open standard for cryptographic tokens, controlling, authentication, information, personal identity, cryptographic key, certificate, digital signatures and so forth. Basically, as a user so far, you meet PKCS11 as a low level CAPI in way of shared module that is somewhere on your hard drive. It allows you to do operation with private keys, with read public keys, read certificates. If you want to learn more, it's open standard so you do not have to buy it as ISO standards and so on so feel free to check Oasis website which provides the standards in full length. So, I showed some pictures already. We want to have the PKCS11 consistent so we want to have it system wide for usage and configuration. There are some examples of the commands that I showed last year or in the previous slides. The one first is P11 tool from GNU TLS. It is identifying a smart card or something on a smart card as PKCS11 URAE. Then there is PKCS11 tool from OpenSC. It's identifying object by its ID, its type and it can have much more switches but it's something completely different than the first one. And for the third example is SSH and that doesn't have any way how to specify what key to use from the smart card or from your token. So, this is not consistent. We basically unified on PKCS11 URAE because they are the strongest and they can express any other possibility what was used in the wild. PKCS11 URAE has also advantage that is standardized URAE scheme. So in the places where usually people use files for private keys, they can use URAE scheme starting with PKCS11 and it is very easy to detect it in the software that is using the smart card. It can uniquely name each object on the system in the smart card. It can name also the tokens, readers, anything in that huge image that I showed previously. You can filter by many PKCS11 attributes that are available or you can use just PKCS11 scheme that will basically say your application use something from the PKCS11 from smart cards. That is basically how it worked in SSH. In the past, you got PKCS11 object. Now you can provide PKCS11 URAE and it will just find in the system what it wants. The other thing that is related to it is P11 kit. And it's related to the PKCS11 URAE because in this URAE, you don't see where is the PKCS11 module that you still need. So P11 kit provides a system wide store for all the PKCS11 modules that are in the system available. Installation is simple as putting text file into this directory. It's usually done by a package manager if you've got Fedora. This is, for example, the OpenSC PKCS11 module and all these modules are loaded automatically by any application that will be using either P11 kit or P11 kit proxy shared object for the application that already have some PKCS11 support. The advantage of it, this is also that if you've got HSM or some other hardware token that is not supported, let's say by OpenSC or it's not supported by any other tool that is in the system, you can plug it into the system by registry as any other and it works. So let's have a look how it works in applications. Did someone use OpenSSH with PKCS11 support? Few people are there. So this is how it looks like in OpenSSH upstream. Let's say it's release seven nine. First thing you need to do is using SSH key again with argument pointing to PKCS11 module and it will list you the public keys that OpenSSH finds in your module. It doesn't say what type of the public key is, if it is from certificate, it doesn't say how big is it, any label is nothing and other thing it didn't support ECDSA until recently. For authentication, you do the same thing. You do not want to provide PKCS11 module to applications if you want to use them. You want just to write PKCS11 something. Public authentication work the same way. You had a specific switch. I guess it shouldn't be I, it should be uppercase I, that's back and OpenSSH listed all the keys that it found there, tried them on a server and if one worked, it tried to authenticate you with this key. If you had more keys on the token, it used the first one that it found that worked and you were not able to choose any other. So we also don't want to write because SSH11 module on command line on configuration file anywhere. And sometimes you would like to filter the keys. So this is how it works with Federa 28 and later. You can use PKCS11 scheme here for the same thing. It also lists the PKCS11 Nury in SSH key again. So you basically know what is the key about if it is authentication key, if it is signature key, if it is the encryption key or if it is something else from the cart. For authentication, you can again use just your scheme and it will do the same thing. And if you want to use some different key, you use a rule like this, which will say that you should use the key number two or key ID two. The same thing works also with SSH agent. You can add any key from smart cart to SSH agent and then it will work from there and you can authenticate anywhere. The same thing will work also in configuration file, which is pretty handy. In this case, you've got also, you see somehow longer PKCS11 Nury that is using directly module paths part of the Nury that allows to skip the PLN kit proxy and directly loads the OpenSC PKCS11 module. But it's just a side note. So the same thing should also work for the HTTPS clients. Let's say we've got some scripts or some application that needs to connect to some API or to transfer some data using Wget or Carl. In the past, it needed like files that are stored somewhere on the hard drive, probably unencrypted if it was in script or it has the encryption key or pin or passphrase somewhere next to it. So in this case, we can provide again PKCS11 Nury for certificate and private key options and connect to TLS server using HTTPS and use TLS client authentication as from web browser. On the other side, there are the HTTPS servers. So for the users to have this nice shiny green lock, the server needs to provide again signature proof that the server is really the server that you want to talk to. And again, the same thing was if the keys get compromised, you can, the users might be talking to something different. This was one of the things that worked for hard bleed, if I remember well. So for HTTPD or NGINX, you can configure your HSM to be able to store private key on the HSM and work with them. For example, NGINX configuration file doesn't support this for the certificate yet, but it's not a big deal because the important thing that you want to have on hardware, on separate hardware is the private key that's the thing that you should care. Did anyone use Firefox with smart cards or UB keys? So you probably know this window. This is probably one of the older windows in the old design, not redesigned throughout the ages. It looks really ugly and it's again asking for clients or for users to write their like module file name. What users should know about module file name? If you know something about it, you can do it. If you don't know, you don't have to do it in Fedora 29 or something, maybe 30, NSS will be loading the P11 by default. So if the website requests to authenticate, request you to authenticate using TLS client authentication, it just should work. So we are getting to the end. So if you wrote some application or you want to write some application using smart card or using some hardware security modules or tokens, or your application might already work. If I didn't mention it here, it might not. If you are writing some application, then you should try to use like some high level crypto library. P11 kit is also a good thing to start if you want to write something with PKCS 11. And if you would be searching how to identify the things on the smart card, PKCS 11, you are saying, you are is something that I can recommend you. If you don't have any smart card or any token, you can try it at home. Most of the current computers have already a TPM tool and there is already a TPM tool, PKCS 11 project that basically allows you to use your TPM as smart card. It has disadvantage that the keys are tied to your machine. You can't move them around as a smart card, but most of the keys that you generate for SSH are for something else are usually left on your computer and you don't move them around. If you would like to do some testing, there is soft HSM software implementation of PKCS 11 that stores data on file system. So it's not more secure than storing the files directly, but it's good for testing. And that's all. So if there is something that you should remember, there are bugs in software and hardware that might expose your private keys. That's something that we do not want to, so to prevent this or minimize consequences, storing the private keys in hardware is a good idea. If you've got Fedora or if you saw Relate Beta, the things that I showed here are already there, so you can feel free to check it, test it and if something will not work, just drop me a mail, write me something. I would be glad for feedback. So I guess that's all. So there is five minutes for questions. Thank you. So you mentioned a couple of times that you can try it out in latest Fedora versions. So has this work been also upstreamed so that other distributions can benefit from it? The PKCS-11 URIS, I offered it upstream but it's not merged yet. Is there any blockers or is it just a matter of time? It's a matter of open BSD, I would say. Like, I've got the feeling that they, like during class five years, they were not at all interested in the PKCS-11 things that are in open SSH just two weeks ago or something, they merged the ECDSA patch for PKCS-11, which is a good thing, but to get them to merge the PKCS-11 URIS that I cannot guess how long it might take. It might be next week, but it might be in five years. And at least the remaining projects, are they still? Sorry? And the remaining projects beyond like open SSH and open BSD, sponsored ones. Like NGINX. NGINX, the other things should be already in upstreams. NGINX and HTTP servers. Any other question? How about GPGPGP? Have you, are you working on that? Is someone working on that? Yeah, that is also another pain point. The problem with GPG is that they do not use PKCS-11. They've got their SS agent or something that directly talks to PCSC as if we can go to this image, they've got something that talks to this thing. So we basically cannot affect it. There is also project that is SS agent, PKCS-11, but I wasn't able to make it working so far. So for GPG, any help would be obviously appreciated, but I didn't manage to make it working so far. So thank you for your talk. I'm not too familiar with P11 kit proxy, but would you be able to use several separate instances of PKCS-11 for example different smart cards to be used? I'm not sure how do you mean it? In the system there can be separate PKCS-11 modules, but using PKCS-11 proxy we unify them and provide everything in one module, which is easier for configuration and everything I said, but it doesn't prevent you from writing something like I showed here. Where was it? I showed here you can write PKCS-11 that doesn't have the module path part, and in that case you basically skip the P11 kit proxy and talk directly to this module. Yeah, okay. But that isn't like it's implemented in this way, but there isn't anything in the proxy, the P11 kit that does similar behavior like this. I'm not sure if there is not like it. I'm not sure if I understand the question correctly, but this is the way how P11 kits are used. PKCS-11 is written, do you want to say something? Sorry, I have no idea. Oh, yeah, yeah, yeah, it's actually bypass processing. Well, there are two ways how the PKCS-11 is evaluated. One way is, for example, as in OpenSSH, where is everything implemented in OpenSSH, and the P11 kit proxy is used only by default if you do not provide this module path. And the other way is going through the P11 kit directly, and in that case, you provide this URL to P11 kit, and then it returns you the results. And I'm actually not sure how it's implemented there, but I think if you provide P11 kit module path, it will talk directly to it, or it will skip it. Whether it directs from multiple cards in the P11 kit, yeah, yeah, it's in the space of a P11 kit as many as you want. Okay, yeah. With different libraries. Yes. Let me use different drivers. Yeah, that's it, okay. Sorry, I didn't understand it correctly. That's all right. So I guess we are out of the time. It's all right. One minute. One more question.