 intermediate level. So target audience would be people who have already executed an SSH command at one point in their developer or administrative lifetime, but have not delved deeper into the possibilities of SSH. So that's the target audience I'm trying to reach. If you already know more about the SSH config file, of course you're free to stay. Maybe there's a little nugget of knowledge that you haven't seen yet or haven't heard yet, but otherwise you probably will be bored just so that you want. So yes, the hashtag if you want to tweet or toot or whatever, it would be msh2022, and the slides will be available later on on my website as well as in the far plan, or the schedule, sorry, wrong event. I'm getting confused. I'm getting old. Before we start, just one word on command line syntax. I have a lot of command line examples. I'm also doing some live demos, so just let me know if the font is too small or something. I can modify a few things. In order to make it more readable for you in the slides, I'm following the POSIX 2017 standard that a new line followed by a backslash shall be interpreted as a line continuation. To give you an example of that, the command ls minus a minus l minus t slash temp could also be written as ls backslash new line minus a backslash new line, et cetera, et cetera. The right-hand notation is just more readable on slides. You can just compress it into one line once you execute them. Right. So should we get into it? I think we should, because otherwise I'm running out of time, but I think we're good on that. That's your usual command, right? That's what you do. If you do SSH on a machine, you have the command SSH, then you have the username, the et sign, and then the host that you want to reach. Once you enter that, whoops, not yet. Once you enter that, the server prompts you for the password or even wants to verify the fingerprint of the server and then asks you for the password. And if you then you do your work, you disconnect, later on you connect again, the server again asks for your password. You type this in over and over and over again. It's a little bit boring. If you're using SCP, for example, to copy files again, you have to execute SCP username and host name, then it asks for the password. That's rather boring and repetitive. Let's see if we can do something about that. This is, of course, the easy example. It could be a lot worse if you have a system administrator who's heavily into HP Lovecraft-inspired server names. You get server names like these, and I think you don't want to type this more than once, right? I mean, that's just calling for problems. Fortunately, this is by the way the god of Tzikronosh, and it's one of the greater old ones, in case you wonder. The thing or the SSH feature that we are going to use, and which I will spend a lot of time on, is called SSH config. So if you do a man5 SSH config, this is the information you will get. I mean, this is a lot more condensed than the man page. Basically, you can configure your SSH command, of course, through the command line options, username and host, and a few other commands and parameters. You can use a user configuration file. So this is a configuration that is applicable to four and to each user. Each user can have his or her own config file, and of course, as a system administrator, you can also provide a system-wide configuration file for the SSH client. In this talk, I'm going to focus on the user configuration file, which is hosted in the user's home page, which is symbolized by the tilde, and it's in a hidden directory called .ssh, and it's called config, surprisingly. What can we do in that config file? For one thing, we can define hostname aliases. So in case of the name of an elder god or something like that, you can just define the host demo, and then specify one or more hostnames it applies to. Oh, the other one. No, sorry, the other way around. So you have one hostname, your target, the server you want to reach is ssh-server.example.com, and with the host statement, you can give it one or more aliases, which is useful, which cuts down on the characters you have to type at the command line quite heavily, and you can not only have one aliase for the same hostname, but you can have several aliases for the same hostname, so you can have demo, d1, or popocut, the petal, if that's more your thing to type. And so instead of ssh-server.example.com, I just type ssh-d1, and that cuts down tremendously on the number of characters I have to type to reach that server. Furthermore, the hostname also has a nice feature to use a wildcard to support grouping similar hostnames. So if I have SMTP, IMAP, and WWW hostnames that are all connecting to the same or resolved to the same machine, I could use the %h feature to use the aliases as the hostname in the fully qualified domain name of the hostname parameter. Right, so that already... So instead of ssh-server.example.com, we can just type ssh-demo. So that's one thing we can cut down on typing. The other thing we can cut down on typing is, of course, the username. So I still have to type ssh-username at host or at alias. Fortunately, the config file also has a user directive, so I can specify a username I want to use for that particular host or alias, which cuts down on the amount of type tremendously. So compare these two statements. The first one is the one I showed you at the beginning. It's ssh-username at hostname, and now I just have to type ssh-demo, and everything I specified in my config file is therefore applied and I don't have to type it myself. So this is all sounds good, but let me show you how this actually works. So I am on my server, on my client, and I want to configure my ssh-config. So I start my preferred editor, and I'm going to my... Let's see if I can type while everyone is watching, and there will be a config file. So I'm using an editor, not an operating system like Emacs, and this is what we just talked about. I have my target server, which has the hostname of ssh-server-example.com, and I define two aliases here. One is demo, the second one is bastion, which I will need in a later example, and I specify my username for this setup, in this case LIDAR, and if everything works as I expect, I should now be able to do a ssh-demo, and I'm connected to the server, and I ask me for my password, and I'm connected to my server. So that would be, as I said before, a tremendous cut-down on the number of characters that we have to type, and as we are all system admins or developers, we want to cut down on the stuff we actually have to do. We much more prefer to wait for the compiler to compile or to the file transfer to complete. Right. So as you noticed, this cuts down on the connectivity, but the server is still asking for a password. So how can we fix this? Well, there's something called ssh-public-key authentication. So that helps us. It uses basically the public-private-key setup that we already know from TLS or other protocols to use keys to authenticate against the ssh-server so it don't have to provide my password every time I want to connect to a server. Additionally, the ssh-public-key authentication improves security considerably and also offers additional usability benefits like single sign-on across multiple ssh-servers. And another neat feature I will show you in a little bit once we got this all set up. The one thing you have to remember for SMMT-Crypto3, you probably know, but I want to emphasize this as I'm targeting maybe new developers or new system administrators, is there are two keys in play with SMMT-Crypto3. There's a public-key and there's a private-key. The public-key you can share with everyone. Sharing is caring. Share your public-key as much as you like. There's no problem with that. Put it on the server. Post it on your website. Send it through fax if you live in Germany. It's fine. On the other hand, the private-key is something you keep to yourself. That should reside on your laptop. That should reside on your PC. And you should not hand that out. You should not put it in a Git repository. You should not put it on a cloud store. This is your private-key and your private-key only, and it should not be shared. If it happens, create a new one. My preview just went out. Okay. This one is working. Okay. No worries. All right. We could do... So basically what it does is the server and the client exchange a few messages encrypting with the public-key. So as a user, if you want to use private-public-key cryptography to log into an SSH server, you have to copy... One time you have to copy your public-key to the server so the server knows your public-key. Once the server knows your public-key, it can then send you during the initiation of the connection. It will send the client a message encrypted with the public-key of the client. The client can therefore decrypt that message with its private-key. It will encrypt an answer that also includes the message that the server sent with the private-key of the client and with the public-key of the server, sends it back to the server and the server decrypts it with its private-key and checks that the messages were transmitted without change, therefore authenticating the client and allowing you to start to work. This is all working under the hood. It's using the Diffie-Hellman-key exchange so we know that it's secure and safe and it's properly implemented, at least in the more used distributions. It's really simple and it's a few commands and you're on your own way to go. What do you need for public-private-key cryptography or public-private-key SSH authentication? You need a key, of course. You need two keys, a public-key and a private-key. The command to create that would be, come on, SSH-keygan. SSH-keygan has quite a lot of parameters. These are the ones I regularly use and I usually recommend. You have the command SSH-keygan or key generator and then you're specifying the type of public-private-key cryptography that you want to use. In this case, I'm using Curve 25519. Oh, it's raining. It's 25519, which is an elliptic curve which creates highly secure but very small keys. This is really a nice feature if you're working with embedded systems and stuff like that. You're happily using an elliptic curve cryptography. Of course, you could also still use RSA keys. They're just 40,000-bit length. It's fine and works as well. They're just a little bit bigger. All modern systems usually support ED25519, so it's good. The A option is a pet-peaf of mine you can leave it out if you want to. That's basically defining the number of rounds that the algorithm computes when it creates the key. The higher number will result in a slower passphrase verification process, but it will increase your resistance to brute force password cracking of the private key in case it gets stolen. A security feature in case your private key gets accidentally uploaded to GitHub or something like that. I think we have to turn up the volume because the rain is getting really loud, thank you. The minus F option is there to specify the file name where you want to save your private key and your public key. You specify a file name, and the extensions will be added as needed. In this case, and I usually do that, I save it into my home directory, into the SSH directory, and I named it in this case demo.ed25519 because I do it. What I would highly recommend is to always, if you're using SSH key again, to always add a comment with the minus capital C option, because if you have multiple keys, figuring out which key belongs to what and which key applies to which server or to which customer can get quite tedious, so having a comment reminding me, oh, I created this for MCH, okay, I can delete it, or oh, I still need it or whatever. So having a comment in the key is always a very nice feature. So let's, no, let's, before we look at it, let's do this. Again, I have my command line and I create a key. As I said before, I have it here in the parallel syntax and here in the command line, I'm just using one line. You should always have a passphrase on your key. And of course, if you type in public, you never type the password correctly again twice. See, okay, three times a charm. Not talking and typing at the same time helps. And once the computer has finished its computations, you get an output like this, and let me show you a little bit more about that output. So that's the output, basically. And the first thing it is, it's telling you, okay, I'm creating an ED25519 key pair and it asks you for a passphrase. As said before, you should always have a passphrase that should no private key without a passphrase except for some very special circumstances where you should really know what you're doing. As we're talking about intermediate level setups, you should always have a passphrase for an SSH key. Once SSH key has finished its work, it also can generate, it tells us it generated two files. The demo ED25519 file, that's your private key. If it has the extension .pub, that's your public key. Please don't mix them up. Please don't upload the private key to GitHub or GitLab or another of these sites. These files are actually text files, so you can take a look at them. So let's take a look at the public file. The public file begins with the information which encryption algorithm, which asymmetric option did we choose? We chose ED25519. Then there's the key information. Again, this is the public key. You can share this freely. And the nice thing about it is at the end of the public key, the command gets added. So if you add this to a server, the administrator also has an option to see whose key is this, why is it there, so choose a proper command and it will your life and the life of your administrator a lot easier. I'm doing this here as it's a demo setup and I scratch it, but you should never do this in public. You can also, of course, take a look at the private key, which basically looks like this and resides on your client. Anyone with a copy of the public key can encrypt data for you, but only you, if you own the private key, can decrypt. So these authorized keys for you are really your thing and you should handle your private key very carefully. On the other hand, the public private key pair is free. They're free to generate. You've seen this takes literally a few seconds to generate once you have the command down. So please, please, please create more than one key. If you connect to different servers, if you connect to different customer servers, for example, have at least a key pair for each customer. If you're connecting to GitHub and to GitLab, please create a public private key pair for GitHub and a separate one for GitLab. Because what happens if you lose one of these keys? If you only have one public private key pair, all of the systems are compromised. If you have a public private key pair for each customer or for each service that you're going to use or for each server even that you're going to connect to, only that server is affected and only you have to make the system administrator of that server aware. There's nothing for... I'm doing a lot of consulting work. There's nothing more embarrassing than to tell the customer, look, I lost the key pair for access to another customer's servers, but your servers are also affected because they use the same public private key pair. That's not a nice thing to communicate. They are free. It doesn't cost much. A great one for each server, for each customer or each service you connect to. Sorry. So, that was the creation of the key pair. Just give me a second. I already mentioned that in order for this to work, we have to copy the public key and only the public key to the server we want to connect to. So, there are several options to do that. You can copy it through the clipboard. You can transfer it as a file, but that's a nice thing. There's also a command for it. So, there's SSH copy ID, which will allow you to copy the public key to a server that you want that public key copied to. Again, it's the same syntax as for all other services, and that, see, I missed the server name there. I will show you in the demo how this actually looks. And where do you copy this? So, SSH copy ID basically takes care of everything. You just tell it, take this public key file, copy it to that server. You don't have to think about directories, file names, et cetera. If you want to do it manually, you need to copy the demo.ed25519.pub public key file to the server into your home directory, into the .ssh directory and into the authorized keys file. This authorized keys file, if it does not exist, needs special permissions. So, if that file does not have 644 file permissions, the whole thing will not work. Again, SSH copy ID will take care of that for you and will create that for you so you don't have to think about that. That's why I'd like to prefer the SSH copy ID command over the manual creation of that file, and also in newer versions. So, if you're on, they'd be an old stable, probably not, but in newer releases, the SSH copy ID file also makes sure that you're not copying the private key to the server, so it's a nice security feature as well. Right. So, let's see if we can do that in my demo environment as well. So, again, the command here, is this readable or should I make it bigger? It's good. Okay. Great. The command is, again, SSH copy ID minus I and the ID file, the public ID file you want to copy, and then the target in the Now System demo. The nice thing about having a config file that we created earlier is these aliases that we created there apply to all SSH-related commands. We can use the same aliases for SCP, for SSH copy ID, and all the other stuff, so it's really nice. In this case, demo is our allies for Lyra at ssh-example.example.com, and it says, okay, yeah, I found the public key file. I'm attempting to log in. Of course, the public key is not uploaded yet, so it still asks me for the password. And remember, please enter Lyra at ssh-server.example.com's password. I enter that, and it tells me, hooray, successfully I connected to the server, I uploaded the public key to the server, and let's try again to log in. So, SSH demo, why is it asking for the password? It shouldn't ask for the password. Yes, of course, because I'm stupid. We still need to tell SSH that it should use the ID file to add the private key to connect to the server. So, let's try again, specifying the identity file that we want to use. Come on. I'm on the server. Thank you, that's a usual mistake I make. Yes, correct. Thanks for staying with me. Awesome. So, third attempt, third time's a charm. Yes, and now it's not asking, and if you look closely, it's not asking for the user's password, it's asking for the passphrase of the private key, which is a different thing, right? Yeah, and it's connected to the server. Hooray, thankfully. So, we now basically have established that we can, let me just log out here again, because otherwise... That wasn't good. That was one log out too many. Yep, all good. So, that's basically, we were basically verifying that the public-private-key authentication was successful. Of course, the next thing would be to modify our configuration file so that we don't have to pass the identity file, the private key, as a parameter while calling the SSH command. So, let's do that quickly, and that would be these two lines. Not SSH. Config. Let's do it properly. And we just add two additional lines. The preferred authentication's public key, it's basically just speeding up things. So, I'm telling the SSH client, don't even bother with username, password, just try public-key authentication and nothing else. And with the line identity file, I specify the file that I want to use for client authentication, public-private-key authentication. And if I save that, and now what I had in mind should work. So, we have the username, the target host, and the identity file that we want to use in the config. And again, it's asking for the passphrase, and I'm able to log in. Yes, Ray. Awesome. It's nice when something works. Again, preferred authentication's public key to speed things up and the identity file to use in that case. And again, public-private-keys are free. They are easy to generate, use a different identity file for each service or for each server that you want to connect to. And we said before SSH demo, and we don't have to type that much, which is awesome. Ray. Next question that usually comes up when I give this talk is, yeah, but we haven't improved anything. We still have to type in something. We don't care if it's the username or the password for the user or the passphrase for the private key. Yes, that's true. But there is a solution for that. So, we are now at the stage where we have public-private-key authentication to the server enabled, which is, of course, more secure, et cetera, et cetera. But how can we work around the issue that I still have to enter a passphrase for the key? And again, it's the passphrase for the key, not the user's password. Well, there's something called SSH agent. And SSH agent is a small program that runs on your machine, creates a socket, and waits for SSH to request the private key unencrypted to do its work. This is very, very convenient, so you basically start SSH agent, you do an SSH add to add that key to the agent. The agent keeps that key in memory and makes it available to the SSH client in case you want to connect to a server. So you only have to present the passphrase once. And after that, the second, third, fifth time, it does not ask for the passphrase of the private key again. That's the hardcore, low-level, command-line-only option. If you're using a graphical user interface, if you're using GNOME key ring or Kvalet, or if you're using a Mac, the SSH agent functionality is usually incorporated into these tools. So if you're using the GNOME desktop, usually the GNOME key ring decrypts or opens has all your SSH private keys unlocked. That was the word I was looking for, unlocked as soon as you log into your operating systems desktop. On macOS, that's the same for the key ring. You log into the macOS and all the SSH client certificates are unlocked once you're unlocked into the system. If you have high-secure requirements, if you have requirements for a high-secure environment, you want to rethink this, and maybe you will have to enter the passphrase every time. And in my advanced, advanced, advanced talk, we can also take a look at storing the private, the passphrase on a Fighter 2 device or use Fighter 2 and other stuff to do two-factor authentication, too many tools in there. But for now, let's take a look at SSH agent again if you're using a graphical user interface on your laptop and of course 2022 is the year of Linux on the desktop. So let's do that. Yes, one more thing on the SSH agent. You can, if you add a key to that SSH agent runs in the background and waits for a connect. And if you add your private key to SSH agent, you can give an option of minus C, which will instruct SSH agent to prompt you, do you really want to use that key now? This is their stopgap to make it a little bit more secure. But again, if you have high-security requirements, you might want to rethink the usage of SSH agent. So again, SSH agent is running in the background and I need to add my private key identities to that agent. So there's a command for it called SSH add. If you give it, if you don't, if you only give it the file name, it will add that key to the SSH agent key ring, minus D removes it, and as I said before, minus C will prompt you for usage every time you want to use it. So to connect without a passphrase, we're almost there. We only have to add our private key to the SSH agent. I think that's good. Oh, I really have to type this. Oh, my God. Let's do it again. SSH add minus E. Ah, the choice of switching between German and American keyboards. Yeah. And... So basically, I'm telling SSH add at the file, demo.ed25519, which is my private key to the SSH agent. Ah, yes. Hooray. Thank you, demo gods. Yeah, I know what the problem is. Let me just... Because I exited out of the session, so I lost all the... Let me first switch to the proper keyboard and then it's easier for me to type. I can type this from you, but apparently that was unsuccessful. Right, so let's try again. SSH add. Yes, and it's asking me for my passphrase. Finally, it works. Every command with SSH requires you to minus I if you specify the ID for the private key except for SSH add. So, SSH agent now knows about this private key and I already committed my passphrase, so the unlocked private key is now stored in the SSH agent memory. And if I do an SSH demo, we should see no password. Yay! One more thing. The nice thing about SSH is once you have this configured and running, what you, for example, could do is an SCP. So, SCP is copy, right? It's file copy. It's like a local copy, but over the net. And I want to copy something from the server to my local machine. So, I give it the alias. Again, the config file works here as well. And then I say, okay, I want something from ETC and now I do a double tap on the tap, on the tap key, the tabulator key, and I get remote tap completion, which is very nice and very convenient because I can't remember the paths and the file names. I'm old. I don't want to think about those. So, once you have public private key authentication set up, you will get remote auto completion, which, in my personal opinion, is worth the hassle to set it up once. Really, it's really, really, really cool, right? So, that was one of the benefits of having public private key authentication. Yeah, let's skip that. One thing I wanted to mention, ooh, need to speed up a little bit. One thing I wanted to mention is this is also available on Windows. So, if you're using Putty, you have basically three commands. You have PuttyGAN, which is your keyGAN for Windows. You have your Putty agent, which is your SSH agent, and you have Putty, which is your SSH client. And with PuttyGAN, what you have to keep in mind is there is a button there to say, which says, save public key. But that is saved in a proprietary format. What you need in order to get public private key going is actually the public key text that is displayed in the top of the Putty key generator. That's the thing you want on the server to actually work if you're connecting to an open SSH server. Other than that, it's completely the same at the bottom. You can specify which key to generate RSA, or in our case, 825519. And you will get a private key. You want to save the private key in this case. And if you start Putty agent, you can add keys here. Again, it's the same as in Linux, but you have fancy buttons and user interface for that. And within Putty, if you go to Connections SSH Auth for authentication, so it's a very convoluted navigation, you can actually specify the private key you want to use, and then you get the same functionality, the same benefits as on Linux if you have to use Windows because your customer only supports Windows as a desktop. There's also more. There's more SSH config magic. What you have to keep in mind for SSH config files is for each parameter, the first value obtained will be used. So you want the more specific stuff on top, and you want the more general stuff at the bottom of the file. Because it's not going to take the last thing, it's going to take the first thing it finds. If you specify something else below, SSH doesn't care about that. So you want more specific stuff at the top, general defaults at the end. How does that look like? So in this example, if I connect to... If I want to connect to a server like Web.prod.example.com, it will take my prod ID private file, the last one at the bottom. But for one server, which is unicorn.prod.example.com, it will not take the prod ID at the bottom, because I have a more specific thing above that, and it will take the pink fluffy ID as the private key for that particular server. And for all the servers that are within the test.example.com domain, it takes the test ID private file. This is something you can easily set up. And again, more general stuff at the bottom, more specific stuff at the top. To give you another example, this is again something from my demo here. We have our demo host entry with the host name and username and preferred authentication. Then we could have several more host entries, and then we have a catch all that is host asterisks, and that's basically setting additional stuff. What I usually want to have is compression, yes, because I have to connect to servers in the US and Asia, so compression does help. And the identities only is also a nice thing, because otherwise SSH will just try every identity file it finds to connect to the server, and you will get errors, and with identities only, it will only use the identity file specified in the host entry to use to connect to the thing. I do have to speed up a little bit. What you could also do is within the .ssh directory, you could have conf.d directory, where you can have multiple files, so you can separate the configuration out for each customer or each server you want to configure, and it pulls those in alphabetic order. Last thing I want to touch on, and that's why I'm skipping this a little bit, is bestian or chump host. What do I mean by that? You have your client, you have something called a chump host or a bestian host that is connected to the internet, and you have the actual server where you want to work on or where you have to do some work on that is not reachable from the internet directly or from your client directly, so you have to go through that chump or bestian host. What you usually do is your ssh into the bestian host, then you're logged in at the bestian host and from there, ssh to the target host, so basically what you do is ssh demo or bestian, let's keep it with demo, right? And then you do, where is it? Yeah, and then your ssh further on, and then your ssh further on to, and I'm using the same username here, I like to explicitly specify my usernames, target.local, which is a machine I can't reach from my client, target, of course, and I'm on my internal server, so that's what you usually do and it's quite cumbersome because you have to log in and log in and log in and log in. Server client, here I'm correct. So what you can actually do is you can specify something called a chump host, so with the option minus capital G, you can define a chump host or a chump host alias that you want to use to get to the target machine. So I want to connect to target.local, but I'm telling ssh to use bestian as an intermediate, as a chump host. Of course, everything that you can specify on the command line, you can also put into the config file. So in this case, I have my specification of my demo host alias and I have an alias for internal, and in there, in the host internal, I can specify proxy chump and the alias and with that, it knows, okay, if I just enter ssh internal, it will automatically use the bestian host to connect to that machine. So, do I have this prepared? No, that's bad. Just give me a sec. There we are. Yeah. So on my client.ssh config, I'll add this and if I go there, so I now enter the bestian host, so if I do an ssh target.local, no, just local, I created an alias. Internal, thank you. I created an alias called internal and it tells me, yeah, you're connecting for the first time, yeah, that's fine. And I haven't configured public-private key authentication for that as well. The nice thing if you configure it that way is that the first connection gets tunneled through the ssh. So you connect to the bestian host and then you connect through that connection to the internal server. So if you pass on the identity file, the ssh agent can talk directly to your internal servers. You don't need agent forwarding and everything else, which is a lot more secure than having agent forwarding running on that bestian host. But of course, it also works with password authentication. Again, if you have an older ssh configuration, you can also fall back to the proxy command. I'm just leaving this here in the slides as a reference. Please, please, please, please use the jump host directive. Use the minus j option. Do not use minus a or agent forwarding anymore. It's insecure. You shouldn't use it. It's outdated. The jump host is the better option. I will skip that. You can read on that if you want to. And that concludes my short and rather long talk. Thank you very much for your participation and have a great day. Let's do one. Thank you. The demo cards are not kind, but it helps to see that you make mistakes. We all make mistakes and then we correct them when we learn. We've probably got time for one or two questions . Good morning. Is it possible with the jump host to also forward local and remote sessions? Yes. You can figure it the same way. As usual. That would be my advanced talk. Maybe next year. Next time. I'm still around. I'm helping with tear down . If you have any questions, feel free to approach me. I will be around. Thanks very much.