Hey I'm in the middle of doing a very similar thing, though I didn't go quite as far as OP. I'm slowly deprecating my old RSA SSH keys and replacing them with a combination of Yubikey, Password Manager, and alternative SSH auth:
1. -sk keys on Yubikeys (with touch enabled) are my primary keys, used for signing and always installed on every host I touch/etc. I initially planned on using the PIV slots on my Yubi's to store the SSH keys instead of the built-in FIDO2 setup, but I hated having to manage multiple moving parts (like getting pscd running on all my clients). The only main benefit of using the PIV slots was being able to set the "cache" touch policy: the YK would cache a touch for up to 15 seconds so doing something like "signing multiple commits during a rebase" would only take one touch instead of num_commits. The side benefit would have been getting to re-use the same SSH key across primary and backup Yubis. The simplicity of the FIDO setup won out and I decided to treat getting annoyed at pressing my Yubikey a bunch of times as a sign I should be doing some sort of alternative setup like a dedicated key. Most prominently, I've now properly setup my Ansible configs with dedicated deploy keys kept as secrets instead of reusing my personal key.
2. I use 1Pass and Bitwarden (work/personal) to store a secondary SSH key that's only used for basic Git auth or other low-risk tasks. This SSH key is loaded from the Password manager directly into ssh-agent via stdin so it never touches my disk; I use fnox to make this backend agnostic.
3. Work uses Teleport, personal uses Tailscale so I rely on those SSH auth most of the time. Nothing much to say here except that Tailscale ssh is really convenient :) .
I've also setup PAM login and sudo on my machines, which brings me back to the old Sun Ray/Smartcard days. None of this is really to prevent security breaches/dedicated attackers or anything, though it certainly makes me harder to compromise. I just wanted to make absolutely sure I couldn't accidentally upload an SSH private key somewhere.
Bender 1 hours ago [-]
I will stick with ssh keys, except that I:
- Put the public authorized keys in /etc/ssh/keys/%u immutable, monitored for checksum changes.
- Have NFTables checks for MSS, TCP SYN TTL, Window sizes, CIDR blocks permitted on the public ssh instances (throw away nodes) and behind a VPN on more important roles.
microtonal 31 minutes ago [-]
Really nice article!
They mention twice that Yubikeys are expensive, but for SSH their cheaper FIDO2 keys work:
YubiKeys with proof-of-presence for SSH auth are great, but I still use the YubiKey's OpenPGP slot for this because my preferred behavior is:
1. Require a PIN followed by touch on the first use after inserting the YubiKey
2. For every following authentication while the key remains connected, only require touch
The extra GnuPG setup needed to make this work is unfortunate, but AFAIK this still isn't possible with fido -sk keys on the YubiKey.
aorth 1 hours ago [-]
Great article! I discovered ssh-tpm-agent last year and was surprised that all my ThinkPads have hardware TPM, even one from 2018! I've switched over to using it for everything and there are no more private keys on my devices.
I also have a Yubikey but it's been a few years since I tried it for ssh keys. I use it as a regular security key. Maybe I should try it for ssh keys as it sounds like the tooling has improved.
Foxboron 1 hours ago [-]
> You will notice though that the private keyfile held on disk uses a .tpm extension this time. This is where an important caveat of the TPM method comes in – Unlike the YubiKey approach, the private key material is actually being stored on disk here. This is because TPM devices do not natively support enrolling SSH keys. So what is happening is that the private key material is being stored on disk in encrypted form, and the TPM is being used to decrypt it.
While this is true, you can create static hierarchy keys that would not resolve into separate key files for the TPM. The issue here is that anyone with access to your TPM can recreate the keys, unless you set a owner password.
I like the idea of using my MacBook's Secure Enclave for ssh keys but what do I do if I develop on a remote VPS (purely because I'd rather have the AI accidentally rm -rf / on a VPS than my own computer)?
entrope 1 hours ago [-]
My sticking point with these has always been SSH agent support: the last time I looked, I couldn't use my Yubikey to authenticate the second hop in an A->B->C sequence with a jump box in the middle. Is that resolved now? This blog doesn't address that use case.
unsnap_biceps 27 minutes ago [-]
If you use ProxyJump (-J) you don't forward the agent off of host A and it works with completely arbitrary chains (A->B->C->D->E->F->...)
wang_li 24 seconds ago [-]
Except that's not really the network path that's going on, right? If I am on A and connect to B and then want to use the network to transfer a bunch of data from B to C over an ssh channel between them, using ProxyJump brings it all back to the bastion box. And if A is in the US and B & C are in Europe connected with a 40 GB LAN, using proxyjump will bring the connection from B->C into Europe->US->Europe. How do I do this with my credentials existing on A being used to log in to B and then the same credentials being used to connect from B to C while only having the one network connection is B->C and not to my bastion host?
stryan 1 hours ago [-]
If you're using -sk keys (i.e. setup via FIDO2 on the Yubikey like the article, not PIV) yes you can forward the keys loaded into your agent like normal, with ssh -A.
tomatocracy 13 minutes ago [-]
PIV keys can also be forwarded through an agent. The only thing to be aware of is that the agent process will effectively cache your PIN if you do this.
ssh-add -s <pkcs11.so>
To mitigate that you can add -c to the ssh-add command. That will make the agent prompt for confirmation (on the host with the PIV key) every time something asks to use it. Doesn't quite get you the same security as just not caching the PIN to start with but it's OK.
entrope 3 minutes ago [-]
Thanks to both you and the other person who replied. I appreciate the education.
makiniq0z 59 minutes ago [-]
There's also an alternative way to using ssh-tpm-agent - the tpm2-pkcs11 stack provides it, here's a guide for setting it up: https://jade.fyi/blog/tpm-ssh/
Not related to the author, but using that setup personally. Similar considerations as in the OPs TPM notes.
rednafi 30 minutes ago [-]
Way too much work for my taste. I use the bitwarden CLI to fetch/store the key. Opted in for ed25519 on day 1. Works great.
luz666 1 hours ago [-]
How are those compared to nitrokeys? I know an IT guy that prefers those, I think he mentioned those are open hardware?
buckle8017 1 hours ago [-]
Many people have issues with nitro key as a business.
Something about stealing an iPhone jailbreak years ago?
TacticalCoder 58 minutes ago [-]
Speaking of which I do something cool with my main desktop (an AMD 7700X, but whatever). It's running Debian and there's zero executable with the suid bit set. There's no sudo command installed. Using "su" to su to root is technically possible but it's like a 40 characters long password, meant to never be entered (and I never ever entered it using su).
The only way I can get root (short of using a local privilege exploitation exploit to gain root) is by logging, using a Yubikey, over ssh, from a dedicated laptop that has nothing but a SSH client. So the Yubikey is plugged into the laptop.
That dedicated laptop has no wifi drivers and is on a dedicated LAN and that dedicated LAN only has two machines on it: my desktop and that laptop.
The laptop serves no purpose other than: "log in as root on the desktop".
Of course both that laptop and the desktop have firewalls.
This means, yes, I allow log in as root using SSH. But password logins are disabled and the only way to log in as root using SSH is to use the Yubikey. And the firewall only allow SSH attempts to port 22 from the MAC address, IP, and dedicated LAN that corresponds to the laptop.
In addition to that my main desktop has multiple user accounts: but none of those users are allowed to su or sudo.
Now, I know, I could use Qubes or whatever.
But what I explained above is how I do it: even myself I cannot get root on my own desktop without my laptop + Yubikey (I'd need to find and enter the 40 chars long root password).
When I really need to do something as root (and once becoming root becomes extremely hard, suddenly you realize how rare are the occasions when you really need to be root), I open my laptop and boot it (and because that laptop has no GUI and basically nothing besides a SSH client, it boots in three seconds or so).
Haters gonna hate but that's been working totally fine for me since years and years.
skydhash 30 minutes ago [-]
Why not use:
1. Virtual terminals? The answer I can think of is usb hid attacks.
2. Serial Consoles? My answer the is encrypted connections on the wire.
Any other reasons than mines?
mrlonglong 4 days ago [-]
Can yubikeys be used for logging onto Linux consoles?
microtonal 29 minutes ago [-]
Yes, it's really nice. The fingerprint scanner on my Thinkpad is terrible (even when enrolling a fingerprint multiple times, etc.), whereas the YubiKey Bio work much better. So I use it as an alternative screen unlock.
ycombinatrix 4 days ago [-]
Yes, via PAM. I use the rssh module myself.
mrlonglong 4 days ago [-]
Nice thanks, I'm getting one to play with.
ycombinatrix 13 hours ago [-]
It's great, I'm completely done with passwords in Linux.
hoppp 1 hours ago [-]
Do you have a backup yubi key? So it's a good idea to buy 2?
project2501a 2 hours ago [-]
Thanks for the info, I have ubiqeys but i did not where to start!
hebetude 1 hours ago [-]
You can also store static passwords on yubikey which requires no direct hardware access. Using a prefix to protect them a little more
buckle8017 1 hours ago [-]
Unfortunately hardware resident keys are basically useless under qubes.
I'd rather have qubes than hardware resident keys.
Rendered at 18:57:40 GMT+0000 (Coordinated Universal Time) with Vercel.
1. -sk keys on Yubikeys (with touch enabled) are my primary keys, used for signing and always installed on every host I touch/etc. I initially planned on using the PIV slots on my Yubi's to store the SSH keys instead of the built-in FIDO2 setup, but I hated having to manage multiple moving parts (like getting pscd running on all my clients). The only main benefit of using the PIV slots was being able to set the "cache" touch policy: the YK would cache a touch for up to 15 seconds so doing something like "signing multiple commits during a rebase" would only take one touch instead of num_commits. The side benefit would have been getting to re-use the same SSH key across primary and backup Yubis. The simplicity of the FIDO setup won out and I decided to treat getting annoyed at pressing my Yubikey a bunch of times as a sign I should be doing some sort of alternative setup like a dedicated key. Most prominently, I've now properly setup my Ansible configs with dedicated deploy keys kept as secrets instead of reusing my personal key.
2. I use 1Pass and Bitwarden (work/personal) to store a secondary SSH key that's only used for basic Git auth or other low-risk tasks. This SSH key is loaded from the Password manager directly into ssh-agent via stdin so it never touches my disk; I use fnox to make this backend agnostic.
3. Work uses Teleport, personal uses Tailscale so I rely on those SSH auth most of the time. Nothing much to say here except that Tailscale ssh is really convenient :) .
I've also setup PAM login and sudo on my machines, which brings me back to the old Sun Ray/Smartcard days. None of this is really to prevent security breaches/dedicated attackers or anything, though it certainly makes me harder to compromise. I just wanted to make absolutely sure I couldn't accidentally upload an SSH private key somewhere.
- Put the public authorized keys in /etc/ssh/keys/%u immutable, monitored for checksum changes.
- Have CIDR restrictions in said public keys.
- Have NFTables checks for MSS, TCP SYN TTL, Window sizes, CIDR blocks permitted on the public ssh instances (throw away nodes) and behind a VPN on more important roles.They mention twice that Yubikeys are expensive, but for SSH their cheaper FIDO2 keys work:
https://www.yubico.com/product/security-key-series/security-...
1. Require a PIN followed by touch on the first use after inserting the YubiKey
2. For every following authentication while the key remains connected, only require touch
The extra GnuPG setup needed to make this work is unfortunate, but AFAIK this still isn't possible with fido -sk keys on the YubiKey.
I also have a Yubikey but it's been a few years since I tried it for ssh keys. I use it as a regular security key. Maybe I should try it for ssh keys as it sounds like the tooling has improved.
While this is true, you can create static hierarchy keys that would not resolve into separate key files for the TPM. The issue here is that anyone with access to your TPM can recreate the keys, unless you set a owner password.
https://man.archlinux.org/man/ssh-tpm-agent.1#Hierarchy_keys
ssh-add -s <pkcs11.so>
To mitigate that you can add -c to the ssh-add command. That will make the agent prompt for confirmation (on the host with the PIV key) every time something asks to use it. Doesn't quite get you the same security as just not caching the PIN to start with but it's OK.
Something about stealing an iPhone jailbreak years ago?
The only way I can get root (short of using a local privilege exploitation exploit to gain root) is by logging, using a Yubikey, over ssh, from a dedicated laptop that has nothing but a SSH client. So the Yubikey is plugged into the laptop.
That dedicated laptop has no wifi drivers and is on a dedicated LAN and that dedicated LAN only has two machines on it: my desktop and that laptop.
The laptop serves no purpose other than: "log in as root on the desktop".
Of course both that laptop and the desktop have firewalls.
This means, yes, I allow log in as root using SSH. But password logins are disabled and the only way to log in as root using SSH is to use the Yubikey. And the firewall only allow SSH attempts to port 22 from the MAC address, IP, and dedicated LAN that corresponds to the laptop.
In addition to that my main desktop has multiple user accounts: but none of those users are allowed to su or sudo.
Now, I know, I could use Qubes or whatever.
But what I explained above is how I do it: even myself I cannot get root on my own desktop without my laptop + Yubikey (I'd need to find and enter the 40 chars long root password).
When I really need to do something as root (and once becoming root becomes extremely hard, suddenly you realize how rare are the occasions when you really need to be root), I open my laptop and boot it (and because that laptop has no GUI and basically nothing besides a SSH client, it boots in three seconds or so).
Haters gonna hate but that's been working totally fine for me since years and years.
1. Virtual terminals? The answer I can think of is usb hid attacks.
2. Serial Consoles? My answer the is encrypted connections on the wire.
Any other reasons than mines?
I'd rather have qubes than hardware resident keys.