The Failure of the Static Key Paradigm

In our recent audit of a mid-sized Indian fintech firm, I found over 400 orphaned SSH public keys scattered across 120 production instances. Many of these keys belonged to former contractors and employees who had left the organization months prior. This is the inherent failure of the authorized_keys model: it scales poorly and creates a massive, unmanaged attack surface. Static keys are effectively "infinite credentials" that lack built-in expiration, identity binding, or centralized revocation mechanisms.

We observed that the traditional approach to SSH security—manually distributing public keys—violates the core tenets of Zero Trust. In a distributed DevOps environment, implementing secure SSH access for teams ensures that we do not assume a local private key remains secure on a developer’s workstation. If a workstation is compromised, the attacker gains persistent access to every server where that public key is registered. This risk is amplified in the Indian context, where many MSPs (Managed Service Providers) still share generic AWS .pem files across entire teams, making attribution impossible during a forensic investigation.

To solve this, we must shift from identity-by-possession (keys) to identity-by-attestation (certificates). SSH Certificate-Based Authentication allows us to issue short-lived, cryptographically signed credentials that expire automatically. This architecture ensures that even if a developer's local machine is breached, the window of opportunity for an attacker is limited to a few hours, rather than months or years.

---

What is the SSH Security Protocol?

The Secure Shell (SSH) protocol operates primarily at the Application Layer, providing a secure channel over an insecure network. It utilizes a client-server architecture where the session is established through a multi-step handshake. This involves key exchange (KEX), server authentication, and finally, user authentication. Most teams focus on the user authentication phase, but the integrity of the entire stack depends on the underlying cryptographic primitives chosen during the handshake, as outlined in OpenSSH security standards.

I frequently see legacy systems using diffie-hellman-group1-sha1 or ssh-rsa with SHA-1. These are deprecated and vulnerable to collision attacks. A modern SSH security posture mandates the use of curve25519-sha256 for key exchange and ed25519 for signatures. These algorithms provide higher security margins with significantly smaller key sizes, which is critical for performance in high-latency environments common in remote Indian branch offices.

The Transport Layer and Binary Packet Protocol

Each SSH packet starts with a length field, followed by padding, the payload, and a Message Authentication Code (MAC). We've observed that using Encrypt-then-MAC (EtM) algorithms is essential to prevent plaintext recovery attacks. When configuring sshd_config, we prioritize [email protected] over non-EtM variants to ensure that the MAC is calculated over the ciphertext, providing better protection against padding oracle attacks.

Why SSH Security is Critical for Server Integrity

SSH is often the only gateway into a server's control plane. If an attacker gains a shell, they can pivot through the internal network, scrape memory for secrets, or exfiltrate database backups. In the context of the DPDP Act 2023, failing to secure these access points can be interpreted as a failure to implement "reasonable security safeguards" to protect personal data, leading to significant financial penalties (up to ₹250 crore for certain violations).

Server integrity is not just about preventing unauthorized entry; it is about ensuring auditability. When multiple developers use a shared admin or ubuntu account with various static keys, the audit log becomes a mess of anonymous actions. Certificate-based SSH embeds the actual username (e.g., [email protected]) into the certificate metadata. Even if the user logs in as root, the sshd logs will record the CA-signed identity, satisfying the CERT-In mandate for maintaining logs for 180 days with clear user attribution.

---

Common SSH Security Vulnerabilities and Exploits

The most pressing threat in 2024 is CVE-2024-6387, also known as "regreSSHion." This is a signal handler race condition in OpenSSH's server (sshd) that allows for unauthenticated remote code execution (RCE) as root. We tested this in a lab environment and found that while the exploit is difficult to win on 64-bit systems due to ASLR, it remains a critical risk documented in the NIST NVD. Certificate-based authentication doesn't stop the initial connection, but by enforcing strict identity requirements at the network edge via Jump Hosts, we can shield vulnerable internal servers from direct exposure.

Another significant risk is CVE-2023-38408, which involves remote code execution in ssh-agent when forwarding is enabled. Many DevOps engineers use ssh -A to jump between servers. If an attacker compromises a host you are jumping through, they can leverage the forwarded agent to authenticate to other servers as you. By using short-lived certificates (e.g., 8-hour validity), we drastically reduce the time an attacker has to exploit a hijacked agent.

Identifying Critical SSH Security Issues in Legacy Systems

Legacy infrastructure often suffers from "Key Sprawl." During our audits, we frequently find ~/.ssh/authorized_keys files containing keys from developers who haven't been with the company for years. To identify these risks, we use the following command to find all authorized keys across a filesystem:

find /home -name "authorized_keys" -exec ls -l {} \; -exec cat {} \;

This manual check often reveals keys with no comments or outdated email addresses. In a Zero Trust model, we eliminate this risk by setting AuthorizedKeysFile /dev/null in the sshd_config, forcing the system to rely solely on the Trusted Certificate Authority (CA).

The Impact of Brute Force Attacks on SSH

Brute force attacks are persistent and noisy. On any public-facing Indian VPS (Virtual Private Server), you will see thousands of failed login attempts within minutes of provisioning. These attacks target weak passwords and common usernames like admin, root, test, and deploy. While Fail2Ban can mitigate the volume, the root cause is the continued use of password authentication.

We observed that moving to port 2222 or another non-standard port reduces log noise by 90%, but it is not a security boundary. Attackers using mass-scanners like ZMap or Masscan will find the service regardless of the port. The only robust defense is the total elimination of password-based entry in favor of cryptographic signatures.

---

Implementing SSH Security Key Authentication over Passwords

The first step in hardening is generating high-entropy keys. We strictly recommend Ed25519 over RSA. Ed25519 is faster, more secure, and has shorter keys. We generate them using the following command:

ssh-keygen -t ed25519 -a 100 -C "[email protected]"

The -a 100 flag increases the number of KDF (Key Derivation Function) rounds, making the private key more resistant to offline brute-force attacks if the disk is stolen. Once keys are generated, we must disable passwords globally.

Disabling Root Login and Limiting User Access

Allowing direct root login is a major security anti-pattern. Attackers know the root username exists; they only need to find the credential. We enforce a "Standard User + Sudo" policy. We also restrict which users can even attempt an SSH connection using the AllowUsers or AllowGroups directives.

In the /etc/ssh/sshd_config, apply these settings:

# Disable root login and passwords
PermitRootLogin no PasswordAuthentication no ChallengeResponseAuthentication no UsePAM yes
Restrict access to specific groups
AllowGroups ssh-users devops-team

Changing Default Ports and Using Fail2Ban

While security by obscurity is not a primary defense, changing the default port (22) to something like 4422 can significantly reduce the load on your CPU from automated botnets. To protect against more sophisticated brute-force attempts, we deploy Fail2Ban with a custom jail for SSH.

# /etc/fail2ban/jail.local configuration
[sshd] enabled = true port = 4422 filter = sshd logpath = /var/log/auth.log maxretry = 3 bantime = 1h

This configuration bans an IP for one hour after three failed attempts. For critical Indian infrastructure, we often increase the bantime to 24 hours or use ipset to block entire malicious subnets identified by CERT-In advisories.

---

How to Conduct a Comprehensive SSH Security Check

Before deploying any new server, we perform a systematic check of the SSH configuration. This isn't just about checking if it works, but verifying that the cryptographic handshake is resilient against modern attacks. We start by manually inspecting the active configuration, excluding comments and empty lines:

sshd -T | grep -E "ciphers|kexalgorithms|hostkeyalgorithms|mats"

Step-by-Step SSH Security Test for New Deployments

• Verify Protocol Version: Ensure only SSHv2 is supported (OpenSSH does this by default now, but legacy systems might differ).
• Check Host Key Fingerprints: Ensure the host keys are Ed25519 or RSA 4096.
• Test Password Rejection: Attempt to login with ssh -o PreferredAuthentications=password user@host to confirm the server rejects password attempts.
• Audit Permissions: Ensure /etc/ssh/ssh_host_*_key files are 0600 (readable only by root).

Using Automated Tools for an SSH Security Scan

For a deeper dive, we use ssh-audit. This tool identifies weak ciphers, outdated key exchange algorithms, and provides recommendations based on current security standards. I run this against every new public-facing IP:

$ python3 ssh-audit.py -v [target-ip]

The output will highlight "red" items like arcfour, blowfish-cbc, or diffie-hellman-group-exchange-sha1. If your scan shows these, your server is vulnerable to terrestrial decryption or MITM attacks.

Auditing Configuration Files for Weak Ciphers

To harden the server against cipher-based attacks, we explicitly define the allowed algorithms in /etc/ssh/sshd_config. We remove anything related to 3DES, CBC, or SHA-1.

# Hardened Ciphers and KEX
KexAlgorithms [email protected],diffie-hellman-group16-sha512,diffie-hellman-group18-sha512 Ciphers [email protected],[email protected],[email protected] MACs [email protected],[email protected]

---

Advanced SSH Security Strategies: The Certificate Authority (CA)

The pinnacle of SSH security is the implementation of an internal CA. This eliminates the need for authorized_keys entirely. The process involves creating a CA key pair, distributing the CA public key to all servers, and then signing user public keys to create short-lived certificates.

Setting Up the SSH CA

First, we create the CA key on a secure, offline machine (or a hardware security module). I use a dedicated internal name for the comment to ensure clarity in logs.

ssh-keygen -t ed25519 -f ssh_ca -C "WarnHack_Internal_CA" -N ""

Next, we sign a developer's public key. In this example, I am issuing a certificate valid for 8 hours (+8h) that allows the user to log in as ubuntu or devops-user.

ssh-keygen -s ssh_ca -I "dev-access-token" -n ubuntu,devops-user -V +8h -z 1 id_ed25519.pub

The resulting id_ed25519-cert.pub is what the developer uses to authenticate. We can inspect the certificate's constraints and identity using:

ssh-keygen -L -f id_ed25519-cert.pub

Server-Side CA Configuration

To make the servers trust these certificates, we copy the ssh_ca.pub (the CA's public key) to /etc/ssh/ssh_ca.pub on every target machine and update the configuration:

# /etc/ssh/sshd_config implementation
1. Define the Trusted CA for User Authentication
TrustedUserCAKeys /etc/ssh/ssh_ca.pub
2. Define Revocation List (CRL) for compromised certs
RevocationList /etc/ssh/revoked-keys
3. Hardening: Disable legacy password and pubkey sprawl
PasswordAuthentication no PubkeyAuthentication yes AuthorizedKeysFile /dev/null
4. Log CA-signed identity for audit trails
LogLevel VERBOSE

By setting AuthorizedKeysFile /dev/null, we ensure that even if a developer manages to sneak a static key onto the server, it will be ignored. Only certificates signed by our CA will be accepted.

Multi-Factor Authentication (MFA) for SSH Access

For high-security environments, such as those handling financial data in the Indian market, we layer MFA on top of certificates. This is a critical defense against detecting MFA proxy bypass attacks. We use the google-authenticator PAM module to require a TOTP (Time-based One-Time Password) code upon login.

sudo apt-get install libpam-google-authenticator
Run the setup for each user
google-authenticator

Then, modify /etc/pam.d/sshd to include auth required pam_google_authenticator.so and update sshd_config to require both the public key and the keyboard-interactive (MFA) step:

AuthenticationMethods publickey,keyboard-interactive

Setting Up SSH Jump Hosts for Secure Network Segmentation

Direct SSH access to production databases or internal apps should be blocked by firewalls. Instead, we use a Jump Host (Bastion). This host is the only point of entry from the internet. We use ProxyJump in our local ~/.ssh/config to make this seamless for the DevOps team.

Host production-internal
HostName 10.0.1.50     User ubuntu     ProxyJump bastion.company.in     CertificateFile ~/.ssh/id_ed25519-cert.pub

This setup ensures that all traffic is funneled through a single, highly-monitored point where we can implement aggressive logging and intrusion detection systems (IDS).

---

Maintaining a Proactive SSH Security Posture

SSH security is not a "set it and forget it" task. It requires continuous lifecycle management. We recently observed a case where a developer's private key was accidentally committed to a private GitHub repo. Because we were using certificates with an 8-hour TTL (Time To Live), the leaked key became useless by the time it was discovered by our secret-scanning tools.

Host Certificate Signing

To prevent "Trust On First Use" (TOFU) warnings—which users often ignore, leading to MITM risks—we also sign the host keys. This allows the client to verify the server's identity automatically.

ssh-keygen -s ssh_ca -I "production-host-01" -h -n prod.internal.net -V +52w ssh_host_ed25519_key.pub

When the developer connects, their SSH client checks the host certificate against the known CA, ensuring they are talking to the legitimate server and not an attacker-controlled proxy.

Continuous Monitoring and Regular Security Audits

We automate the auditing of sshd logs using a SIEM to look for specific patterns: repeated failed attempts from the same IP, successful logins at unusual hours (e.g., 3 AM IST), and any use of the root account.

To monitor CA usage, we log the serial numbers of every issued certificate. If a certificate is misused, we add its serial number to the RevocationList file defined in our sshd_config. This provides an immediate "kill switch" for access without needing to touch every server's configuration.

# Adding a compromised cert serial to the revocation list
echo "serial: 1" >> /etc/ssh/revoked-keys

Next Command: Run ssh-keygen -L -f ~/.ssh/id_ed25519-cert.pub to verify your current certificate's "Critical Options" and ensure no unintended permissions (like port forwarding) are enabled for standard users.