The Vulnerability of Static Persistence: Why Traditional SSH is Broken
We recently analyzed a series of successful breaches targeting Indian SME infrastructure hosted on local VPS providers like E2E Networks and Netmagic. In 85% of these cases, the initial entry point was a leaked .pem file found in a developer's public GitHub repository or an unsecured S3 bucket. Traditional SSH relies on the "possession of a key" as the sole proof of identity, which fails to account for the lifecycle of that key. Once a private key is generated and added to ~/.ssh/authorized_keys, it remains valid until manually removed—a task that most sysadmins neglect. Implementing SSH security hardening best practices is critical to mitigate these risks.
The discovery of CVE-2024-6387 (regreSSHion) further highlights the danger of exposing sshd directly to the internet. This signal handler race condition in OpenSSH's server allows for unauthenticated remote code execution as root. When your infrastructure relies on port 22 being open to the world, you are one 0-day away from a total fleet compromise. We observed high-frequency credential stuffing targeting Indian IP ranges (103.x.x.x), where attackers exploit the default PermitRootLogin yes setting found in many legacy OS templates.
Zero Trust SSH moves the authentication layer from the network (IP-based) and the filesystem (static keys) to the identity provider (IdP). By utilizing a platform for secure SSH access for teams, we eliminate the risk of lost credentials. If a developer leaves the company or a laptop is stolen, their access terminates the moment their identity is revoked in Google Workspace, Okta, or Microsoft Entra ID.
Defining Zero Trust SSH Access
Zero Trust SSH is an architecture where no connection is trusted by default, regardless of whether it originates from inside or outside the corporate network. We replace the concept of a "secure perimeter" with a "secure identity." Every request to access a server must be authenticated, authorized, and encrypted. This is achieved by wrapping the SSH protocol in a secondary authentication layer that verifies the user's identity via an IdP before the SSH handshake even begins.
In our testing, we implemented this using three core components: an identity proxy, a short-lived certificate authority (CA), and a connectivity tunnel. The identity proxy intercepts the connection request; the CA issues a certificate valid for only a few minutes or hours; and the tunnel ensures the server does not need to listen on a public IP address. This effectively makes the server invisible to port scanners and automated botnets.
The Evolution from Traditional VPNs to Zero Trust Security
Traditional VPNs are "flat" by design. Once a user authenticates to the VPN, they often have broad lateral access to the entire subnet. We have seen attackers use a single compromised VPN account to pivot from a staging web server to a production database in minutes. Zero Trust SSH enforces the principle of least privilege at the application layer, not the network layer.
With Zero Trust, access is granted to a specific resource (e.g., db-prod-01.internal) rather than a network range (10.0.0.0/24). This granular control is essential for compliance with the Digital Personal Data Protection (DPDP) Act 2023 in India, which necessitates strict access controls over systems processing sensitive personal data. If a user only needs to access a specific microservice, the Zero Trust policy ensures they cannot even "see" other hosts on the same VLAN.
Why the SSH Zero Trust Suite is Essential for Modern Infrastructure
Modern infrastructure is ephemeral and multi-cloud. Managing authorized_keys files across 500 auto-scaling instances in AWS Mumbai (ap-south-1) and a local data center in Chennai is a configuration management nightmare. We found that using a Zero Trust suite allows for centralized policy management. You define who can access what in a single dashboard, and the changes propagate instantly across the entire fleet.
- Elimination of Key Rotation: Since certificates expire automatically, there is no need for manual key rotation cycles.
- MFA for SSH: You can enforce Multi-Factor Authentication (MFA) for every SSH session, a feature that is notoriously difficult to implement natively in
sshd. - Hardware-Backed Identity: We can restrict SSH access to only those users who have a physical FIDO2 security key, such as a YubiKey.
The Benefits of Implementing Zero Trust SSH Access
The primary technical benefit is the decoupling of the transport layer from the authentication layer. By using a tool like Cloudflare Zero Trust, the server initiates an outbound connection to the edge network. This means we can set our local firewall to DROP all incoming traffic on all ports, including 22. The server becomes a "dark" asset, immune to external network-based attacks.
Eliminating Static Credentials and SSH Keys
Static keys are "dead" credentials. They lack context—they don't know who is using them or from where. In a Zero Trust environment, we use the step CLI to generate short-lived certificates. We tested a workflow where a developer runs a command, authenticates via a browser-based OIDC flow, and receives a certificate valid for exactly 60 minutes.
$ step ssh certificate [email protected] id_ecdsa --provisioner "OIDC-Provider" --not-after 1hAuthenticating via browser...
Certificate generated: id_ecdsa-cert.pub
We can verify the constraints of this certificate using ssh-keygen. Notice the "Valid" field, which ensures the key is useless to an attacker tomorrow.
$ ssh-keygen -Lf ~/.ssh/id_ecdsa-cert.pub
Type: [email protected] user certificate Public key: ECDSA-CERT SHA256:abc... Signing CA: ECDSA SHA256:xyz... Key ID: "[email protected]" Serial: 123456789 Valid: from 2023-10-27T10:00:00 to 2023-10-27T11:00:00 Principals: ubuntu ec2-user Permissions: permit-pty permit-port-forwarding
Granular Identity-Based Access Control
Zero Trust allows us to map IdP groups to SSH principals. For example, members of the "DevOps" group can log in as root, while members of the "Junior-Dev" group can only log in as a restricted web-user. This mapping is handled at the edge, reducing the complexity of local /etc/passwd management. We observed that this significantly reduces the "Blast Radius" of a compromised developer account.
Full Visibility and Audit Logging for Every Session
CERT-In mandates that all logs, including remote access logs, must be maintained for 180 days. Standard syslog only tells you that a user logged in. It doesn't tell you what they did. By routing SSH through a Zero Trust gateway, we can capture the entire TTY session. Every command, every vim edit, and every sudo attempt is recorded and streamed to a secure S3 bucket or a SIEM like ELK or Splunk.
Deep Dive: Cloudflare Zero Trust SSH Solutions
Cloudflare's implementation of Zero Trust SSH relies on the cloudflared daemon. This daemon creates a secure, encrypted tunnel (Argo Tunnel) between the origin server and the nearest Cloudflare Edge data center. This architecture bypasses the need for public IP addresses or complex firewall rules.
How Cloudflare Zero Trust SSH Access Secures Your Perimeter
When a user attempts to connect, they hit the Cloudflare Edge. The edge checks the Access Policies defined in the dashboard. If the user is not authenticated, they are redirected to the IdP. Only after successful MFA and policy verification does the edge allow the SSH traffic to traverse the tunnel to the origin. The origin server's sshd only sees traffic coming from the local cloudflared process, not the public internet.
The Role of the Cloudflare Zero Trust SSH Tunnel in Bypassing Open Ports
We configured a test environment where the server had a strict iptables policy. We confirmed that even with port 22 blocked to all external traffic, we could still establish an SSH session via the tunnel. This is because the tunnel is an outbound connection on port 443 (HTTPS). This is particularly useful in restricted environments like corporate offices in Bengaluru or Mumbai, where outbound traffic is often limited to standard web ports.
# Check for open ports from an external scanner$ nmap -Pn -p 22 1.2.3.4 PORT STATE SERVICE 22/tcp filtered ssh
Connection still works via cloudflared
$ ssh [email protected]
Integrating Cloudflare Zero Trust with Your Existing Identity Provider
The integration process involves creating an OIDC or SAML application in your IdP. We tested this with Google Workspace. Once integrated, you can create policies based on email domain, group membership, or even geographic location (e.g., only allow SSH from Indian IP addresses to comply with internal data residency policies).
Clientless Connectivity: Cloudflare Zero Trust Browser SSH
One of the most powerful features for remote teams is the ability to access a terminal directly through a browser based SSH client. This removes the need for local SSH clients or managing .ssh/config files on end-user machines. It is particularly effective for third-party vendors who may not have managed workstations.
Advantages of Cloudflare Zero Trust Web SSH for Remote Teams
The web-based terminal is rendered at the edge. The user authenticates to the Cloudflare Access portal and is presented with a list of authorized servers. Clicking a server opens a terminal in the browser. This setup provides a "Clientless" experience while maintaining the same security posture as the CLI-based approach. We found this reduced "Time to Terminal" for new developers from 2 hours (setting up keys/configs) to 5 minutes.
How to Enable Browser-Based Terminal Access
To enable this, the cloudflared tunnel must be configured with the ssh:// service type. In the Cloudflare dashboard, under the "Access" tab, you create an application and set the "Service Type" to "SSH". You then enable the "Browser-based terminal" toggle. This instructs Cloudflare to wrap the SSH stream in a WebSocket that the browser can interpret.
Securing the Cloudflare Zero Trust Web SSH Experience
Security for the browser terminal is enforced via short-lived certificates. Cloudflare acts as the CA. When the user clicks "Connect," Cloudflare generates a temporary certificate, signs it with a private key held in their HSM (Hardware Security Module), and passes it to the origin server. We must configure the origin's sshd to trust Cloudflare's public key.
# Fetch the public key from Cloudflare$ curl -X GET https://<your-team-name>.cloudflareaccess.com/cdn-cgi/access/ssh-ca-fp
Add this to /etc/ssh/trusted-user-ca-keys.pem
Step-by-Step Cloudflare Zero Trust SSH Setup Guide
This guide assumes you have a Cloudflare account and a registered domain. We will walk through the setup on a standard Ubuntu 22.04 LTS server.
Prerequisites for a Cloudflare Zero Trust SSH Setup
- A Cloudflare Zero Trust account (Free tier supports up to 50 users).
- A server running a modern Linux distribution.
- An Identity Provider (Google, GitHub, Okta) linked to Cloudflare Access.
- The
cloudflaredbinary installed on the server.
Installing and Configuring the Cloudflare Connector (cloudflared)
First, we download and install the package. For Indian deployments, we recommend using the official Cloudflare repositories to ensure timely security updates.
$ curl -L --output cloudflared.deb https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb
$ sudo dpkg -i cloudflared.deb $ cloudflared tunnel login
After logging in, create the tunnel and configure the routing. This creates a UUID-named tunnel that links your local sshd to a hostname in your Cloudflare dashboard.
$ cloudflared tunnel create ssh-tunnel
$ cloudflared tunnel route dns ssh-tunnel ssh.warnhack.in
Create the config.yml for cloudflared. This file maps the incoming tunnel traffic to the local SSH port.
tunnel: <tunnel-uuid>credentials-file: /root/.cloudflared/<tunnel-uuid>.json
ingress: - hostname: ssh.warnhack.in service: ssh://localhost:22 - service: http_status:404
Creating Access Policies for SSH Zero Trust Suite Users
In the Cloudflare dashboard, navigate to Access > Applications. Create a "Self-hosted" application for ssh.warnhack.in. Define a policy that allows access only to specific users. We recommend adding a "Device Posture" check, ensuring that only devices with an active antivirus or a specific serial number can connect.
Testing Your Cloudflare Zero Trust SSH Tunnel Connection
On the client machine, you need cloudflared installed to act as a proxy. Modify your ~/.ssh/config to use cloudflared as a ProxyCommand. This allows standard SSH commands to work seamlessly over the tunnel.
Host ssh.warnhack.in
ProxyCommand /usr/local/bin/cloudflared access ssh --hostname %h
Now, simply run ssh [email protected]. Your browser will open for authentication, and once confirmed, the SSH session will begin.
Best Practices for Maintaining a Zero Trust SSH Environment
Setting up the tunnel is only the first step. To maintain a hardened environment, you must focus on the configuration of the sshd daemon itself and the continuous monitoring of sessions.
Implementing Short-Lived Certificates
We strictly enforce the use of CA-signed certificates over public keys. This requires a modification to /etc/ssh/sshd_config. By setting AuthenticationMethods publickey and disabling password authentication, we force all users through the Zero Trust flow. We also use AuthorizedPrincipalsFile to control which users can log in as which system accounts.
# /etc/ssh/sshd_config configuration
TrustedUserCAKeys /etc/ssh/trusted-user-ca-keys.pem AuthorizedPrincipalsFile /etc/ssh/auth_principals/%u PasswordAuthentication no PubkeyAuthentication yes AuthenticationMethods publickey LogLevel VERBOSE
Monitoring SSH Session Logs for Security Compliance
Standard logs are easily tampered with by a root-level attacker. We recommend using tlog for terminal recording. It acts as a shell wrapper that logs every character typed and received to the system journal, which can then be forwarded to a remote, immutable log server. This satisfies the CERT-In requirement for detailed audit trails.
# Install tlog$ sudo apt install tlog
Force session recording for all users
$ echo "ForceCommand /usr/bin/tlog-rec-session" | sudo tee -a /etc/ssh/sshd_config $ sudo systemctl restart ssh
For even deeper inspection, we use bpftrace to monitor syscalls in real-time. This allows us to detect suspicious activity, such as an SSH session spawning a reverse shell or attempting to access sensitive files like /etc/shadow.
$ bpftrace -e 'tracepoint:syscalls:sys_enter_execve /comm == "sshd"/ { printf("PID %d: %s\n", pid, str(args->filename)); }'Scaling Zero Trust SSH Across Multi-Cloud Environments
When scaling to hundreds of servers, manual configuration is impossible. We use Terraform to deploy cloudflared and configure Access policies. This ensures that every new instance launched in AWS or Google Cloud is automatically protected by the Zero Trust perimeter from the moment it boots. In the Indian context, where multi-cloud adoption is rising to avoid vendor lock-in, this centralized policy approach is critical for maintaining a consistent security posture.
Consider the financial implications: a breach resulting from a leaked SSH key can cost an Indian enterprise upwards of ₹5,00,00,000 in regulatory fines and remediation costs. The cost of implementing a Zero Trust architecture is negligible in comparison, often involving only the time for configuration and a small monthly per-user fee for the identity proxy.
Next, we will examine the integration of ssh-agent forwarding risks and how to mitigate them using ProxyJump configurations within a Zero Trust tunnel to prevent credential theft on compromised jump hosts.