The Perpetual Noise of SSH Scans

In any production environment, the sshd daemon is the most targeted service. Within seconds of an IPv4 address going live on a public-facing cloud provider, I observe automated scanners initiating brute-force attempts. These aren't sophisticated actors; they are botnets cycling through common credentials like admin:admin or root:123456. However, the sheer volume of these attempts can obfuscate more targeted attacks, such as the signal handler race condition known as RegreSSHion (CVE-2024-6387).

Why SSH Log Analysis is Non-Negotiable

Monitoring SSH logs isn't just about security posture; it is about operational visibility. We use log monitoring and threat detection to identify compromised internal accounts, detect lateral movement attempts, and validate that our firewall policies are actually dropping traffic. In the context of the Indian Digital Personal Data Protection (DPDP) Act 2023, maintaining these logs is a critical component of "reasonable security safeguards." Failing to detect a breach due to unmonitored logs could lead to significant penalties, often reaching up to ₹250 crore for major non-compliance.

Distinguishing Log Locations Across Distributions

The first hurdle in SSH log analysis is knowing where the data lives. Linux distributions have diverged in how they handle authentication logs. On Debian-based systems like Ubuntu, we look at /var/log/auth.log. On RHEL-based systems, including CentOS, AlmaLinux, and Rocky Linux, the data is written to /var/log/secure.

# Checking log existence on Ubuntu/Debian
ls -l /var/log/auth.log
Checking log existence on RHEL/CentOS
ls -l /var/log/secure

The Shift to Systemd Journal

Many modern distributions are moving away from flat text files in /var/log and toward the systemd journal. If you find your log files empty or missing, you must query journalctl directly. I prefer using the -u ssh flag to isolate SSH daemon events.

# Querying the last hour of SSH logs via journalctl
journalctl -u ssh --since "1 hour ago"

---

Anatomy of an SSH Login Event

To build a parser, we must understand the structure of the log entries. A typical failed login attempt contains a timestamp, the hostname, the process ID of the sshd instance, the failure message, the target username, and the source IP address.

Breaking Down a Failed Attempt

Consider this log entry: Jul 24 10:15:32 production-server sshd[12345]: Failed password for invalid user admin from 192.168.1.50 port 54322 ssh2.

• Jul 24 10:15:32: The local system time of the event.
• production-server: The hostname where the event occurred.
• sshd[12345]: The service name and the specific Process ID (PID) handling the connection.
• Failed password: The specific event type.
• invalid user admin: Indicates the attacker attempted a username that does not exist on the system.
• from 192.168.1.50: The source IP of the attacker.

Successful vs. Failed Authentication

I look for Accepted password or Accepted publickey to identify successful logins. A sudden spike in Accepted messages followed by Failed password messages is a classic indicator of a successful brute-force attack.

# Identify successful logins quickly
grep "Accepted" /var/log/auth.log

---

Manual SSH Log Analysis Techniques

Before jumping into Python, we use standard Linux utilities to perform quick triage. This is essential for incident response when you don't have time to write a script.

Using Grep to Filter SSH Events

Grep is our primary tool for filtering. To extract a list of all IPs that failed to log in, I use a pipeline that filters for "Failed password," extracts the IP, and counts the occurrences.

grep 'Failed password' /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -nr

In this command, $(NF-3) tells awk to print the third-to-last field, which is consistently the IP address in standard OpenSSH security logs.

Tracking Real-Time Connections with Tail

When I am actively monitoring a server under attack, I use tail -f combined with grep. This provides a live stream of failure events.

tail -f /var/log/auth.log | grep --line-buffered "Failed password"

The --line-buffered flag is critical here; without it, grep might buffer the output, causing a delay in the visibility of the attack.

Analyzing Failed Logins with lastb

The lastb command reads the /var/log/btmp file, which records all failed login attempts in a binary format. This is often faster than parsing text logs for a quick overview of recent failures.

# View the last 20 failed login attempts
lastb -n 20

---

Leveraging Python for Automated SSH Log Analysis

Manual analysis fails when dealing with "low-and-slow" attacks. These are distributed attacks where each IP only attempts a few logins per hour, staying under the threshold of standard tools like Fail2Ban. We can build a custom Python parser to maintain state and detect these patterns.

Building the Regex Engine

I use Python's re module to extract structured data from the unstructured log lines. The following regex pattern is designed to capture the timestamp, username, and IP address.

import re
from collections import Counter
Regex to capture Timestamp, Username, and IP from SSH failure
It handles both "invalid user" and existing user failure strings
SSH_FAIL_PATTERN = r'(\w{3}\s+\d+\s\d{2}:\d{2}:\d{2}).*Failed password for (?:invalid user )?(\w+) from ([\d\.]+)'
def parse_ssh_failures(file_path):     failures = []     try:         with open(file_path, 'r') as f:             for line in f:                 match = re.search(SSH_FAIL_PATTERN, line)                 if match:                     failures.append({                         "timestamp": match.group(1),                         "user": match.group(2),                         "ip": match.group(3)                     })     except FileNotFoundError:         print(f"Error: {file_path} not found.")     return failures

Implementing Threshold-Based Alerting

Once we have the data in a list of dictionaries, we can count failures per IP. If an IP exceeds a specific threshold, we can trigger an alert or an automated firewall block.

def detect_brute_force(failures, threshold=5):
ip_list = [f['ip'] for f in failures]     counts = Counter(ip_list)     block_list = [ip for ip, count in counts.items() if count > threshold]     return block_list
Execution logic
logs = parse_ssh_failures('/var/log/auth.log') suspects = detect_brute_force(logs, threshold=10) for ip in suspects:     print(f"ALERT: Potential Brute Force detected from {ip}")

Handling Log Rotation in Python

A common mistake I see is scripts failing after a log rotation (e.g., when auth.log becomes auth.log.1). To handle this, your script should monitor the file's inode or detect when the file size shrinks.

import time
import os
def follow_log(file_path):     f = open(file_path, 'r')     f.seek(0, os.SEEK_END) # Start at the end of the file     while True:         line = f.readline()         if not line:             time.sleep(0.1)             # Check if file was rotated             if not os.path.exists(file_path) or os.stat(file_path).st_size < f.tell():                 f.close()                 f = open(file_path, 'r')             continue         yield line

---

Security Best Practices and Threat Detection

In the Indian context, we see a massive influx of brute-force traffic from compromised IoT devices. Many Indian SMEs use legacy RHEL 7 or CentOS 7 instances that are now End-of-Life (EOL). When hardening Linux infrastructure, these systems are frequently targeted by botnets originating from residential IP pools of major ISPs like Bharti Airtel and Reliance Jio.

Identifying Suspicious Geographies

While blocking entire countries is rarely practical for global businesses, I often use GeoIP lookups to flag anomalies. If your employees are exclusively based in Bengaluru and Mumbai, a successful SSH login from an IP in Eastern Europe should trigger an immediate P1 incident.

Detecting CVE-2024-6387 (RegreSSHion) Patterns

The RegreSSHion vulnerability involves a race condition. While the exploit itself might not leave a "Failed password" log, the preparatory scanning and the multiple connection attempts required to time the race condition create significant log noise. Look for frequent Timeout before authentication or Connection closed by authenticating user messages in short bursts.

# Checking for connection timeouts which may indicate exploitation attempts
grep "Timeout before authentication" /var/log/auth.log | awk '{print $10}' | sort | uniq -c

Setting Up Automated Alerts

I recommend integrating your Python parser with a messaging API like Slack or Telegram. For critical Indian infrastructure, alerts should also be logged to a centralized Syslog server that is compliant with CERT-In directives regarding log retention (currently 180 days).

---

Hardening Your SSH Configuration

Detection is only half the battle. We must harden the configuration to reduce the attack surface. For many organizations, the most effective strategy is implementing secure SSH access for teams to eliminate the need for exposed ports entirely.

Modifying sshd_config

Edit /etc/ssh/sshd_config and ensure the following parameters are set. These changes effectively neutralize most brute-force scripts.

# Recommended Hardening Settings
PermitRootLogin no PasswordAuthentication no PubkeyAuthentication yes MaxAuthTries 3 LogLevel VERBOSE

Setting LogLevel VERBOSE is crucial. It provides additional details, such as the fingerprint of the SSH key used, which is invaluable during a forensic investigation.

Implementing Port Knocking or VPNs

For high-value targets, I move SSH off port 22. While "security by obscurity" is not a primary defense, it eliminates 99% of the automated log noise. Alternatively, we restrict SSH access to a specific WireGuard or Tailscale interface.

# Restrict SSH to a specific internal IP (e.g., VPN gateway)
In /etc/ssh/sshd_config
ListenAddress 10.8.0.1

Using Fail2Ban as a Secondary Layer

Even with a custom parser, Fail2Ban is useful for immediate blocking at the iptables or nftables level. I configure a custom jail for SSH that is more aggressive than the default settings.

# Example Fail2Ban filter for SSH
[sshd] enabled = true port = ssh filter = sshd logpath = /var/log/auth.log maxretry = 3 bantime = 3600

To verify the current status of blocked IPs, I use the client tool:

$ fail2ban-client status sshd

The next technical step is to integrate these logs into a SIEM (Security Information and Event Management) platform like ELK (Elasticsearch, Logstash, Kibana) or Wazuh. This allows for long-term trend analysis and correlation with other system events, such as unauthorized sudo attempts or unexpected binary executions.