During a recent audit of a logistics platform based in Pune, we discovered that the entire production database credential set was visible via a simple docker inspect command. The engineering team was using standard environment variables in a docker-compose.yml file, assuming that because the server was "internal," the risk was minimal. This is a common fallacy in Indian SME environments where the overhead of Kubernetes is avoided in favor of Docker Compose or standalone Swarm clusters.
The Risks of Using Environment Variables for Sensitive Data
Environment variables are inherently leaky. They are visible to any process that can query the Docker API, they show up in ps output in some configurations, and they are almost always captured in application logs during crash dumps. If an attacker gains limited shell access or exploits a local file inclusion (LFI) vulnerability, /proc/self/environ becomes a goldmine.
Visibility in Docker History and Inspection
When you bake credentials into an image using the ENV instruction in a Dockerfile, those secrets are persisted in the image layers. Anyone with access to the image can run docker history to see them. Even if you pass them at runtime using -e, they are stored in plain text within the container's configuration on the host disk.
$ docker inspect --format='{{range .Config.Env}}{{println .}}{{end}}' compromised_container
DB_PASSWORD=my-super-secret-pw API_KEY=12345-abcde
CVE-2021-41092: The Danger of Improper Permissions
The CVE-2021-41092 vulnerability highlights why relying on standard filesystem isolation isn't enough. Improper permissions on /var/lib/docker/containers allowed local unprivileged users to traverse and potentially read sensitive data from container filesystems. If your secrets are sitting in a .env file or passed as raw environment variables, a lateral movement within the host becomes trivial.
What are Docker Secrets and Why Do They Matter?
Docker secrets provide a native, encrypted-at-rest mechanism for managing sensitive data like secure SSH access for teams, SSL certificates, and passwords. Unlike environment variables, secrets are only shared with the specific services that need them. In a Swarm environment, they are transmitted over an encrypted TLS connection and mounted into a tmpfs (in-memory) filesystem inside the container.
Core Benefits of Native Docker Secret Handling
- Encryption at Rest: Secrets are stored in the Swarm Raft log, which can be encrypted using an autolock key.
- In-Memory Only: Inside the container, secrets are mounted at
/run/secrets/<secret_name>. They never hit the container's persistent storage. - Least Privilege: You explicitly grant access to a secret on a per-service basis. If a service is compromised, it only exposes the secrets assigned to it.
- Compliance Alignment: Using Docker secrets helps meet the "Security by Design" requirements of India's DPDP Act 2023 by ensuring sensitive credentials are handled with technical safeguards.
How Docker Swarm Encrypts and Distributes Secrets
Docker Swarm uses a distributed consensus algorithm (Raft) to manage the state of the cluster. When you create a secret, it is sent to the Swarm Manager. The manager stores it in the Raft log. By default, this log is encrypted on disk if you enable the --autolock feature.
Activating Swarm Mode and Encryption
To use Docker secrets, your Docker engine must be in Swarm mode. Even a single-node setup benefits from this.
$ docker swarm init --advertise-addr 127.0.0.1
$ docker swarm update --autolock=true
When --autolock is enabled, the Docker daemon will require a manual unlock key after every restart to decrypt the Raft log and access the secrets. This prevents an attacker from stealing the physical disk or a backup of /var/lib/docker/swarm and extracting credentials.
Step-by-Step: Creating and Managing Secrets in a Swarm Cluster
I recommend never passing secrets as command-line arguments to avoid them being recorded in the shell history (~/.bash_history). Instead, pipe the secret from stdin or a file.
# Creating a secret from stdin
$ printf "my-super-secret-pw" | docker secret create db_root_password -
Listing active secrets
$ docker secret ls ID NAME CREATED UPDATED v9876543210abcdef db_root_password 2 minutes ago 2 minutes ago
Assigning Secrets to Specific Services
When creating a service, you must explicitly link the secret. We can also map the secret to a different filename inside the container and set specific ownership.
$ docker service create \
--name auth-service \ --secret source=db_root_password,target=db_password,uid=1000,gid=1000,mode=0400 \ my-auth-img:latest
In this example, the secret is mounted at /run/secrets/db_password with read-only permissions (0400) for the user with UID 1000. This follows the principle of least privilege, ensuring the application process can read the secret but cannot modify it or allow other users to see it.
Docker Compose Secrets Management for Development and Testing
While Swarm is the production standard, developers often use Docker Compose locally. Compose supports a secrets directive that mimics Swarm behavior by mounting files into /run/secrets/.
Defining Secrets in a docker-compose.yml File
The following configuration demonstrates how to handle both external secrets (pre-existing in Swarm) and file-based secrets (local development).
version: '3.8'
services: backend: image: node:18-alpine deploy: replicas: 1 secrets: - api_key - db_config environment: # Using _FILE suffix is a common pattern for apps to read from /run/secrets/ API_KEY_PATH: /run/secrets/api_key DB_CONFIG_PATH: /run/secrets/db_config
secrets: api_key: external: true db_config: file: ./configs/db_config.json
Managing External vs. File-Based Secrets
The external: true flag tells Docker that the secret already exists in the Swarm cluster. This is crucial for CI/CD pipelines. You create the secret once on the production manager node, and the Compose deployment simply references it.
For local development, the file: key is used. However, ensure that ./configs/db_config.json is included in your .gitignore. I often see Indian dev teams accidentally committing these "local" secret files to private GitHub or GitLab repos, which is a significant risk if those platforms are ever breached.
Transitioning Secrets from Local Compose to Production Swarm
The biggest hurdle for Indian MSPs moving from standalone containers to Swarm is the architectural change. Standalone docker run or docker-compose up (without Swarm) does not support the secrets object in the same secure way.
To bridge this gap:
- Initialize a single-node Swarm on the production VPS.
- Convert
.envvariables todocker secret createcommands. - Modify the application code to read from
/run/secrets/instead of environment variables.
If your application doesn't support reading from a file path for its configuration, you can use a small entrypoint script to read the secret and export it as an environment variable at runtime. While this re-introduces some risk, it still keeps the secret out of the image layers and the static Compose file.
#!/bin/sh
entrypoint.sh
if [ -f /run/secrets/db_password ]; then export DB_PASSWORD=$(cat /run/secrets/db_password) fi exec "$@"
Best Practices for Secure Docker Secrets Management
Securing the secret is only the first step. You must also manage the lifecycle of that secret.
Implementing the Principle of Least Privilege
Never use the root user inside your container to read secrets if it's not necessary. Use the uid and gid options in the --secret flag to restrict access. If you are using an Alpine-based image, the default node or nginx users often have UIDs like 1000 or 101.
Strategies for Secret Rotation and Versioning
Docker secrets are immutable. You cannot update a secret named db_password. Instead, you must create a new version and update the service to use it.
# Create version 2 of the secret
$ printf "new-secure-password-2024" | docker secret create db_password_v2 -
Update the service to use the new secret
$ docker service update \ --secret-rm db_password_v1 \ --secret-add source=db_password_v2,target=db_password \ auth-service
This rolling update ensures zero downtime. Swarm will start new tasks with the new secret and terminate the old tasks once the new ones are healthy.
Monitoring and Auditing Secret Access
We use docker service inspect to verify which secrets are attached to a running service. This is a vital part of a security audit.
$ docker service inspect --format '{{ json .Spec.TaskTemplate.ContainerSpec.Secrets }}' auth-service
In India, CERT-In frequently advises on monitoring for unauthorized access to container runtimes. You should log all calls to the Docker API, specifically POST /secrets/create and DELETE /secrets/{id}. Tools like Falco can be configured to alert when a process other than the main application binary attempts to read from /run/secrets/.
Advanced Integration: Docker Secrets and External Vaults
For larger organizations, native Docker secrets might not be enough. If you are managing hundreds of secrets across multiple environments, a dedicated secret manager is necessary.
When to Use HashiCorp Vault or AWS Secrets Manager
Native Docker secrets lack features like:
- Dynamic secret generation (e.g., temporary DB credentials).
- Detailed audit logging of who accessed which secret.
- Automatic TTL-based revocation.
If you are operating on AWS (which has a large footprint in the Mumbai region), using AWS Secrets Manager via an IAM role assigned to the EC2 instance running Docker is a robust alternative. The application can query the AWS API directly, bypassing the Docker secret mechanism entirely.
Automating Secret Injection in CI/CD Pipelines
In a typical GitLab or GitHub Actions pipeline for an Indian startup, we use the following workflow:
- The runner authenticates with the production Swarm manager via SSH.
- The runner checks if the secret exists:
docker secret inspect my_api_key || echo $CI_SECRET | docker secret create my_api_key -. - The runner executes
docker stack deploy -c docker-compose.yml my_app.
This ensures that secrets are never stored in the CI/CD environment's logs and are only injected into the production environment at the moment of deployment.
CVE-2024-21626: A Critical Warning
Recent research into runc (the underlying runtime for Docker) revealed CVE-2024-21626. This vulnerability allows internal file descriptor leakage. An attacker could exploit this to access the host filesystem. If an attacker gains access to the host, they could potentially retrieve Docker secret chunks stored in /var/lib/docker/swarm/secrets if the Swarm is not locked. This underscores the absolute necessity of enabling --autolock on all production nodes.
Auditing with Dive
To ensure no secrets were accidentally baked into your images during the build process (a common mistake when developers use ARG instead of SECRET), use the dive tool.
$ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:ro wagoodman/dive <image_id>
Look for .env files, config.json files, or shell history files in the image layers. If you find them, you must rebuild the image and rotate those credentials immediately, as they are now compromised.
Building a Robust Security Posture with Docker
Moving away from environment variables to Docker secrets is the single most effective way to harden a non-Kubernetes container environment. It mitigates the risk of credential leakage via logs, API inspection, and filesystem vulnerabilities.
Final Checklist for Securing Containerized Applications
- [ ] Is Swarm mode initialized with
--autolock=true? - [ ] Are all sensitive credentials removed from
docker-compose.ymlenvironment blocks? - [ ] Do containers run as a non-root user with specific UID/GID access to secrets?
- [ ] Is
/run/secrets/verified as atmpfsmount inside the container? - [ ] Are secret files (
.env,*.key) included in.gitignoreand.dockerignore? - [ ] Have you audited your images with
diveto ensure no secrets are in the layer history?
By implementing these patterns, Indian engineering teams can achieve a high level of security and compliance without the operational complexity of a full Kubernetes orchestrator. The focus should always be on reducing the blast radius of a potential container compromise.
Next Command
To check if your current Swarm manager is locked and protecting your secrets at rest, run:
$ docker node inspect self --format '{{ .Spec.Role }}' && docker swarm unlock-key
If the command returns a key, ensure it is stored in a secure physical or digital vault (like a hardware security module or a corporate password manager) and not on the server itself.