Where a secret lives
| Place | When | Permissions |
|---|---|---|
file /srv/secrets/<stack>/<name>, inside the container — Docker secret /run/secrets/<name> |
the app can read the secret from a file | directories root 700, files root 400 |
/opt/<stack>/.env |
compose itself needs the value (labels, substitution), or the app reads only environment variables | root 600 |
.env.example in git |
variable names only, no values | — |
compose file, git (even private), command line, a shared chat, /tmp |
never | — |
/srv/secrets lives outside /opt. If /opt is mounted into a Docker management panel (Dockhand, Portainer)
or ends up in a config archive, the secrets won’t be there.
Setup
A directory for secrets
install -d -m 700 /srv/secrets /srv/secrets/<stack>Directories are root 700, files in them root 400. Docker runs as root, so mounting a file into a container still works.
Move the secret into a file — off screen
From a file:
install -m 400 /opt/<stack>/token.txt /srv/secrets/<stack>/token wc -c < /srv/secrets/<stack>/token # not 0 — and nothing on screenFrom a config line — through a variable, with no output and no trailing newline:
key=$(sed -nE 's/^[[:space:]]*apiKey:[[:space:]]*"?([^"[:space:]]+)"?.*/\1/p' config.yml) printf '%s' "$key" > /srv/secrets/<stack>/key; unset key chmod 400 /srv/secrets/<stack>/keyAttach it as a Docker secret
services: traefik: environment: CF_DNS_API_TOKEN_FILE: /run/secrets/cf_api_token # the app reads the file secrets: - cf_api_token - crowdsec_lapi_key secrets: cf_api_token: file: /srv/secrets/traefik/cf_api_token crowdsec_lapi_key: file: /srv/secrets/traefik/crowdsec_lapi_keyWithout Swarm this is a plain file mount at
/run/secrets/<name>. The difference from a volume: the compose file shows right away that it’s a secret, and all secrets are described in one place.The app reads the secret from a file
Method Example a *_FILEvariableCF_DNS_API_TOKEN_FILE,MARIADB_ROOT_PASSWORD_FILE,POSTGRES_PASSWORD_FILE— the official MariaDB and Postgres images support this; Postgres only forPOSTGRES_PASSWORD,POSTGRES_USER,POSTGRES_DBandPOSTGRES_INITDB_ARGSa …Fileoption in the configcrowdsecLapiKeyFilein the CrowdSec plugin for Traefik (CrowdSec with Traefik behind Cloudflare)a users file Basic Auth in Traefik: basicauth.usersfile=/pathinstead ofbasicauth.users=…Basic Auth hash — as a users file
A label gets its value when compose starts, so it can’t be a secret: the hash in
basicauth.users=${APP_AUTH}can only come from.env. To get it out of there, use a users file:# Traefik: a directory with users files — one mount for every service services: traefik: volumes: - /srv/secrets/traefik/basicauth:/basicauth:ro # the service: the label points to a path inside the Traefik container - "traefik.http.middlewares.app-auth.basicauth.usersfile=/basicauth/app"The file
/srv/secrets/traefik/basicauth/appholdsuser:hashlines in htpasswd format,$unescaped.
Passwords: new, for a command, leaked
A new password goes straight where it belongs
new=$(openssl rand -hex 20) # not printed on screen
# … apply new: to the database, to .env …
unset new
Keep passwords for people in a password manager (such as Vaultwarden) and passwords for services on the server. Those are two different places, and that’s how it should be.
A password for a command — not on the command line
An argument like -p'password' stays in shell history (~/.bash_history) and shows in the process list. If the
password is already in the container’s environment, take it from there, inside the container:
docker exec db sh -c 'exec mariadb-dump -u root -p"$MARIADB_ROOT_PASSWORD" --single-transaction app' | gzip > app.sql.gz
The outer single quotes make the shell inside the container expand $MARIADB_ROOT_PASSWORD.
A password leaked — change it off screen
For MariaDB:
new=$(openssl rand -hex 20)
docker exec -e NEW="$new" db sh -c 'mariadb -u root -p"$MARIADB_ROOT_PASSWORD" -e "ALTER USER IF EXISTS root@localhost IDENTIFIED BY \"$NEW\"; ALTER USER IF EXISTS root@\"%\" IDENTIFIED BY \"$NEW\";"' \
&& sed -i "s/^MARIADB_ROOT_PASSWORD=.*/MARIADB_ROOT_PASSWORD='$new'/" .env && echo "changed"
unset new; docker compose up -d # so the container gets the new password
The && makes .env change only if the database accepted the new password. Details in MariaDB in a container.
Other ways to keep secrets
| Approach | What you get | Cost |
|---|---|---|
Docker secrets from /srv/secrets |
the secret is outside /opt, shows up in compose as a secret, mode 400 |
needs *_FILE support or reading from a file |
.env with mode 600 |
works with everything; labels can’t do without it | sits in /opt next to the configs and travels with them into any archive |
| encrypted secrets in git (sops, age) | secrets are versioned, a server can be rebuilt from scratch | the decryption key still has to live somewhere; one more tool in the chain |
| a secrets manager (Vault, Infisical) | rotation, audit, role-based access | one more service to keep alive |
For a handful of your own servers the first two are enough: /srv/secrets for everything that can read files,
.env 600 for the rest.
If it doesn’t work
A secret got into git — even a private repository
- Cause
- Git history keeps it forever. You can scrub the history, but it takes time and doesn’t guarantee no copy is left anywhere.
- Fix
- Treat the secret as leaked and change the value.
A password ended up in a chat, a ticket or a screenshot
- Cause
- Everyone with access to the conversation has seen the value.
- Fix
- Change it so the new password never appears on screen. The MariaDB command is in the “Passwords” section above.
.env with mode 644
- Cause
- Any user on the server can read it.
- Fix
chmod 600 /opt/*/.env.
The same secret variable sits in the .env of several stacks, but only one reads it
- Cause
- The key was copied “just in case” — say, the bouncer key into CrowdSec’s
.env. When the key changes, the copies get forgotten. - Fix
- Delete the extra copies.
A container needs one variable, but it has env_file: .env
- Cause
- The whole
.envgoes inside: hashes, neighbouring services’ keys, domains. - Fix
- Remove
env_fileand pass what’s needed throughenvironment:(.env feeds compose substitution, env_file goes into the container).
A script made backup copies of .env in /tmp
- Cause
- Every user on the server can read
/tmp. - Fix
- Make copies in
/rootand delete them right after.
A key from a file is rejected, though it’s correct
- Cause
- The file ends with a newline: some programs read the key together with
\n. - Fix
- Write the file with
printf '%s', notecho.
Changed a secret’s path in compose — the container won’t be created
- Cause
- There’s no file at the new path yet, and without it compose doesn’t create the container.
- Fix
- Put the file in place first, then
git pullandup -din one command (The secret file comes before up -d).
How to check
find /srv/secrets -exec stat -c '%a %U %s %n' {} \; # mode, owner, size — no contents
stat -c '%a %n' /opt/*/.env # 600
docker exec traefik sh -c 'test -s /run/secrets/cf_api_token && echo "token in place"'
Plus a secrets search over what goes into git, with values masked (Server configs in git, step 5).