Security
Secrets Management & Vault — store, rotate, and inject
Keep secrets out of code, env files, and git. Store static secrets in Vault's KV engine, issue short-lived dynamic DB/cloud credentials on demand, authenticate apps with AppRole and humans with OIDC, and manage the lease → renew → revoke → rotate lifecycle. Fast lookup for on-call and interviews, with the cloud-native equivalents (AWS/GCP/Azure).
Docs-checkedvaultsecretsrotationkvdynamic-secrets
Quick reference — reach for these first
Store a static secret
vault kv put secret/APP/config db_pass=PASS api_key=KEYWrite/overwrite a KV v2 secret; each put is a new version.Read one field
vault kv get -field=db_pass secret/APP/configReturn just the value (no metadata) so it pipes cleanly into an env var.Get a dynamic DB cred
vault read database/creds/ROLEVault creates a unique username/password in the DB with a lease (e.g. 1h).Machine auth (AppRole)
vault login -method=approle role_id=RID secret_id=SIDExchange role_id + secret_id for a short-lived token; no human in the loop.Revoke a lease (destructive)
vault lease revoke LEASE_IDImmediately invalidates the credential — the DB user is dropped. Irreversible.Short-lived DB credential lifecycle
Text equivalent: An app authenticates to Vault with AppRole and gets a token, then reads database/creds/ROLE. Vault creates a unique user in the database with a one-hour lease and returns the credential. The app connects to the database with it. When the lease expires or is revoked, Vault deletes the database user and the old credential stops working.
Common tasks
KV basics · Dynamic DB creds · Auth & lifecycle
| KV basics | Dynamic DB creds | Auth & lifecycle |
|---|---|---|
| Confirm you can reach Vault and are authenticated | vault status && vault token lookup | Sealed=false and a token with a non-expired TTL is shown. |
| Enable the KV v2 engine (if not already) | vault secrets enable -version=2 -path=secret kv | "Success! Enabled the kv secrets engine at: secret/" (or already-enabled error). |
| Store a static secret | vault kv put secret/APP/config db_pass=PASS api_key=KEY | vault kv get secret/APP/config shows version 1 and both keys. |
| Read a single field for an app/CI | export DB_PASS=$(vault kv get -field=db_pass secret/APP/config) | echo ${#DB_PASS} prints a non-zero length; value never echoed to logs. |
| Roll a static secret to a new version | vault kv put secret/APP/config db_pass=NEWPASS api_key=KEY | vault kv metadata get secret/APP/config shows current_version incremented. |
| Enable the database secrets engine | vault secrets enable database | "Success! Enabled the database secrets engine at: database/". |
| Configure a DB connection (least-priv admin) | vault write database/config/DB plugin_name=postgresql-database-plugin allowed_roles="ROLE" connection_url="postgresql://{{username}}:{{password}}@HOST:5432/DBNAME" username="VAULTADMIN" password="ADMINPASS" | No error; vault read database/config/DB lists ROLE under allowed_roles. |
| Create a dynamic role with a short TTL | vault write database/roles/ROLE db_name=DB default_ttl=1h max_ttl=24h creation_statements=@create.sql | vault read database/roles/ROLE shows default_ttl 1h. |
| Issue a dynamic DB credential | vault read database/creds/ROLE | Output has a unique username, password, lease_id, and lease_duration 1h. |
| Enable AppRole auth for machines | vault auth enable approle | vault auth list shows approle/ mounted. |
| Bind a policy to an AppRole and get its IDs | vault write auth/approle/role/APP token_policies="app-policy" token_ttl=20m && vault read auth/approle/role/APP/role-id | A role_id is returned; generate a secret_id with vault write -f auth/approle/role/APP/secret-id. |
| Log in as the AppRole (machine) | vault login -method=approle role_id=RID secret_id=SID | A client token with the expected policies and a 20m TTL is returned. |
| Renew a lease before it expires | vault lease renew LEASE_ID | New lease_duration printed; stays <= max_ttl. |
| Revoke a lease / secret-id (DESTRUCTIVE — prod-changing) | vault lease revoke LEASE_ID # or: vault write auth/approle/role/APP/secret-id-accessor/destroy secret_id_accessor=ACC | "revocation operations queued successfully"; the old credential immediately fails to authenticate. |
Give an app a 1-hour database credential, then prove the old one dies
dynamic-secrets
1
Enable the database secrets engine
vault secrets enable database
# Success! Enabled the database secrets engine at: database/2
Register the Postgres connection with a least-privileged admin and the role allow-list
vault write database/config/my-pg plugin_name=postgresql-database-plugin allowed_roles="app" connection_url="postgresql://{{username}}:{{password}}@db:5432/app" username="vaultadmin" password="ADMINPASS"
# (no output; connection stored)3
Create a role that issues 1h creds via a CREATE ROLE statement
vault write database/roles/app db_name=my-pg default_ttl=1h max_ttl=24h creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";"
# Success! Data written to: database/roles/app4
App reads a fresh, unique credential on demand
vault read database/creds/app
# lease_id database/creds/app/8xKq...
# lease_duration 1h
# username v-approle-app-7Gf2...
# password A1b2-C3d4-E5f65
App connects to Postgres with that user/password — succeeds
psql "postgresql://v-approle-app-7Gf2...:A1b2-C3d4-E5f6@db:5432/app" -c 'select 1;'
# ?column?
# ----------
# 16
Lease is revoked (or TTL lapses) — DESTRUCTIVE, prod-changing: Vault drops the DB user
vault lease revoke database/creds/app/8xKq...
# All revocation operations queued successfully!7
VERIFY: the old credential no longer authenticates
psql "postgresql://v-approle-app-7Gf2...:A1b2-C3d4-E5f6@db:5432/app" -c 'select 1;'
# psql: error: FATAL: password authentication failed for user "v-approle-app-7Gf2..."Troubleshooting — symptom → first check → next check
| Symptom | First check | Establishes | Next check |
|---|---|---|---|
| 403 "permission denied" on a read/write | vault token lookup (policies) + vault policy read POLICY for the exact path and capability. | Whether it is an authz gap (token lacks capability on that path) vs. a bad mount/path. | Add the missing capability on the precise path (remember KV v2 reads use secret/data/APP, metadata uses secret/metadata/APP), then re-auth to pick up new policies. |
| "No value found at ..." / secret not found | vault kv list secret/APP and vault kv metadata get secret/APP/config to confirm mount, path, and version. | Wrong engine mount, wrong path, or you read a deleted/old KV v2 version. | Fix the path or pass -version=N; for KV v2 use the logical path (secret/APP/config), not the raw data/ path on the CLI. |
| DB auth fails mid-request after it worked earlier | vault lease lookup LEASE_ID to see remaining TTL / whether it expired. | The dynamic cred's lease lapsed (or hit max_ttl) while the app held it. | Renew proactively well before TTL, or re-read database/creds/ROLE to fetch a fresh cred and reconnect; raise default_ttl only if justified. |
| An AppRole secret_id was leaked (logs, image, CI output) | vault list auth/approle/role/APP/secret-id and identify the accessor. | A machine credential is exposed and can mint tokens until destroyed. | Destroy that secret_id (secret-id-accessor/destroy), rotate to a freshly generated one, and tighten secret_id_ttl/num_uses + CIDR binding. DESTRUCTIVE. |
| A real secret was committed to git | git log -p -S 'SECRET' to confirm it is in history, not just the working tree. | The secret is compromised the moment it is pushed — rewriting history does not un-leak it. | Rotate the secret at its source immediately (new DB password / API key), update Vault, then purge from history; prefer dynamic secrets so nothing static exists to leak. |
| Token works locally but app gets 403 in prod | Compare policies on the human token vs. the AppRole token (vault token lookup under each). | Human path (OIDC) is over-privileged while the machine AppRole is missing a path. | Scope the AppRole policy to exactly the paths the workload needs; keep humans on OIDC/SSO and machines on AppRole, never share a root/static token. |
References
- Vault KV v2 secrets enginedeveloper.hashicorp.com
- Database secrets engine (dynamic creds)developer.hashicorp.com
- AppRole auth methoddeveloper.hashicorp.com
- Lease, renew & revoke conceptsdeveloper.hashicorp.com
- vault kv command referencedeveloper.hashicorp.com