A team ships a feature on Friday afternoon. Staging is green. The deploy runs against prod. Forty seconds later, every request to /api/orders returns a 502. Logs say nginx cannot resolve the upstream orders-svc by name. The incident channel lights up.
Six minutes of docker compose ps later, somebody finds the cause: in docker-compose.prod.yml, the orders service is named orders (without the -svc suffix). The dev compose file uses orders-svc. The nginx config — copied straight from staging — references orders-svc. Staging worked because staging's compose matched. Prod broke because prod's didn't. Nothing about the service was wrong. The service-name string drifted across compose files, and the deploy was the moment that drift became visible.
This is "compose drift," and it is the single most common cause of "works on dev, broken on prod" deploy incidents on small VPS setups. It is also one of the easiest classes of failure to catch before the deploy runs. A 30-line preflight script applied at the start of every deploy catches roughly nine out of ten of these incidents, and turns the remaining one into a fast diagnosis instead of a slow one.
This guide walks the four shapes drift takes, the specific preflight that catches each shape, and how to wire it into your deploy without slowing things down.
What "drift" actually means here
Most teams have at least three compose files that describe the same stack: a base file, a docker-compose.dev.yml overlay, and a docker-compose.prod.yml overlay. Sometimes there is a fourth for staging. The overlays exist because dev needs hot-reload mounts and exposed ports while prod needs internal networks and resource limits. That separation is correct and necessary.
The problem is that the overlays are edited independently. A new env var goes into dev.yml because the engineer added it during local work. The same env var does not make it into prod.yml because the deploy ran on a different day, on a different machine, and nobody crossed-checked. Six weeks later the field is load-bearing in prod and nobody remembers why staging works.
Drift is what happens when files that should describe the same logical stack stop describing the same logical stack. The deploy is when the divergence pays its bill.
The four shapes of compose drift
Almost every drift incident is one of these four. Knowing the shapes is half the diagnosis.
1. Env var drift
A variable is set in one compose file but missing or differently-named in another. The app reads it at startup, sees undefined, and either crashes immediately or — worse — silently falls back to a default that points at the wrong database.
Common variants:
STRIPE_KEYin dev,STRIPE_API_KEYin prodDATABASE_URLset indev.ymland read from.env, but in prod set via shell environment that is not exported into the compose context- A new feature flag that defaults to
truein dev andfalsein prod because nobody updatedprod.yml
2. Volume path drift
A bind-mount in dev (./data:/app/data) does not exist in prod, or points to a different host path. The app starts, but writes to a path that vanishes when the container restarts.
The painful version: the dev path is bind-mounted, the prod path is a named volume, and the app's behavior depends on which one it has. Dev and prod each work in isolation but produce subtly different output formats.
3. Network alias / service name drift
The error from the opening of this article. Service names — the strings other services use as DNS hostnames inside the compose network — differ between files. nginx upstreams, app config, and inter-service calls all assume one name. The compose file in front of you assumes another.
Variants: a service is renamed in dev and not in prod; an alias is added in prod.yml (aliases: ["orders-svc"]) but absent in dev.yml, so dev code that uses orders-svc works in prod but not dev (or vice versa).
4. Restart policy and depends_on drift
restart: unless-stopped in prod, no restart policy in dev. depends_on includes a healthcheck condition in dev but not prod. The result is a stack that starts cleanly in dev (where the engineer waits ten seconds before testing) but races on prod where the database is still initializing when the API tries to connect.
This is the subtlest of the four. Nothing is wrong; the timing is wrong. It usually surfaces as "the deploy works most of the time."
Why teams don't catch this
Three structural reasons.
The diff is small. A drift bug usually fits inside a single line. docker-compose.prod.yml has orders instead of orders-svc. Code review reads "renamed service" and approves. The reviewer cannot easily check whether the rename was applied to all dependents, because the dependents are in nginx config, application code, and a different compose file.
Local tests don't see it. The dev compose works on the dev machine. The CI compose works in CI. Each environment validates its own file in isolation, never against another. A test suite that runs docker compose up and pings localhost:3000 passes regardless of whether prod.yml is consistent with dev.yml.
The deploy is the first time both files are loaded together. And by then the deploy is already running.
The fix is to load both files together earlier — specifically, to resolve both files into their merged form, then compare the resolved configurations against each other before any container starts.
The preflight, as a script
docker compose config is the load-bearing tool. It reads the base file plus any overlays, resolves all variable substitutions, and prints the final merged compose specification as YAML. Run it once for dev, once for prod, and you have two canonicalized stack descriptions you can diff.
A working preflight, ~30 lines:
#!/usr/bin/env bash
set -euo pipefail
PROD_FILES=(-f docker-compose.yml -f docker-compose.prod.yml)
DEV_FILES=(-f docker-compose.yml -f docker-compose.dev.yml)
PROD=$(docker compose "${PROD_FILES[@]}" config 2>/dev/null)
DEV=$(docker compose "${DEV_FILES[@]}" config 2>/dev/null)
# 1. Service names must match exactly
prod_services=$(echo "$PROD" | yq '.services | keys | .[]' | sort)
dev_services=$(echo "$DEV" | yq '.services | keys | .[]' | sort)
if [[ "$prod_services" != "$dev_services" ]]; then
echo "DRIFT: service names differ"
diff <(echo "$prod_services") <(echo "$dev_services") || true
exit 1
fi
# 2. Required env var keys must appear in both (values may differ)
for svc in $prod_services; do
prod_keys=$(echo "$PROD" | yq ".services.$svc.environment | keys | .[]" 2>/dev/null | sort || true)
dev_keys=$(echo "$DEV" | yq ".services.$svc.environment | keys | .[]" 2>/dev/null | sort || true)
if [[ "$prod_keys" != "$dev_keys" ]]; then
echo "DRIFT: env var keys differ for service: $svc"
diff <(echo "$prod_keys") <(echo "$dev_keys") || true
exit 1
fi
done
# 3. Network aliases must match
prod_aliases=$(echo "$PROD" | yq '.services | to_entries | map({(.key): (.value.networks // {} | to_entries | map(.value.aliases // []) | flatten)}) | .[]' | sort)
dev_aliases=$(echo "$DEV" | yq '.services | to_entries | map({(.key): (.value.networks // {} | to_entries | map(.value.aliases // []) | flatten)}) | .[]' | sort)
if [[ "$prod_aliases" != "$dev_aliases" ]]; then
echo "DRIFT: network aliases differ"
diff <(echo "$prod_aliases") <(echo "$dev_aliases") || true
exit 1
fi
echo "preflight ok: no drift detected"
The script answers four questions: do both stacks define the same services, do those services declare the same env var keys, do they share the same network aliases, and (implicitly, by exit code) is it safe to deploy. Run it before docker compose up -d and you have caught nine out of ten drift incidents.
yq is the only non-standard dependency. Install it with apt install yq or the static binary from the project releases page.
A worked example: catching a real drift
Push a change to dev.yml that adds a FEATURE_FLAGS_URL env var, but forget to add it to prod.yml. Run the preflight:
$ ./scripts/compose-preflight.sh
DRIFT: env var keys differ for service: api
< FEATURE_FLAGS_URL
> SENTRY_DSN
> STRIPE_KEY
< SENTRY_DSN
< STRIPE_KEY
exit 1
The script blocks the deploy. The fix is one line in prod.yml. Total time from "preflight failed" to "preflight passes": under a minute. Compare to the failure mode where the deploy runs, the API silently picks up an undefined flag URL, and a different team discovers it three hours later when the experiments dashboard is empty.
What the preflight cannot catch
Be honest about the limits. The preflight is structural — it compares shapes. Three classes of bug live below it.
- Image tag mismatches. Both files declare a service
api, but one points atmyapp:1.4.2and the other atmyapp:latest. The preflight passes; the runtime behavior diverges. Adding a "image tags must match" check to the preflight is one moreyqline, but in practice most teams want different tags between dev and prod (:devand:prod). Decide your policy and codify it. - Host kernel and Docker version drift. The compose files match. The host running them does not. A volume mount option that works on Ubuntu 22 fails on Ubuntu 20. No file diff catches this. The mitigation is to run the same Docker version everywhere, and check it in the deploy script.
- External dependencies. A managed Postgres in prod, a containerized Postgres in dev. The compose files are deliberately different. The preflight will flag it as drift; you will mark the difference as expected. Build an allowlist for known-acceptable divergences so the preflight does not become noise.
The preflight catches a specific, common, expensive class of bug. It does not catch everything, and pretending otherwise will erode trust in it.
CI vs deploy script
Both. Run the preflight in CI on every PR that touches a compose file — that is where it stops bad changes from landing. Run it again at the start of the deploy script — that is where it catches drift introduced by a hand-edit on the VPS that never made it back into git. The two checks are not redundant; they cover different escape routes.
The deploy-time check is especially important because compose files on production VPSes are surprisingly often hand-edited. An ops person fixes a port collision at 2 AM, never opens a PR, and the file in git no longer matches the file on disk. The next deploy that pulls from git will overwrite the fix and reintroduce the bug. A preflight that compares the on-disk file against the resolved compose catches the divergence before the overwrite.
The long-term fix: one base file, thin overlays, no shared state outside compose
The preflight is a backstop. The real fix is structural: keep docker-compose.yml as the canonical description of every service, every env var, every network, every volume. Use dev.yml and prod.yml only for the small differences — port mappings, mount paths, resource limits. Anything that appears in only one overlay is a candidate for either being moved to the base or being explicitly justified in a comment.
A repo where 95% of the stack lives in the base file and 5% lives in overlays does not have drift incidents. A repo where 60% of the stack is duplicated across overlays has drift incidents every other month.
Compose drift is not a Docker problem. It is a synchronization problem in two text files. Treat the synchronization as load-bearing — with a preflight in CI and at deploy time, plus a discipline of keeping overlays thin — and the class of bug stops happening. The minute you spend writing the preflight pays for itself the first time a Friday-afternoon deploy does not turn into an incident.
Related in the StoicSoft network
If you work in AI-assisted coding, shared terminal sessions, or agent-driven shell workflows like the ones above, 1devtool is the StoicSoft network's tool for safer AI-assisted terminal work — shared sessions with auditing, preflight policy, and tiered model routing built in.