From 58a36705291b5aca89fe21cfae319c95b5d69f8e Mon Sep 17 00:00:00 2001 From: "benjamin.baker" Date: Thu, 10 Sep 2026 15:50:36 -0400 Subject: [PATCH] Move migrations out of the api container into a migrate service The dev stack ran schema work from the api entrypoint while the pull and airgap stacks ran it from a dedicated migrate service. Same repo, two different init paths, and the dev one was missing a step. That gap was real, not theoretical: the first run of this service on a stack that had been up for days created 13 default settings. The entrypoint only ran `db upgrade` and `plugin upgrade-all`, never the seeds, so any patch adding a permission or a setting reached pull/airgap sites and silently skipped dev. endpoints.agentupdate was exactly that shape of change. Running init from the api entrypoint has two other problems. Under `restart: unless-stopped` a failing migration crash-loops, re-attempting partially-applied schema work and burying the original error under retry noise; the migrate service uses `restart: "no"` so a failure exits once and stays readable. And `plugin upgrade-all` was tolerated as a warning, so the app would start with plugin tables missing - the shape of the endpointlogcollections bug. Chained with && it now stops the rollout instead. No fallback path is kept in the entrypoint. Compose honours service_completed_successfully even when a single service is named, so `up -d api` still runs migrate to completion first - verified, api stays in Created until migrate exits 0. A skip-flag would have meant the safe behaviour depended on remembering to set a variable, and deployment here goes through an install script that invokes compose, so there is no second caller to protect. The entrypoint keeps only the database wait, which is about this container's readiness to serve rather than schema state. The seeds are all create-if-missing, so running them on every start is a no-op once current: the second run reported 0 permissions, 0 settings and scopes already up to date. Ordering in the chain is load-bearing - plugin tables cannot be created before the plugins are installed, and seedscopes only links permissions that already exist as rows, so seeds run last. migrate carries the full app environment including CORS_ORIGINS even though it never serves a request, because create_app() validates config before any CLI command dispatches and the production config refuses to build without an explicit allowlist. It mounts instance_data because bootstrap-plugins.sh writes plugins.json there; it and api never run concurrently, so the shared mount is not contended. The Dockerfile's standalone `docker run` note now states that the image does not migrate itself and lists the sequence to run first. --- Dockerfile | 12 +++++++- docker-compose.yml | 55 ++++++++++++++++++++++++++++++++++++ scripts/docker-entrypoint.sh | 35 ++++++++++------------- 3 files changed, 81 insertions(+), 21 deletions(-) diff --git a/Dockerfile b/Dockerfile index f5ed2ae6..382cb5f8 100644 --- a/Dockerfile +++ b/Dockerfile @@ -17,6 +17,13 @@ # docker build -t shopdb-flask . # Run (with .env): # docker run --env-file .env -p 5001:5001 shopdb-flask +# +# Note: this image does not migrate its own database on start. A plain +# `docker run` expects the schema to already be current; run the same sequence +# the compose `migrate` service does first: +# flask db upgrade && ./scripts/bootstrap-plugins.sh && flask plugin upgrade-all +# && flask seed permissions && flask seed settings && flask seed reference-data +# The supported path is `docker compose up`, which runs that for you. # ---- Stage 1: build the Vue frontend ---- FROM node:20-slim AS frontendbuild @@ -142,7 +149,10 @@ USER shopdb EXPOSE 5001 -# Entrypoint runs migrations and bootstraps plugins on first deployment +# Entrypoint waits for the database, then execs the CMD. Migrations, plugin +# bootstrap and seeds are NOT run here - they belong to the `migrate` service +# in the compose files so they happen exactly once per deployment rather than +# on every container restart. ENTRYPOINT ["/app/scripts/docker-entrypoint.sh"] # Probe the public health route, not the login endpoint. Probing POST diff --git a/docker-compose.yml b/docker-compose.yml index c80a3588..117d591f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -46,12 +46,67 @@ services: timeout: 5s retries: 5 + # Schema, plugin bootstrap and seed data all happen here, not in the api + # container. One place, one ordering, and a failure stops the rollout instead + # of being retried forever: restart "no" means a bad migration exits once and + # leaves the error as the last thing in the log. The api service below will + # not start at all until this exits 0, so `up -d api` still migrates first - + # compose honours service_completed_successfully even when a single service + # is named on the command line. + migrate: + build: . + restart: "no" + depends_on: + db: + condition: service_healthy + environment: + FLASK_ENV: production + DATABASE_URL: mysql+pymysql://shopdb:${MYSQL_PASSWORD}@db:3306/shopdb_flask?charset=utf8mb4 + SECRET_KEY: ${SECRET_KEY:?SECRET_KEY must be set} + JWT_SECRET_KEY: ${JWT_SECRET_KEY:?JWT_SECRET_KEY must be set} + # Required even though this service never serves a request: create_app() + # runs config validation before any CLI command dispatches, and the + # production config refuses to build without an explicit CORS allowlist. + CORS_ORIGINS: ${CORS_ORIGINS:?CORS_ORIGINS must be set} + LOG_LEVEL: ${LOG_LEVEL:-INFO} + # Chained with && so the first failure stops the rest. Ordering is + # load-bearing: plugin tables cannot be created before the plugins are + # installed, and seedscopes only links permissions that already exist as + # rows, so the seeds run last. + # + # The seeds are all create-if-missing (see shopdb/cli/__init__.py), so + # running them on every start is a no-op once the site is current. They run + # here rather than only on first deploy because a patch that adds a + # permission or a default setting has to reach an existing site on the + # upgrade it already performs - that is the whole point of this service. + # No entrypoint override: the image entrypoint now only waits for the + # database and execs, which is exactly the preamble this service wants. + command: + - sh + - -c + - > + flask db upgrade && + /app/scripts/bootstrap-plugins.sh && + flask plugin upgrade-all && + flask seed permissions && + flask seed settings && + flask seed reference-data + volumes: + - ./plugins:/app/plugins:ro + - ./scripts:/app/scripts:ro + # plugins.json is written here by bootstrap-plugins.sh, so this service + # needs the same instance volume the api uses. They never run at the same + # time, so the shared mount is not contended. + - instance_data:/app/instance + api: build: . restart: unless-stopped depends_on: db: condition: service_healthy + migrate: + condition: service_completed_successfully environment: FLASK_ENV: production DATABASE_URL: mysql+pymysql://shopdb:${MYSQL_PASSWORD}@db:3306/shopdb_flask?charset=utf8mb4 diff --git a/scripts/docker-entrypoint.sh b/scripts/docker-entrypoint.sh index 5ca9043e..38b92357 100644 --- a/scripts/docker-entrypoint.sh +++ b/scripts/docker-entrypoint.sh @@ -1,12 +1,25 @@ #!/bin/bash # Docker container entrypoint for ShopDB Flask. -# Runs migrations, bootstraps plugins on first deployment, then starts the app. +# Waits for the database to answer, then starts whatever it was given. +# +# This deliberately does NOT run migrations, plugin bootstrap or seeds. That +# sequence lives in the `migrate` service in docker-compose.yml (and its +# equivalents in the pull/airgap compose files) so there is exactly one place +# it happens. Running it here as well would mean every api restart re-attempted +# schema work, and a failing migration under `restart: unless-stopped` would +# crash-loop and bury the original error under retry noise. +# +# `up -d api` is still safe on its own: compose honours the +# service_completed_successfully dependency even when a single service is +# named, so the migrate service runs to completion first. set -e echo "==> ShopDB Flask Container Starting" -# Wait for database to be ready (optional, useful with docker-compose) +# Wait for database to be ready. Kept here because it is about this container's +# own readiness to serve, not about schema state. `flask db current` is a cheap +# round-trip that proves the connection works. if [ -n "$DATABASE_URL" ]; then echo "==> Waiting for database..." for i in {1..30}; do @@ -19,23 +32,5 @@ if [ -n "$DATABASE_URL" ]; then done fi -# Run migrations -echo "==> Running database migrations" -flask db upgrade || { - echo "ERROR: Database migrations failed" - exit 1 -} - -# Bootstrap plugins on first deployment -echo "==> Checking plugin installation" -/app/scripts/bootstrap-plugins.sh - -# Run plugin migrations after bootstrap (creates tables for enabled plugins) -# Safe to run even if no plugins are enabled - it will just do nothing -echo "==> Running plugin migrations" -flask plugin upgrade-all || { - echo "WARNING: Plugin migrations failed - some features may not work" -} - echo "==> Starting application" exec "$@"