The dev stack ran schema work from the api entrypoint while the pull and airgap stacks ran it from a dedicated migrate service. Same repo, two different init paths, and the dev one was missing a step. That gap was real, not theoretical: the first run of this service on a stack that had been up for days created 13 default settings. The entrypoint only ran `db upgrade` and `plugin upgrade-all`, never the seeds, so any patch adding a permission or a setting reached pull/airgap sites and silently skipped dev. endpoints.agentupdate was exactly that shape of change. Running init from the api entrypoint has two other problems. Under `restart: unless-stopped` a failing migration crash-loops, re-attempting partially-applied schema work and burying the original error under retry noise; the migrate service uses `restart: "no"` so a failure exits once and stays readable. And `plugin upgrade-all` was tolerated as a warning, so the app would start with plugin tables missing - the shape of the endpointlogcollections bug. Chained with && it now stops the rollout instead. No fallback path is kept in the entrypoint. Compose honours service_completed_successfully even when a single service is named, so `up -d api` still runs migrate to completion first - verified, api stays in Created until migrate exits 0. A skip-flag would have meant the safe behaviour depended on remembering to set a variable, and deployment here goes through an install script that invokes compose, so there is no second caller to protect. The entrypoint keeps only the database wait, which is about this container's readiness to serve rather than schema state. The seeds are all create-if-missing, so running them on every start is a no-op once current: the second run reported 0 permissions, 0 settings and scopes already up to date. Ordering in the chain is load-bearing - plugin tables cannot be created before the plugins are installed, and seedscopes only links permissions that already exist as rows, so seeds run last. migrate carries the full app environment including CORS_ORIGINS even though it never serves a request, because create_app() validates config before any CLI command dispatches and the production config refuses to build without an explicit allowlist. It mounts instance_data because bootstrap-plugins.sh writes plugins.json there; it and api never run concurrently, so the shared mount is not contended. The Dockerfile's standalone `docker run` note now states that the image does not migrate itself and lists the sequence to run first.
212 lines
9.8 KiB
YAML
212 lines
9.8 KiB
YAML
# shopdb-flask single-site docker-compose template.
|
|
#
|
|
# Per ADR-004, each adopting facility runs its own stack. This template
|
|
# brings up MySQL + the API container and exposes the API on port 5001.
|
|
# The Vue frontend is served separately by the API in production builds
|
|
# (see register_frontend_routes in shopdb/__init__.py); for dev, run
|
|
# `npm run dev` in frontend/ on a separate port.
|
|
#
|
|
# Usage:
|
|
# cp .env.example .env
|
|
# # edit .env with site-specific secrets and origins
|
|
# docker compose up -d
|
|
#
|
|
# Refresh after pulling new code:
|
|
# docker compose build api
|
|
# docker compose up -d api
|
|
|
|
services:
|
|
db:
|
|
image: mysql:8.4
|
|
# utf8mb4 server-wide so the auto-created MYSQL_DATABASE is utf8mb4, not the
|
|
# image default. Keeps every site's schema on the same charset/collation.
|
|
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
|
|
restart: unless-stopped
|
|
environment:
|
|
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:?MYSQL_ROOT_PASSWORD must be set}
|
|
MYSQL_DATABASE: shopdb_flask
|
|
MYSQL_USER: shopdb
|
|
MYSQL_PASSWORD: ${MYSQL_PASSWORD:?MYSQL_PASSWORD must be set}
|
|
volumes:
|
|
- db_data:/var/lib/mysql
|
|
# Bind MySQL to loopback only. The api container reaches db over the
|
|
# compose network regardless of this mapping; the published port is just
|
|
# for local admin tools (mysqldump, a client on the host). Exposing 3306
|
|
# on all interfaces would put the database on the facility network.
|
|
ports:
|
|
- "127.0.0.1:${MYSQL_PORT:-3306}:3306"
|
|
healthcheck:
|
|
# 127.0.0.1, not localhost: localhost means the unix socket, and the
|
|
# entrypoint's init pass answers on the socket while running the server
|
|
# with --skip-networking. A socket ping therefore reports healthy DURING
|
|
# init, and the api/migrate services start against a server that is about
|
|
# to restart. Over TCP the probe stays red until the real server listens.
|
|
test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-u", "root", "-p${MYSQL_ROOT_PASSWORD}"]
|
|
interval: 10s
|
|
timeout: 5s
|
|
retries: 5
|
|
|
|
# Schema, plugin bootstrap and seed data all happen here, not in the api
|
|
# container. One place, one ordering, and a failure stops the rollout instead
|
|
# of being retried forever: restart "no" means a bad migration exits once and
|
|
# leaves the error as the last thing in the log. The api service below will
|
|
# not start at all until this exits 0, so `up -d api` still migrates first -
|
|
# compose honours service_completed_successfully even when a single service
|
|
# is named on the command line.
|
|
migrate:
|
|
build: .
|
|
restart: "no"
|
|
depends_on:
|
|
db:
|
|
condition: service_healthy
|
|
environment:
|
|
FLASK_ENV: production
|
|
DATABASE_URL: mysql+pymysql://shopdb:${MYSQL_PASSWORD}@db:3306/shopdb_flask?charset=utf8mb4
|
|
SECRET_KEY: ${SECRET_KEY:?SECRET_KEY must be set}
|
|
JWT_SECRET_KEY: ${JWT_SECRET_KEY:?JWT_SECRET_KEY must be set}
|
|
# Required even though this service never serves a request: create_app()
|
|
# runs config validation before any CLI command dispatches, and the
|
|
# production config refuses to build without an explicit CORS allowlist.
|
|
CORS_ORIGINS: ${CORS_ORIGINS:?CORS_ORIGINS must be set}
|
|
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
|
# Chained with && so the first failure stops the rest. Ordering is
|
|
# load-bearing: plugin tables cannot be created before the plugins are
|
|
# installed, and seedscopes only links permissions that already exist as
|
|
# rows, so the seeds run last.
|
|
#
|
|
# The seeds are all create-if-missing (see shopdb/cli/__init__.py), so
|
|
# running them on every start is a no-op once the site is current. They run
|
|
# here rather than only on first deploy because a patch that adds a
|
|
# permission or a default setting has to reach an existing site on the
|
|
# upgrade it already performs - that is the whole point of this service.
|
|
# No entrypoint override: the image entrypoint now only waits for the
|
|
# database and execs, which is exactly the preamble this service wants.
|
|
command:
|
|
- sh
|
|
- -c
|
|
- >
|
|
flask db upgrade &&
|
|
/app/scripts/bootstrap-plugins.sh &&
|
|
flask plugin upgrade-all &&
|
|
flask seed permissions &&
|
|
flask seed settings &&
|
|
flask seed reference-data
|
|
volumes:
|
|
- ./plugins:/app/plugins:ro
|
|
- ./scripts:/app/scripts:ro
|
|
# plugins.json is written here by bootstrap-plugins.sh, so this service
|
|
# needs the same instance volume the api uses. They never run at the same
|
|
# time, so the shared mount is not contended.
|
|
- instance_data:/app/instance
|
|
|
|
api:
|
|
build: .
|
|
restart: unless-stopped
|
|
depends_on:
|
|
db:
|
|
condition: service_healthy
|
|
migrate:
|
|
condition: service_completed_successfully
|
|
environment:
|
|
FLASK_ENV: production
|
|
DATABASE_URL: mysql+pymysql://shopdb:${MYSQL_PASSWORD}@db:3306/shopdb_flask?charset=utf8mb4
|
|
SECRET_KEY: ${SECRET_KEY:?SECRET_KEY must be set}
|
|
JWT_SECRET_KEY: ${JWT_SECRET_KEY:?JWT_SECRET_KEY must be set}
|
|
CORS_ORIGINS: ${CORS_ORIGINS:?CORS_ORIGINS must be set}
|
|
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
|
ZABBIX_URL: ${ZABBIX_URL:-}
|
|
ZABBIX_TOKEN: ${ZABBIX_TOKEN:-}
|
|
COLLECTOR_API_KEY: ${COLLECTOR_API_KEY:-}
|
|
COLLECTOR_API_KEY_ENDPOINTS: ${COLLECTOR_API_KEY_ENDPOINTS:-}
|
|
# This file is the source-build dev stack (wilm/dev-be); stable and edge
|
|
# ship through the pull/airgap compose files, not this one. So default the
|
|
# badge label to DEV here - a raw source build is by definition not a
|
|
# release - while still allowing an override from .env.
|
|
SHOPDB_ENV_LABEL: ${SHOPDB_ENV_LABEL:-DEV}
|
|
ports:
|
|
- "${API_PORT:-5001}:5001"
|
|
# Terminate TLS in gunicorn with the mounted self-signed cert so the browser
|
|
# sees a secure context. navigator.clipboard and other secure-context-only
|
|
# web APIs refuse to run over plain http on a non-localhost origin, so the
|
|
# test box (accessed by FQDN) needs real https. A site swaps in its own cert
|
|
# under ./certs or fronts this with its IIS/nginx proxy.
|
|
command:
|
|
- gunicorn
|
|
- --bind
|
|
- 0.0.0.0:5001
|
|
# Async workers, not threads. The endpoint agent long-polls
|
|
# /api/endpoints/commands/next and re-polls immediately on a 204, so every
|
|
# agent holds a connection essentially continuously and fleet size IS the
|
|
# concurrency. Under gthread that meant 4 x 8 = 32 slots total, and a
|
|
# single worker's 8 threads could fill with sleeping agents while the CPU
|
|
# sat idle - measured: 8 of 9 threads in timed sleep, UI p95 24s.
|
|
#
|
|
# A held connection under gevent is a greenlet, not an OS thread, so idle
|
|
# is nearly free. Measured on this stack (devnotes/gevent-sizing-simulation.md):
|
|
# 700 simulated agents plus UI, integration and kiosk traffic served with
|
|
# zero errors at UI median 7.5ms, peak RSS 479MB. CPU, not memory or DB
|
|
# connections, is the binding resource.
|
|
- --worker-class
|
|
- gevent
|
|
- --workers
|
|
- "4"
|
|
# Per-worker greenlet ceiling. 4 x 2000 is far above any realistic fleet;
|
|
# it exists so a connection spike queues rather than being refused.
|
|
- --worker-connections
|
|
- "2000"
|
|
# 60, not 30: the timeout must exceed _CLAIM_HOLD_SECONDS in
|
|
# plugins/endpoints/api/routes.py or gunicorn kills a healthy long-poll
|
|
# mid-hold. Cheap here because a stuck connection costs a greenlet.
|
|
- --timeout
|
|
- "60"
|
|
- --access-logfile
|
|
- "-"
|
|
# %(D)s is request duration in microseconds. Without it a slow request is
|
|
# indistinguishable from a fast one in the log, and latency has to be
|
|
# reconstructed from outside the container. Keep in sync with the
|
|
# Dockerfile CMD.
|
|
- --access-logformat
|
|
- '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s %(D)sus "%(f)s" "%(a)s"'
|
|
- --error-logfile
|
|
- "-"
|
|
- --certfile
|
|
- /app/certs/server.crt
|
|
- --keyfile
|
|
- /app/certs/server.key
|
|
- wsgi:app
|
|
healthcheck:
|
|
# Match the TLS listener: probe over https and accept the self-signed cert
|
|
# (-k). The Dockerfile default probes http and would stay red once gunicorn
|
|
# only speaks TLS.
|
|
# /api/dashboard/health is the public probe route and answers 200 even when
|
|
# the database is down, so the body's status field is what decides health.
|
|
# Probing POST /api/auth/login instead logged a 400 every 30s and exercised
|
|
# the credential path from a health check.
|
|
test: ["CMD-SHELL", "curl --silent --insecure https://localhost:5001/api/dashboard/health | grep -q '\"status\":\"ok\"'"]
|
|
interval: 30s
|
|
timeout: 5s
|
|
start_period: 10s
|
|
retries: 3
|
|
volumes:
|
|
- ./plugins:/app/plugins:ro
|
|
# Standalone operator scripts shipped inside generated bundles (the
|
|
# endpoint-agent uninstaller). Read-only, same pattern as plugins: the
|
|
# site-agent packaging route reads uninstall-shopdbagent.ps1 from here at
|
|
# download time, so it must be visible in the container.
|
|
- ./scripts:/app/scripts:ro
|
|
# Self-signed TLS material for the dev/test https listener. Read-only;
|
|
# generated on the host under ./certs and never baked into the image.
|
|
- ./certs:/app/certs:ro
|
|
# /app/instance is WRITTEN state, not code: plugins.json (which plugins
|
|
# this site has enabled), uploaded floor plans, branding, model and
|
|
# application images, employee photos, warranty proofs, slides,
|
|
# printed-part files, and the Dell OAuth token. Without this volume a
|
|
# `docker compose build api && up -d api` recreates the container and
|
|
# takes all of it with it, so the site comes back with its plugins
|
|
# disabled and MySQL rows pointing at files that no longer exist.
|
|
- instance_data:/app/instance
|
|
|
|
volumes:
|
|
db_data:
|
|
instance_data:
|