Files
shopdb-flask/docker-compose.yml
benjamin.baker 6ae251d2ca Move API to gevent and restore the claim hold to 25s
The claim poll holds a connection open server-side. Under gthread that
pinned one of a worker's 8 OS threads, so 14 agents at hold=25 left a
worker with 8 of 9 threads parked in timed sleep and UI requests hashed
to that worker waiting 12-24s. Hold was cut to 5s as an interim fix.

gevent removes the constraint rather than working around it: a held
connection is a greenlet and the per-worker ceiling is worker-connections
(2000), not 8 threads, so an idle hold no longer costs a UI request
anything. Measured at 700 simulated agents plus UI, integration and kiosk
traffic: zero errors, UI median 7.5ms, peak RSS 479MB. CPU is the binding
resource, not memory or DB connections.

With that gone the 5s hold is a liability. Hold length sets the fleet's
request rate, not its concurrency, because the agent re-polls immediately
on a 204 - so hold=5 is 5x the requests of hold=25, against a burst
bottleneck that measured as TLS handshake CPU. 25 also restores the
margin against the agent's MinIdlePollDuration=5s floor used to tell a
normal hold-window close apart from a transient failure; at 5 it was
~20ms.

Timeout raised to 60 so gunicorn cannot kill a healthy long-poll
mid-hold, and the access log now carries request duration - without it a
slow request is indistinguishable from a fast one and latency has to be
reconstructed from outside the container. Healthcheck moved off
POST /api/auth/login, which logged a 400 every 30s and exercised the
credential path from a probe.
2026-09-06 18:08:29 -04:00

157 lines
7.2 KiB
YAML

# shopdb-flask single-site docker-compose template.
#
# Per ADR-004, each adopting facility runs its own stack. This template
# brings up MySQL + the API container and exposes the API on port 5001.
# The Vue frontend is served separately by the API in production builds
# (see register_frontend_routes in shopdb/__init__.py); for dev, run
# `npm run dev` in frontend/ on a separate port.
#
# Usage:
# cp .env.example .env
# # edit .env with site-specific secrets and origins
# docker compose up -d
#
# Refresh after pulling new code:
# docker compose build api
# docker compose up -d api
services:
db:
image: mysql:8.4
# utf8mb4 server-wide so the auto-created MYSQL_DATABASE is utf8mb4, not the
# image default. Keeps every site's schema on the same charset/collation.
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:?MYSQL_ROOT_PASSWORD must be set}
MYSQL_DATABASE: shopdb_flask
MYSQL_USER: shopdb
MYSQL_PASSWORD: ${MYSQL_PASSWORD:?MYSQL_PASSWORD must be set}
volumes:
- db_data:/var/lib/mysql
# Bind MySQL to loopback only. The api container reaches db over the
# compose network regardless of this mapping; the published port is just
# for local admin tools (mysqldump, a client on the host). Exposing 3306
# on all interfaces would put the database on the facility network.
ports:
- "127.0.0.1:${MYSQL_PORT:-3306}:3306"
healthcheck:
# 127.0.0.1, not localhost: localhost means the unix socket, and the
# entrypoint's init pass answers on the socket while running the server
# with --skip-networking. A socket ping therefore reports healthy DURING
# init, and the api/migrate services start against a server that is about
# to restart. Over TCP the probe stays red until the real server listens.
test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-u", "root", "-p${MYSQL_ROOT_PASSWORD}"]
interval: 10s
timeout: 5s
retries: 5
api:
build: .
restart: unless-stopped
depends_on:
db:
condition: service_healthy
environment:
FLASK_ENV: production
DATABASE_URL: mysql+pymysql://shopdb:${MYSQL_PASSWORD}@db:3306/shopdb_flask?charset=utf8mb4
SECRET_KEY: ${SECRET_KEY:?SECRET_KEY must be set}
JWT_SECRET_KEY: ${JWT_SECRET_KEY:?JWT_SECRET_KEY must be set}
CORS_ORIGINS: ${CORS_ORIGINS:?CORS_ORIGINS must be set}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
ZABBIX_URL: ${ZABBIX_URL:-}
ZABBIX_TOKEN: ${ZABBIX_TOKEN:-}
COLLECTOR_API_KEY: ${COLLECTOR_API_KEY:-}
COLLECTOR_API_KEY_ENDPOINTS: ${COLLECTOR_API_KEY_ENDPOINTS:-}
# This file is the source-build dev stack (wilm/dev-be); stable and edge
# ship through the pull/airgap compose files, not this one. So default the
# badge label to DEV here - a raw source build is by definition not a
# release - while still allowing an override from .env.
SHOPDB_ENV_LABEL: ${SHOPDB_ENV_LABEL:-DEV}
ports:
- "${API_PORT:-5001}:5001"
# Terminate TLS in gunicorn with the mounted self-signed cert so the browser
# sees a secure context. navigator.clipboard and other secure-context-only
# web APIs refuse to run over plain http on a non-localhost origin, so the
# test box (accessed by FQDN) needs real https. A site swaps in its own cert
# under ./certs or fronts this with its IIS/nginx proxy.
command:
- gunicorn
- --bind
- 0.0.0.0:5001
# Async workers, not threads. The endpoint agent long-polls
# /api/endpoints/commands/next and re-polls immediately on a 204, so every
# agent holds a connection essentially continuously and fleet size IS the
# concurrency. Under gthread that meant 4 x 8 = 32 slots total, and a
# single worker's 8 threads could fill with sleeping agents while the CPU
# sat idle - measured: 8 of 9 threads in timed sleep, UI p95 24s.
#
# A held connection under gevent is a greenlet, not an OS thread, so idle
# is nearly free. Measured on this stack (devnotes/gevent-sizing-simulation.md):
# 700 simulated agents plus UI, integration and kiosk traffic served with
# zero errors at UI median 7.5ms, peak RSS 479MB. CPU, not memory or DB
# connections, is the binding resource.
- --worker-class
- gevent
- --workers
- "4"
# Per-worker greenlet ceiling. 4 x 2000 is far above any realistic fleet;
# it exists so a connection spike queues rather than being refused.
- --worker-connections
- "2000"
# 60, not 30: the timeout must exceed _CLAIM_HOLD_SECONDS in
# plugins/endpoints/api/routes.py or gunicorn kills a healthy long-poll
# mid-hold. Cheap here because a stuck connection costs a greenlet.
- --timeout
- "60"
- --access-logfile
- "-"
# %(D)s is request duration in microseconds. Without it a slow request is
# indistinguishable from a fast one in the log, and latency has to be
# reconstructed from outside the container. Keep in sync with the
# Dockerfile CMD.
- --access-logformat
- '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s %(D)sus "%(f)s" "%(a)s"'
- --error-logfile
- "-"
- --certfile
- /app/certs/server.crt
- --keyfile
- /app/certs/server.key
- wsgi:app
healthcheck:
# Match the TLS listener: probe over https and accept the self-signed cert
# (-k). The Dockerfile default probes http and would stay red once gunicorn
# only speaks TLS.
# /api/dashboard/health is the public probe route and answers 200 even when
# the database is down, so the body's status field is what decides health.
# Probing POST /api/auth/login instead logged a 400 every 30s and exercised
# the credential path from a health check.
test: ["CMD-SHELL", "curl --silent --insecure https://localhost:5001/api/dashboard/health | grep -q '\"status\":\"ok\"'"]
interval: 30s
timeout: 5s
start_period: 10s
retries: 3
volumes:
- ./plugins:/app/plugins:ro
# Standalone operator scripts shipped inside generated bundles (the
# endpoint-agent uninstaller). Read-only, same pattern as plugins: the
# site-agent packaging route reads uninstall-shopdbagent.ps1 from here at
# download time, so it must be visible in the container.
- ./scripts:/app/scripts:ro
# Self-signed TLS material for the dev/test https listener. Read-only;
# generated on the host under ./certs and never baked into the image.
- ./certs:/app/certs:ro
# /app/instance is WRITTEN state, not code: plugins.json (which plugins
# this site has enabled), uploaded floor plans, branding, model and
# application images, employee photos, warranty proofs, slides,
# printed-part files, and the Dell OAuth token. Without this volume a
# `docker compose build api && up -d api` recreates the container and
# takes all of it with it, so the site comes back with its plugins
# disabled and MySQL rows pointing at files that no longer exist.
- instance_data:/app/instance
volumes:
db_data:
instance_data: