b5737f47c1
Taken from the other site, who hit it in production today (their commit
db75ca7): agents that fail the TLS handshake retry in a loop, and with four
blocking sync workers a handful of them pinned every worker. Real UI requests
queued for 20+ seconds while the CPU sat idle - the box was not busy, it was
blocked.
gthread, 4 workers x 8 threads, and a 30-second timeout rather than 60. A stuck
connection now holds one thread out of 32 instead of one worker out of four, and
it lets go in half the time, which is the difference between absorbing a retry
storm and queueing behind it.
Their server, our agent. The client behaviour that caused it is the endpoint
agent, so it will reach any site running one - which makes this a product fix
rather than something that belongs on one site's branch.
Applied to the Dockerfile as well as docker-compose.yml. A compose file
overrides the image CMD, but docker-compose.airgap.yml, .channel.yml and
.pull.yml do not specify a command at all, so they take the image default - and
so does anybody running the image by hand. Fixing only compose would have left
every air-gapped install on sync workers.
WJ production is unaffected either way: it runs waitress under IIS, already with
--threads=8, so it was never exposed to this. That is worth knowing before
anyone treats it as a prod hardening item that was missing.
Scope: product
155 lines
6.8 KiB
Docker
155 lines
6.8 KiB
Docker
# shopdb-flask single-site container.
|
|
#
|
|
# One image, one site. Per ADR-004, each adopting facility runs its own
|
|
# stack with its own DB, secrets, and enabled-plugin list. This image
|
|
# bundles all 13 catalog plugins (computers, employees, geenforce,
|
|
# knowledgebase, machines, measuringtools, network, notifications,
|
|
# printedparts, printers, slides, usb, warranty); a site installs + enables
|
|
# the ones it wants with `flask plugin install <name>` (or, declaratively,
|
|
# `flask plugin apply-profile <profile.json>`). Per ADR-013 a future lean
|
|
# build stages only the chosen plugin directories into this image.
|
|
#
|
|
# The frontend is built in a first stage and its dist output is copied into
|
|
# the final image so Flask can serve the SPA (register_frontend_routes in
|
|
# shopdb/__init__.py resolves <repo>/frontend/dist).
|
|
#
|
|
# Build:
|
|
# docker build -t shopdb-flask .
|
|
# Run (with .env):
|
|
# docker run --env-file .env -p 5001:5001 shopdb-flask
|
|
|
|
# ---- Stage 1: build the Vue frontend ----
|
|
FROM node:20-slim AS frontendbuild
|
|
|
|
WORKDIR /build
|
|
|
|
# Optionally trust the site's corporate root CA so npm's TLS to the registry
|
|
# validates behind an inspecting proxy. The .pem is gitignored and absent from a
|
|
# clean checkout, so the copy is made optional by pairing it with a file that
|
|
# always exists; the cert is installed only when actually provided. Without it
|
|
# npm 10.x behind an inspecting proxy fails silently and leaves an incomplete
|
|
# node_modules; a clean build with no proxy is unaffected.
|
|
COPY corp-root-ca.pem* frontend/package.json /tmp/corpca/
|
|
# node:20-slim ships without the ca-certificates package (no update-ca-certificates
|
|
# and no /usr/local/share/ca-certificates), so install it before trusting the cert.
|
|
RUN if [ -f /tmp/corpca/corp-root-ca.pem ]; then \
|
|
apt-get update && apt-get install -y --no-install-recommends ca-certificates && \
|
|
rm -rf /var/lib/apt/lists/* && \
|
|
cp /tmp/corpca/corp-root-ca.pem /usr/local/share/ca-certificates/corp-root-ca.crt && \
|
|
update-ca-certificates; \
|
|
fi
|
|
|
|
# Copy only the manifests first so `npm ci` caches on dependency changes.
|
|
COPY frontend/package*.json ./frontend/
|
|
# Trust the corp CA for npm's TLS only when it was actually installed above. The
|
|
# guard keeps NODE_EXTRA_CA_CERTS unset on a clean checkout (no cert), so Node
|
|
# does not warn about a missing cert path; a proxy-inspected build still gets
|
|
# the cert because the file is present.
|
|
# Install the frontend dependencies. vite and the rest of the build toolchain
|
|
# live under devDependencies, so include them explicitly (--include=dev).
|
|
# Behind an inspecting proxy (Zscaler) each package fetch is slow (40-50s) and
|
|
# npm's default concurrency piles up parallel sockets that the proxy resets
|
|
# mid-stream, making npm abort with "Exit handler never called" and leaving an
|
|
# incomplete node_modules with no vite. Serializing to a single socket
|
|
# (maxsockets 1) lets every fetch complete; the retry loop and raised timeouts
|
|
# ride out any remaining transient reset, and the final test hard-verifies the
|
|
# toolchain landed so a truncated install fails loudly here rather than as a
|
|
# confusing "vite: not found" three steps later.
|
|
RUN if [ -f /usr/local/share/ca-certificates/corp-root-ca.crt ]; then \
|
|
export NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/corp-root-ca.crt; \
|
|
fi; \
|
|
cd frontend && \
|
|
npm config set fetch-retries 5 && \
|
|
npm config set fetch-retry-factor 2 && \
|
|
npm config set fetch-retry-mintimeout 20000 && \
|
|
npm config set fetch-retry-maxtimeout 120000 && \
|
|
npm config set fetch-timeout 600000 && \
|
|
npm config set maxsockets 1 && \
|
|
( npm ci --include=dev --no-audit --no-fund \
|
|
|| { echo "npm ci attempt 1 failed, retrying..."; sleep 5; npm ci --include=dev --no-audit --no-fund; } \
|
|
|| { echo "npm ci attempt 2 failed, retrying..."; sleep 10; npm ci --include=dev --no-audit --no-fund; } ) && \
|
|
test -x node_modules/.bin/vite || { \
|
|
echo "ERROR: node_modules/.bin/vite missing after npm ci (proxy likely truncated the install)."; \
|
|
exit 1; \
|
|
}
|
|
|
|
# The frontend keeps its repo-relative layout here, because npm prebuild runs
|
|
# `node ../scripts/stage-frontend.mjs`, which stages each plugin's frontend/ into
|
|
# the Vite tree and codegens routes.gen.js. That script resolves its repo root as
|
|
# its own directory's parent, so it needs scripts/ and plugins/ as SIBLINGS of
|
|
# frontend/. Building with frontend/ flattened to the stage root made `../scripts`
|
|
# resolve to /scripts and the build failed on a missing module.
|
|
COPY scripts/stage-frontend.mjs ./scripts/
|
|
COPY plugins/ ./plugins/
|
|
|
|
COPY frontend/ ./frontend/
|
|
RUN if [ -f /usr/local/share/ca-certificates/corp-root-ca.crt ]; then \
|
|
export NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/corp-root-ca.crt; \
|
|
fi; \
|
|
cd frontend && npm run build
|
|
# Output lands in /build/frontend/dist (Vite default), copied into the final
|
|
# stage below.
|
|
|
|
# ---- Stage 2: Python application image ----
|
|
FROM python:3.14-slim AS base
|
|
|
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
|
PYTHONUNBUFFERED=1 \
|
|
PIP_NO_CACHE_DIR=1 \
|
|
PIP_DISABLE_PIP_VERSION_CHECK=1
|
|
|
|
WORKDIR /app
|
|
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends \
|
|
build-essential \
|
|
default-libmysqlclient-dev \
|
|
pkg-config \
|
|
curl \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
COPY requirements.txt ./
|
|
RUN pip install --no-cache-dir -r requirements.txt \
|
|
&& pip install --no-cache-dir gunicorn
|
|
|
|
COPY shopdb/ ./shopdb/
|
|
COPY plugins/ ./plugins/
|
|
COPY migrations/ ./migrations/
|
|
COPY scripts/ ./scripts/
|
|
COPY wsgi.py ./
|
|
|
|
# Built SPA from stage 1. Flask serves it via register_frontend_routes.
|
|
COPY --from=frontendbuild /build/frontend/dist ./frontend/dist
|
|
|
|
# Create instance/ IN THE IMAGE, owned by the app user, before the chown below.
|
|
# Docker seeds an empty named volume from the image's content at the mountpoint,
|
|
# ownership included. Without this the mountpoint is created root-owned 0755 and
|
|
# the container, which runs as shopdb, cannot write plugins.json or any upload
|
|
# into its own instance directory.
|
|
RUN mkdir -p /app/instance
|
|
|
|
RUN useradd --create-home --shell /bin/bash shopdb \
|
|
&& chown -R shopdb:shopdb /app
|
|
USER shopdb
|
|
|
|
EXPOSE 5001
|
|
|
|
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
|
|
CMD curl --silent http://localhost:5001/api/auth/login -X POST \
|
|
-H "Content-Type: application/json" -d '{}' \
|
|
| grep -q "VALIDATION_ERROR" || exit 1
|
|
|
|
# Threaded workers, matching docker-compose.yml - see the reasoning there. A
|
|
# compose file overrides this CMD, but an image run without one (a plain
|
|
# `docker run`, a site's own orchestrator) would otherwise still get the
|
|
# blocking sync default and the same starvation.
|
|
CMD ["gunicorn", \
|
|
"--bind", "0.0.0.0:5001", \
|
|
"--worker-class", "gthread", \
|
|
"--workers", "4", \
|
|
"--threads", "8", \
|
|
"--timeout", "30", \
|
|
"--access-logfile", "-", \
|
|
"--error-logfile", "-", \
|
|
"wsgi:app"]
|