main was 16 days and 259 commits behind the line the product is actually built on, at contract 0.30.0 against 0.35.0, with no per-PC credential code at all. It was neither current nor curated - just stale. This brings it forward and, at the same time, makes it mean something: main is what a receiving site installs, and nothing else. WHAT IT CARRIES. Twelve plugins - backups, computers, endpoints, guides, knowledgebase, machines, network, notifications, printers, slides, tools, warranty - plus applications, which is manifest-less core and always ships. The only manifest dependency inside that set is endpoints on computers, and both are in it, so the closure is self-contained. WHAT IT DOES NOT. employees, geenforce, measuringtools, printedparts and usb stay on the site line. So do the Windows installer tree, the VM rehearsal helpers, the agent tooling config, one site's data loader, the internal audit and findings records, and the planner spreadsheet. 1,135 tracked files against 1,438. A REMOVAL THAT BREAKS THE APP IS NOT A REMOVAL, and two nearly shipped: deploy/site-profile-universal.json still named all seventeen plugins. It is the file docs/DEPLOY.md tells a site to provision from, so a fresh install would have failed at `flask plugin apply-profile` with PluginError - the first thing a receiving site does, broken by the branch meant to serve them. It now names the twelve that exist, and its comment says the two must agree. agent/ was never refreshed. Whoever populated this branch copied shopdb/, plugins/, frontend/ and tests/ from the site line and left the agent tree at its September state: 22 C# and PowerShell sources missing, including AgentCredentials, EnrollmentService, AgentHttpFactory and the fleet-alert client, plus one file the site line had deleted still present. The old tree is internally consistent, so it compiles, and nothing in CI builds it - every gate was green over a server that speaks enrollment and per-PC credentials to a bundled agent source that cannot. Resynced whole. DECISIONS THIS ENCODES, rather than defers: - The plugin-authoring lab builds printedparts, whose manifest depends on employees, so it could not stay. The lab remains on the site line and main ships no build-along tutorial. That is a real loss for an adopting developer and is recorded here rather than discovered later. - Recognition, Training and Recertification notifications are no longer seeded. Each splits one notification into a card per employee and renders a photo, which needs a plugin this branch does not bundle. Seeding them would offer an author a type that cannot resolve anybody. A site that installs employees gets them from that plugin's own seed. - Windows/IIS is documented as the legacy path throughout, and the README leads with Docker, because that is what a receiving site gets. The API description is regenerated for THIS branch - 347 paths and 483 operations, down from 415 and 573 - so the published contract describes what is here rather than what the site line serves. docs/CONFIG.md lost the two measuring-tool label keys the code no longer declares, which two lanes had decided opposite ways. Suite: 3124 passed, 6 skipped. Naming and style gates pass. The docs gates run rather than skip, and every index line count is derived from the file it names.
501 lines
27 KiB
Markdown
501 lines
27 KiB
Markdown
<!-- docmeta
|
|
purpose: howto
|
|
audience: adopter, developer
|
|
status: current
|
|
summary: Stand up a new site on Docker at a connected facility, from clone to first admin to TLS.
|
|
loadwith: none
|
|
-->
|
|
# Deploy a site with Docker (connected facility)
|
|
|
|
> **Topology: Docker, at a site with network access. This is the path a new
|
|
> site takes,** and the only one this line ships. No internet at all ->
|
|
> [DEPLOY-AIRGAP](DEPLOY-AIRGAP.md).
|
|
> **Where this sits:** **install** -> operate -> update -> upgrade.
|
|
> Forward: [UPGRADE](UPGRADE.md) for later versions,
|
|
> [BACKUP-RESTORE](BACKUP-RESTORE.md) for day-to-day custody of the data.
|
|
|
|
shopdb-flask is single-tenant per ADR-004. Each adopting facility runs its own
|
|
stack: own database, own users, own enabled plugins, own secrets. This page is
|
|
the runbook for a fresh site deploy that has network access, and it is also the
|
|
reference for the environment variables and for how the application is published
|
|
and learns who is calling it - other pages link here rather than restate it.
|
|
|
|
### Which compose file
|
|
|
|
| File | For |
|
|
|---|---|
|
|
| `docker-compose.yml` | this page: a site that can build the image from source |
|
|
| `docker-compose.prebuilt.yml` | the same stack built from an already-compiled `frontend/dist/`, for a box where npm cannot reach the registry. See [DOCKER-PREBUILT-APPROACH](deployment/DOCKER-PREBUILT-APPROACH.md) |
|
|
| `docker-compose.airgap.yml` | [DEPLOY-AIRGAP](DEPLOY-AIRGAP.md): runs a loaded image, never builds, and migrates itself with a one-shot `migrate` service |
|
|
| `docker-compose.pull.yml` | dev and pilot workstations that pull a published channel image instead of building |
|
|
| `docker-compose.channel.yml` | overlay adding `SHOPDB_ENV_LABEL` so a stable and an edge stack can run side by side |
|
|
| `docker-compose.backup.yml` | opt-in backup overlay (restic repository, watcher, one-shot). See [BACKUP-RESTORE](BACKUP-RESTORE.md) |
|
|
|
|
## Prerequisites
|
|
|
|
- Docker 24+ and Docker Compose v2 (or equivalent container runtime)
|
|
- A reverse proxy with TLS termination (nginx, traefik, Caddy, a corporate load balancer), or the certificate pair described in Step 2 - the application itself is not a TLS terminator by design
|
|
- A MySQL backup destination (offsite recommended)
|
|
- A clone of the repository (the product is distributed as source, not as a public image)
|
|
|
|
## Step 1: Clone and configure
|
|
|
|
```bash
|
|
git clone <your-remote>/shopdb-flask.git
|
|
cd shopdb-flask
|
|
cp .env.example .env
|
|
```
|
|
|
|
Edit `.env`:
|
|
|
|
| Variable | Required | Notes |
|
|
|----------|----------|-------|
|
|
| `FLASK_ENV` | Yes | `production` for live sites |
|
|
| `SECRET_KEY` | Yes | `python -c "import secrets; print(secrets.token_urlsafe(64))"` |
|
|
| `JWT_SECRET_KEY` | Yes | Same generation, different value |
|
|
| `DATABASE_URL` | venv only | `mysql+pymysql://shopdb:PASSWORD@db:3306/shopdb_flask?charset=utf8mb4`. Under this compose file the services build their own from `MYSQL_PASSWORD` and never read this line - see the note below |
|
|
| `CORS_ORIGINS` | Yes | Comma-separated explicit origins. Wildcard rejected. |
|
|
| `MYSQL_ROOT_PASSWORD` | Yes | Container only |
|
|
| `MYSQL_PASSWORD` | Yes | Container only, must match `DATABASE_URL` password |
|
|
| `MYSQL_PORT` | No | Default 3306 |
|
|
| `API_PORT` | No | Default 5001 |
|
|
| `LOG_LEVEL` | No | Default INFO |
|
|
| `ZABBIX_URL`, `ZABBIX_TOKEN` | No | Only if printers plugin uses Zabbix |
|
|
| `COLLECTOR_API_KEY` | No | Shared key for `/api/collector/*` ingest. Required only if unattended collectors push data. Endpoint fails closed (denies) when unset. |
|
|
| `COLLECTOR_API_KEY_<PLUGIN>` | No | Per-plugin override (e.g. `COLLECTOR_API_KEY_COMPUTERS`), checked before the shared key (ADR-006) |
|
|
| `EMPLOYEE_DB_HOST/USER/PASSWORD/NAME` | No | Read-only HR directory for notifications + kiosks. No safe default for the password. |
|
|
| `TRUSTED_PROXY_COUNT` | No | venv/IIS only. Under compose it is pinned to 1 for the bundled nginx. See [Step 7](#step-7-tls-and-the-proxy-that-is-already-there) |
|
|
|
|
[CONFIG](CONFIG.md) and `shopdb/config.py` are authoritative for every other
|
|
variable and its default.
|
|
|
|
**`.env` is not handed to the container wholesale.** None of the compose files
|
|
use `env_file:`; compose substitutes `.env` values into the `environment:` block
|
|
it declares, and only the variables named there reach the application. As
|
|
shipped, the `api` service forwards `SECRET_KEY`, `JWT_SECRET_KEY`,
|
|
`CORS_ORIGINS`, `LOG_LEVEL`, `ZABBIX_URL`, `ZABBIX_TOKEN`, `COLLECTOR_API_KEY`,
|
|
`COLLECTOR_API_KEY_ENDPOINTS` and `SHOPDB_ENV_LABEL`. Anything else in this
|
|
table - a per-plugin collector key for some other plugin, the `EMPLOYEE_DB_*`
|
|
set, the `CMMC_USB_DB_*` set, `PLUGIN_TRUSTED_KEYS` - needs a line adding to
|
|
that block as well, or it is set in a file the application never reads.
|
|
|
|
Two entries are set by the compose file itself and ignored from `.env`:
|
|
|
|
- **`DATABASE_URL`** is composed in-file as
|
|
`mysql+pymysql://shopdb:${MYSQL_PASSWORD}@db:3306/shopdb_flask?charset=utf8mb4`,
|
|
on both `api` and `migrate`. `MYSQL_PASSWORD` is therefore the value that
|
|
matters; editing `DATABASE_URL` in `.env` changes nothing under compose.
|
|
- **`TRUSTED_PROXY_COUNT`** is pinned to 1 because the stack contains its own
|
|
nginx. See [Step 7](#step-7-tls-and-the-proxy-that-is-already-there).
|
|
|
|
`SHOPDB_ENV_LABEL` defaults to `DEV` here on purpose: this file is the
|
|
source-build template, and a raw source build is by definition not a release.
|
|
Set it to the site's label for a production stack.
|
|
|
|
## Step 2: Bring up the stack
|
|
|
|
```bash
|
|
docker compose build
|
|
docker compose up -d
|
|
```
|
|
|
|
### What comes up
|
|
|
|
Four services, and knowing which is which saves most of the debugging:
|
|
|
|
| Service | Does | Listens |
|
|
|---|---|---|
|
|
| `db` | MySQL 8.4, utf8mb4 server-wide, volume `db_data` | `127.0.0.1:${MYSQL_PORT:-3306}` - loopback only, for `mysqldump` and a local client. `api` reaches it over the compose network regardless |
|
|
| `migrate` | schema, plugin bootstrap and seeds, then exits. `restart: "no"` | nothing |
|
|
| `api` | gunicorn, gevent workers, serves the compiled SPA | `127.0.0.1:5002` - debugging bypass only |
|
|
| `nginx` | **terminates TLS** and proxies to `api` | `${API_PORT:-5001}` -> 443 |
|
|
|
|
So the address a browser uses is `https://<host>:${API_PORT}`, served by nginx,
|
|
and `5002` is a back door for `curl` on the host.
|
|
|
|
**TLS lives in nginx, not in gunicorn.** `nginx.conf` reads
|
|
`/etc/nginx/certs/server.crt` and `server.key`, and `./certs` is mounted there
|
|
read-only. The repository ships a self-signed pair and `certs/openssl.cnf`;
|
|
replace both files with the site's own certificate and
|
|
`docker compose restart nginx`. The gunicorn command carries no `--certfile`,
|
|
by design - one TLS terminator, one place to rotate a key.
|
|
|
|
The reason there is TLS at all rather than plain http: browser APIs that require
|
|
a secure context, `navigator.clipboard` among them, refuse to run over plain
|
|
http on a non-localhost origin, so a stack reached by hostname needs real TLS
|
|
somewhere.
|
|
|
|
**Why gevent and not threads.** The endpoint agent long-polls
|
|
`/api/endpoints/commands/next` and re-polls immediately on a 204, so every agent
|
|
holds a connection essentially continuously and fleet size IS the concurrency.
|
|
Under threads that was 4 x 8 = 32 slots for the whole site, and a worker's
|
|
threads filled with sleeping agents while the CPU sat idle. A held connection
|
|
under gevent is a greenlet, so idle is nearly free. Before changing `--workers`,
|
|
read the connection arithmetic in the comment beside it: workers x (pool size +
|
|
overflow) has to stay under the MySQL `max_connections`, and at the shipped 4
|
|
workers it is already about 90 of 151.
|
|
|
|
### Building where npm cannot reach the registry
|
|
|
|
The image builds the Vue frontend in a first stage and copies the compiled SPA
|
|
in, so `docker compose build` needs no separate Node step. Behind a proxy that
|
|
re-signs HTTPS (Zscaler and the like), that first stage is where the build dies:
|
|
npm inside the container does not trust the corporate root CA, and the failure
|
|
reads `npm ci failed` or `Exit handler never called`.
|
|
|
|
Two ways out, in order of preference:
|
|
|
|
1. **Give the build the corporate root CA.** Export it to `corp-root-ca.pem` in
|
|
the repository root - [DEVELOPMENT-SETUP](DEVELOPMENT-SETUP.md#0b-corp-network-ssl-cert---if-you-are-behind-a-gezscaler-proxy)
|
|
has the one-line PowerShell that bundles every trusted root - then
|
|
`docker compose build --no-cache`.
|
|
2. **Build the frontend on the host and skip npm in the image entirely.** That
|
|
is `docker-compose.prebuilt.yml` plus `Dockerfile.prebuilt`, and it is also
|
|
two to three minutes instead of fifteen. See
|
|
[DOCKER-PREBUILT-APPROACH](deployment/DOCKER-PREBUILT-APPROACH.md).
|
|
|
|
(For a bare-metal venv install instead: `cd frontend && npm ci && npm run build`,
|
|
which writes `frontend/dist/` for Flask to serve.)
|
|
|
|
The MySQL container initializes its volume on first run; its healthcheck probes
|
|
over TCP rather than the socket, because the socket answers during init while
|
|
the server is still about to restart. Watch the bring-up with
|
|
`docker compose logs -f migrate api`: if `ProductionConfig.validate()` raises,
|
|
the container exits with the offending env-var named in the log. Fix `.env` and
|
|
`docker compose up -d` again.
|
|
|
|
## Step 3: The schema, and what the `migrate` service already did
|
|
|
|
**On this stack you do not run these by hand.** The `migrate` service runs them
|
|
on every `up`, in one place and one order, and `api` will not start until it
|
|
exits 0:
|
|
|
|
```
|
|
flask db upgrade && /app/scripts/bootstrap-plugins.sh && flask plugin upgrade-all
|
|
&& flask seed permissions && flask seed settings && flask seed reference-data
|
|
```
|
|
|
|
`up -d api` still migrates first: compose honours
|
|
`service_completed_successfully` even when a single service is named. A failing
|
|
migration exits once and leaves the error as the last line of
|
|
`docker compose logs migrate`, rather than crash-looping under a restart policy
|
|
and burying it.
|
|
|
|
The rest of this section says what each command covers, because you will need
|
|
that when one of them fails, when you are on a venv install with no `migrate`
|
|
service, or when you are pruning a lean site (below). To run one by hand:
|
|
`docker compose exec api flask <command>`.
|
|
|
|
`flask db upgrade` applies the core Alembic chain: the baseline migration plus
|
|
every later migration, which together create all core AND bundled-plugin tables
|
|
through the chain head. `flask plugin upgrade-all` then stamps each bundled
|
|
plugin's own migration chain (the `alembic_version_<plugin>` tables) and applies
|
|
any plugin-specific migrations added after the ownership cutover. Both commands
|
|
are idempotent, so re-running them is safe. See ADR-008 for why plugin schema
|
|
splits into per-plugin chains from the cutover forward.
|
|
|
|
**Lean sites (ADR-014):** the core chain creates every bundled plugin's tables,
|
|
so a site that ships only some plugins still has the others' (empty) tables. To
|
|
carry only core + chosen-plugin tables, prune the rest once, at initial
|
|
provisioning, after the two commands above:
|
|
|
|
```bash
|
|
docker compose exec api flask plugin prune-schema # dry-run, review
|
|
docker compose exec api flask plugin prune-schema --yes --force
|
|
```
|
|
|
|
It drops the tables of every plugin not installed on this site. `--force` is
|
|
needed because the core chain seeds a few plugin reference tables (default
|
|
access protocols, etc.); at first provisioning those hold only seeded defaults,
|
|
before any site data. It refuses to drop a table that holds rows without
|
|
`--force`, so it is safe to leave out of routine upgrades - run it only when
|
|
provisioning a lean site or after deliberately removing a plugin. Installing a
|
|
pruned plugin later recreates its tables automatically.
|
|
|
|
**Charset:** the schema is utf8mb4 (`utf8mb4_unicode_ci`). The docker-compose `db` service sets `--character-set-server=utf8mb4`, so the auto-created `shopdb_flask` database is utf8mb4. If you point at an external MySQL instead of the bundled container, create the database as utf8mb4 first, or it inherits the server default (often latin1) and the schema silently drifts:
|
|
|
|
```sql
|
|
CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
|
```
|
|
|
|
`DATABASE_URL` must keep `?charset=utf8mb4` so the connection matches. On MySQL older than 5.7 also enable `innodb_large_prefix=ON` + `innodb_file_format=Barracuda`, or the utf8mb4 indexes exceed the 767-byte prefix limit (error 1071). MySQL 5.7+ and 8.0 need no extra config.
|
|
|
|
## Step 4: Seed permissions, settings, and reference data
|
|
|
|
The `migrate` service runs all three on every `up`, and they are idempotent, so
|
|
this is a no-op once a site is current and is how a later release's new
|
|
permission or new default setting reaches an existing site. They are listed here
|
|
because you need to know what they cover, and because a venv install runs them
|
|
itself:
|
|
|
|
```bash
|
|
docker compose exec api flask seed permissions
|
|
docker compose exec api flask seed settings
|
|
docker compose exec api flask seed reference-data
|
|
```
|
|
|
|
- `seed permissions` - creates the RBAC permission rows and the default roles
|
|
the app checks with `@require_permission`. Run this before anyone logs in or
|
|
permission checks have nothing to match.
|
|
- `seed settings` - writes the default Setting rows (branding, ServiceNow
|
|
integration, floor-map placeholders, search toggles, site identity). A site
|
|
overrides these later in Settings or the setup wizard.
|
|
- `seed reference-data` - creates default `ModelType`, `AssetStatus`,
|
|
`LocationType`, `CommunicationType`, `OperatingSystem`, `RelationshipType` rows
|
|
seeded with the platform contract values (`partof`, `controls`, `connectedto`).
|
|
(`Vendor`, `Location`, and `BusinessUnit` are not seeded here; they come from
|
|
`seed demo`.)
|
|
|
|
## Step 5: Pick plugins to enable
|
|
|
|
The image bundles every plugin in the tree. [PROJECT-MAP](PROJECT-MAP.md) lists
|
|
them with their versions, generated from the code; [PLUGINS](PLUGINS.md) is the
|
|
page for deciding which ones a site should carry. Only enabled plugins are
|
|
loaded.
|
|
|
|
`scripts/bootstrap-plugins.sh`, inside the `migrate` service, has already
|
|
installed a starting set on a site with no `instance/plugins.json`: backups,
|
|
computers, endpoints, guides, knowledgebase, machines, network, notifications,
|
|
printers, slides, tools, warranty. It installs them **disabled**, and it skips
|
|
itself entirely once the registry file exists, so it never fights a site's own
|
|
choices.
|
|
|
|
That is every bundled plugin, so on this line the starting set and the shelf
|
|
are the same list - a site turns one on rather than installing it:
|
|
|
|
```bash
|
|
docker compose exec api flask plugin list
|
|
docker compose exec api flask plugin enable warranty
|
|
```
|
|
|
|
Or put the whole list in a profile and keep that in version control, which is
|
|
the thing to do when more than one site is involved:
|
|
|
|
```bash
|
|
docker compose exec api flask plugin apply-profile deploy/site-profile-universal.json
|
|
```
|
|
|
|
`deploy/site-profile.example.json` is the template to copy for a site of your
|
|
own. Enabling a plugin needs `docker compose restart api` to take effect.
|
|
|
|
To install a sister-site or third-party plugin (per ADR-003), drop its directory into `<repo>/plugins/<name>/` (the docker-compose mounts this read-only into the container) and run `flask plugin install <name>`.
|
|
|
|
## Step 6: Create the first admin (setup wizard)
|
|
|
|
The primary path is the first-run setup wizard. Once the stack is up and the DB
|
|
is seeded, browse to the site (through the reverse proxy configured in Step 7,
|
|
or directly at the API port during bring-up) and go to `/setup`. The wizard
|
|
creates the first admin account and captures site identity (facility name,
|
|
optional logo). It runs only while `setup_complete` is false; after it finishes
|
|
the route redirects to the app.
|
|
|
|
Headless alternative (no browser, e.g. automated provisioning):
|
|
|
|
```bash
|
|
docker compose exec api flask seed admin --username admin --email admin@facility.example.com
|
|
# Password is generated and printed once. Store in your password manager.
|
|
```
|
|
|
|
Subsequent users are managed through the UI.
|
|
|
|
## Step 7: TLS, and the proxy that is already there
|
|
|
|
**The stack ships its own reverse proxy.** The `nginx` service terminates TLS on
|
|
`${API_PORT:-5001}` and proxies to `api` over the compose network, forwarding
|
|
`X-Real-IP`, `X-Forwarded-For`, `X-Forwarded-Proto` and `Host`, with 90-second
|
|
proxy timeouts so an agent's long poll is not cut mid-hold. For most sites
|
|
Step 7 is therefore: drop your certificate into `./certs`, set `API_PORT`, and
|
|
stop.
|
|
|
|
Put a corporate load balancer or an existing nginx in FRONT of that only if the
|
|
site already has one. The shape then is proxy -> nginx -> api, which is two
|
|
proxies, and `TRUSTED_PROXY_COUNT` has to say so (below). A site that would
|
|
rather terminate TLS in its own proxy can instead remove the `nginx` service and
|
|
publish `api` directly - a configuration like this one:
|
|
|
|
```nginx
|
|
server {
|
|
listen 443 ssl;
|
|
server_name shopdb.facility-a.example.com;
|
|
|
|
ssl_certificate /etc/ssl/certs/shopdb.crt;
|
|
ssl_certificate_key /etc/ssl/private/shopdb.key;
|
|
|
|
location / {
|
|
proxy_pass http://localhost:5001;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
proxy_set_header X-Forwarded-Proto $scheme;
|
|
}
|
|
}
|
|
```
|
|
|
|
(That block proxies to plain http, so it belongs on the same host as the
|
|
container or on a trusted network segment.)
|
|
|
|
### Real client addresses: `TRUSTED_PROXY_COUNT`
|
|
|
|
Behind a proxy the application sees the proxy's connection, not the browser's.
|
|
`TRUSTED_PROXY_COUNT` is the number of proxies **actually** in front of this
|
|
instance, and the application then enables ProxyFix, which resolves the client
|
|
address, scheme, host and prefix from the `X-Forwarded-*` headers.
|
|
|
|
**`docker-compose.yml` already sets it to 1 on the `api` service**, because the
|
|
bundled nginx is that one proxy. It is not read from `.env` there - the value
|
|
belongs to the stack's shape, not to the site. Change it in the compose file,
|
|
not in `.env`, and only when you change the shape: 2 if you put a corporate load
|
|
balancer in front of nginx, 0 if you delete the nginx service and publish `api`
|
|
directly with nothing ahead of it.
|
|
|
|
- **It is 0 by default, deliberately.** ProxyFix makes the application trust
|
|
those headers, and any caller can send them. Enabled with nothing in front,
|
|
a caller can claim a different source address or scheme - and the login rate
|
|
limit and the account lockout are keyed on the client address.
|
|
- **Leave it at 0 behind a proxy** and the application is not wrong about the
|
|
client so much as blind to it: every request looks like it came from the
|
|
proxy. The whole site then shares one rate-limit bucket, and the audit log
|
|
records the proxy for every action.
|
|
- Getting the scheme wrong has its own symptom: an absolute redirect built from
|
|
the proxy's view sends an https page to an http URL the browser refuses. That
|
|
is what made a redirect on the endpoints list unfollowable behind IIS.
|
|
- `.env.example` also carries `TRUSTED_PROXY_COUNT`, defaulting to 0. That line
|
|
is for a venv or IIS install, where nothing sets it for you; under this
|
|
compose file the compose value wins and the `.env` line is inert.
|
|
|
|
`CORS_ORIGINS` is an exact origin match (scheme + host + port, no path), so it
|
|
has to name the address the browser actually uses, not the container's.
|
|
|
|
## Step 8: Backups
|
|
|
|
Per-site backups are the site's responsibility: nightly dump to offsite storage,
|
|
14-day retention, a restore rehearsed quarterly. The database is not the whole
|
|
backup - `instance/` holds uploaded floor plans, branding, `plugins.json` and
|
|
tokens that are not in MySQL, and under compose it is the `instance_data` named
|
|
volume. [BACKUP-RESTORE](BACKUP-RESTORE.md) is the procedure for both, including
|
|
the compose backup overlay.
|
|
|
|
```bash
|
|
docker compose exec -T db mysqldump -u root -p"${MYSQL_ROOT_PASSWORD}" shopdb_flask | gzip > backup-$(date +%F).sql.gz
|
|
docker compose run --rm -v "$PWD:/backup" api tar czf /backup/instance-$(date +%F).tar.gz -C /app/instance .
|
|
```
|
|
|
|
## Step 9: Updates
|
|
|
|
Moving this site to a newer version is [UPGRADE](UPGRADE.md) - back up, pull,
|
|
rebuild, migrate, re-seed, restart, plus the version-specific notes. One thing
|
|
belongs here rather than there, because it is about how this stack is built:
|
|
|
|
### The `instance_data` volume, and stacks that predate it
|
|
|
|
`up -d api` REPLACES the container. Everything the site has written lives in the
|
|
`instance_data` volume for exactly this reason: the enabled-plugin list
|
|
(`plugins.json`), uploaded floor plans and branding, model and application
|
|
images, employee photos, warranty proofs, slides, printed-part files, and the
|
|
Dell OAuth token.
|
|
|
|
**`docker compose down` is safe; `docker compose down -v` is not.** The `-v`
|
|
deletes the named volumes, which is both `instance_data` and `db_data`: the
|
|
site's files and its database in one command. There is no confirmation prompt.
|
|
|
|
If your stack predates that volume, those files are in the old container's
|
|
writable layer and an update discards them. Move them across ONCE, before the
|
|
next rebuild:
|
|
|
|
```bash
|
|
docker compose cp api:/app/instance ./instance-rescued # BEFORE pulling new code
|
|
docker compose up -d api # creates the volume
|
|
docker compose cp ./instance-rescued/. api:/app/instance
|
|
docker compose exec -u root api chown -R shopdb:shopdb /app/instance
|
|
docker compose restart api
|
|
|
|
docker compose exec api flask plugin list # the site's plugins, enabled again
|
|
```
|
|
|
|
Two things decide whether this works.
|
|
|
|
- **The `chown` is not optional.** `docker compose cp` writes the files with the
|
|
copying user's numeric uid, which is only `shopdb` by coincidence if your host
|
|
account happens to be uid 1000. Get it wrong and the site reads its restored
|
|
files fine and cannot write new ones. Every copy into a container in these
|
|
pages ends with this same `chown`, for this reason.
|
|
- **Restore `instance/` BEFORE `plugin upgrade-all`.** That command works from
|
|
`plugins.json`, so against an empty instance directory it upgrades nothing and
|
|
reports success.
|
|
|
|
The symptom of having missed it is a site that comes back with its plugins
|
|
disabled and image URLs that 404: the MySQL rows survived, the files did not.
|
|
Re-enabling by hand works, but `flask plugin apply-profile <profile.json>` puts
|
|
the same list back in one command and is the thing to keep in version control.
|
|
|
|
## Importing an existing database
|
|
|
|
A site coming from an earlier ShopDB, or a test stack loading a copy of
|
|
production, restores the dump and then brings the schema forward:
|
|
|
|
```bash
|
|
docker compose exec -T db mysql -u root -p"${MYSQL_ROOT_PASSWORD}" shopdb_flask < dump.sql
|
|
docker compose up -d migrate # re-runs upgrade, bootstrap and the seeds
|
|
docker compose restart api
|
|
```
|
|
|
|
The order matters and so does the second line. A restored dump carries plugin
|
|
TABLES but not `instance/plugins.json`, which is where the enabled-plugin list
|
|
lives (ADR-008 and the note in Step 9). Without a bootstrap pass the site comes
|
|
up with a full schema and no plugins registered, which reads as "all my data is
|
|
gone". `flask plugin apply-profile` puts a known list back in one command.
|
|
|
|
Restore `instance/` at the same time if you have it - see Step 9.
|
|
|
|
## Common issues
|
|
|
|
| Symptom | Cause | Fix |
|
|
|---------|-------|-----|
|
|
| `ConfigError: SECRET_KEY is required in production` | `.env` missing or blank | Set `SECRET_KEY` in `.env`, re-up |
|
|
| `ConfigError: CORS_ORIGINS must be a comma-separated allowlist` | `.env` has `*` | Set explicit origins |
|
|
| `PluginVersionError: requires core_version X but framework is Y` | Plugin pinned a too-narrow range | Update `manifest.json` `core_version` or pin framework version |
|
|
| 500s after `flask db upgrade` | Migration ran but app cached old schema | `docker compose restart api` |
|
|
| Cannot reach API after restart | Reverse proxy not pointing at the container's exposed port | Confirm `API_PORT` and proxy config |
|
|
| `npm ci failed` / `Exit handler never called` during build | A proxy re-signs HTTPS and npm in the build stage does not trust the corporate root CA | Stage `corp-root-ca.pem` in the repo root and `docker compose build --no-cache`, or build the frontend on the host ([DOCKER-PREBUILT-APPROACH](deployment/DOCKER-PREBUILT-APPROACH.md)) |
|
|
| `database is unhealthy`, or `db` never goes green | MySQL still initializing, or the published port is taken on the host | `docker compose logs db`; if the port clashes, set `MYSQL_PORT=3307` in `.env`. The mapping is loopback-only and does not affect `api` |
|
|
| `Access denied for user 'shopdb'` | `MYSQL_PASSWORD` changed after the `db_data` volume was initialized | MySQL sets the app user's password only on first init. Change it in MySQL, or drop the volume on a site with nothing in it yet |
|
|
| `Could not connect to database` from `api` or `migrate` | Something is pointing at `localhost` instead of the `db` service | Inside the stack the host is `db`. `localhost` in a container is that container |
|
|
| `ERR_CERT_AUTHORITY_INVALID` in the browser | The shipped `certs/server.crt` is self-signed | Expected on a first bring-up. Replace the pair in `certs/` with the site's own certificate and `docker compose restart nginx` |
|
|
| A plugin is installed but nothing appears in the nav | Installed is not enabled, and enabling needs a restart | `flask plugin enable <name>` then `docker compose restart api` |
|
|
| The first build takes 15 minutes | npm install and the Vite build inside the image | Expected once; later builds hit the layer cache. To avoid npm in the image entirely see [DOCKER-PREBUILT-APPROACH](deployment/DOCKER-PREBUILT-APPROACH.md) |
|
|
|
|
## Health check
|
|
|
|
```bash
|
|
# Through nginx, on API_PORT. -k because the shipped certificate is self-signed
|
|
# until you replace it.
|
|
curl -sk -X POST -H "Content-Type: application/json" \
|
|
-d '{}' https://localhost:5001/api/auth/login | jq .
|
|
# Expect: {"status": "error", "data": {"error": {"code": "VALIDATION_ERROR", ...}}}
|
|
|
|
curl -sk https://localhost:5001/api/dashboard/health # no authentication needed
|
|
```
|
|
|
|
`api` has its own healthcheck over plain http inside the stack, and the
|
|
debugging bypass on the host is `curl -s http://127.0.0.1:5002/api/dashboard/health`.
|
|
Comparing the two separates "the app is broken" from "nginx or the certificate
|
|
is broken": bypass green and `:5001` red means the problem is TLS or the proxy.
|
|
If both are red or return no JSON, `docker compose logs api`.
|
|
|
|
## References
|
|
|
|
- [UPGRADE](UPGRADE.md) - moving this site to a newer version
|
|
- [DOCKER-PREBUILT-APPROACH](deployment/DOCKER-PREBUILT-APPROACH.md) - building
|
|
the image where npm cannot reach the registry
|
|
- [DEV-BUILD-TEST](DEV-BUILD-TEST.md) - pulling a prebuilt stack onto a Windows
|
|
box to look at it
|
|
- [DEPLOY-AIRGAP](DEPLOY-AIRGAP.md) - the same stack where nothing can be pulled
|
|
- [BACKUP-RESTORE](BACKUP-RESTORE.md) - backup and restore procedure
|
|
- [CONFIG](CONFIG.md) - every environment variable and every Setting key
|
|
- [PLUGIN-QUICKSTART](PLUGIN-QUICKSTART.md) - building a custom plugin for your
|
|
site
|
|
- [ADR-004](adr/ADR-004-deployment-topology.md) - per-site instances rationale;
|
|
[ADR-003](adr/ADR-003-plugin-distribution.md) bundled vs external plugins;
|
|
[ADR-006](adr/ADR-006-collector-contract.md) per-plugin collector endpoints
|
|
- [shopdb/config.py](../shopdb/config.py) - all the env vars in one place
|