ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d60ed602a13e26100be3ce100391cf58b3a7a5f0
shopdb-flask/docs/DEPLOY.md
cproudlock 417f8a3dd4 Keep a site's own files when its container is replaced 
`db_data` was a volume and the instance directory was not, so the documented
update path - `docker compose build api && up -d api` - recreated the container
and discarded everything the site had written. `plugins.json` is only the loud
part: maps, branding, model and application images, employee photos, warranty
proofs, slides, printed-part files and the Dell OAuth token all live under
instance_path too. MySQL rows survive and point at files that are gone, so the
second symptom is images 404ing rather than an error anybody sees.

Reported by an adopting site, which read it as having updated too fast. It had
not; nothing it could have done differently would have kept those files.

DEPLOY.md had been telling sites to back up `instance/` since it was written.
The template never gave them anything to back up.

The air-gap `migrate` service mounts the volume too, because
`flask plugin upgrade-all` rewrites plugins.json and that service exits
immediately after.

The image now creates instance/ ITSELF, owned by the app user. Docker seeds an
empty named volume from image content at the mountpoint, ownership included;
with no such directory in the image the mountpoint is created root-owned 0755
and the container, which runs as shopdb, cannot write into its own instance
directory. Caught by running the built image rather than by reading it: the
volume mounted clean and `touch` came back Permission denied. Verified fixed the
same way.

A stack that predates the volume needs its files moved across ONCE, while the
old container still exists - the volume is seeded from image content, and the
image ships instance/ empty, so it comes up empty rather than inheriting the old
container's writable layer. DEPLOY.md carries the procedure, including the chown
after `docker compose cp`, which writes files under the copying user's numeric
uid rather than the app user's.

Also here, found while checking what an upgrade actually runs: the connected
update steps ran `flask db upgrade` and stopped. Per-plugin Alembic chains
(ADR-008) are not part of that, so a connected site taking an image with a
bumped plugin migration ran the core chain and silently skipped every plugin
chain. The air-gap stack had it right all along. Both commands are in Step 9
now, plus a `db current` check against `db heads`.
2026-08-19 19:34:28 -04:00

286 lines
13 KiB
Markdown
Raw Blame History

# Per-Site Deployment Runbook
shopdb-flask is single-tenant per ADR-004. Each adopting facility runs its own stack: own DB, own users, own enabled plugins, own secrets. This document is the runbook for a fresh site deploy.
## Prerequisites
- Docker 24+ and Docker Compose v2 (or equivalent container runtime)
- A reverse proxy with TLS termination (nginx, traefik, Caddy, GE corporate LB) -- the framework does not terminate TLS itself
- A MySQL backup destination (offsite recommended)
- Access to the internal GE Aerospace git server, or a clone of the repo
## Step 1: Clone and configure
```bash
git clone <internal-git-server>/ge-aerospace/shopdb-flask.git
cd shopdb-flask
cp .env.example .env
```
Edit `.env`:
| Variable | Required | Notes |
|----------|----------|-------|
| `FLASK_ENV` | Yes | `production` for live sites |
| `SECRET_KEY` | Yes | `python -c "import secrets; print(secrets.token_urlsafe(64))"` |
| `JWT_SECRET_KEY` | Yes | Same generation, different value |
| `DATABASE_URL` | Yes | `mysql+pymysql://shopdb:PASSWORD@db:3306/shopdb_flask` (matches docker-compose) |
| `CORS_ORIGINS` | Yes | Comma-separated explicit origins. Wildcard rejected. |
| `MYSQL_ROOT_PASSWORD` | Yes | Container only |
| `MYSQL_PASSWORD` | Yes | Container only, must match `DATABASE_URL` password |
| `MYSQL_PORT` | No | Default 3306 |
| `API_PORT` | No | Default 5001 |
| `LOG_LEVEL` | No | Default INFO |
| `ZABBIX_URL`, `ZABBIX_TOKEN` | No | Only if printers plugin uses Zabbix |
| `COLLECTOR_API_KEY` | No | Shared key for `/api/collector/*` ingest. Required only if unattended collectors push data. Endpoint fails closed (denies) when unset. |
| `COLLECTOR_API_KEY_<PLUGIN>` | No | Per-plugin override (e.g. `COLLECTOR_API_KEY_COMPUTERS`), checked before the shared key (ADR-006) |
| `EMPLOYEE_DB_HOST/USER/PASSWORD/NAME` | No | Read-only HR directory for notifications + kiosks. No safe default for the password. |
## Step 2: Bring up the stack
```bash
docker compose build
docker compose up -d
```
The Docker image builds the Vue frontend in a first stage and copies the
compiled SPA into the API image, so `docker compose build` produces a
self-contained image with the UI already built. No separate Node step is
needed for a container deploy. (For a bare-metal/venv install instead, build
the frontend by hand: `cd frontend && npm ci && npm run build`, which writes
`frontend/dist/` for Flask to serve.)
The MySQL container initializes its volume on first run. The API container waits for `db` to be healthy via `healthcheck`. Check logs:
```bash
docker compose logs -f api
```
If `ProductionConfig.validate()` raises, the container exits with the offending env-var named in the log. Fix `.env` and `docker compose up -d` again.
## Step 3: Initialize the database schema
```bash
docker compose exec api flask db upgrade
docker compose exec api flask plugin upgrade-all
```
`flask db upgrade` applies the core Alembic chain: the baseline migration plus
every later migration, which together create all core AND bundled-plugin tables
through the chain head. `flask plugin upgrade-all` then stamps each bundled
plugin's own migration chain (the `alembic_version_<plugin>` tables) and applies
any plugin-specific migrations added after the ownership cutover. Both commands
are idempotent, so re-running them is safe. See ADR-008 for why plugin schema
splits into per-plugin chains from the cutover forward.
**Lean sites (ADR-014):** the core chain creates every bundled plugin's tables,
so a site that ships only some plugins still has the others' (empty) tables. To
carry only core + chosen-plugin tables, prune the rest once, at initial
provisioning, after the two commands above:
```bash
docker compose exec api flask plugin prune-schema # dry-run, review
docker compose exec api flask plugin prune-schema --yes --force
```
It drops the tables of every plugin not installed on this site. `--force` is
needed because the core chain seeds a few plugin reference tables (default
access protocols, etc.); at first provisioning those hold only seeded defaults,
before any site data. It refuses to drop a table that holds rows without
`--force`, so it is safe to leave out of routine upgrades - run it only when
provisioning a lean site or after deliberately removing a plugin. Installing a
pruned plugin later recreates its tables automatically.
**Charset:** the schema is utf8mb4 (`utf8mb4_unicode_ci`). The docker-compose `db` service sets `--character-set-server=utf8mb4`, so the auto-created `shopdb_flask` database is utf8mb4. If you point at an external MySQL instead of the bundled container, create the database as utf8mb4 first, or it inherits the server default (often latin1) and the schema silently drifts:
```sql
CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
```
`DATABASE_URL` must keep `?charset=utf8mb4` so the connection matches. On MySQL older than 5.7 also enable `innodb_large_prefix=ON` + `innodb_file_format=Barracuda`, or the utf8mb4 indexes exceed the 767-byte prefix limit (error 1071). MySQL 5.7+ and 8.0 need no extra config.
## Step 4: Seed permissions, settings, and reference data
Run all three seeders. They are idempotent, so re-running them on an existing
database is safe (it adds anything missing and leaves existing rows alone).
```bash
docker compose exec api flask seed permissions
docker compose exec api flask seed settings
docker compose exec api flask seed reference-data
```
- `seed permissions` - creates the RBAC permission rows and the default roles
the app checks with `@require_permission`. Run this before anyone logs in or
permission checks have nothing to match.
- `seed settings` - writes the default Setting rows (branding, ServiceNow
integration, floor-map placeholders, search toggles, site identity). A site
overrides these later in Settings or the setup wizard.
- `seed reference-data` - creates default `ModelType`, `AssetStatus`,
`LocationType`, `CommunicationType`, `OperatingSystem`, `RelationshipType` rows
seeded with the platform contract values (`partof`, `controls`, `connectedto`).
(`Vendor`, `Location`, and `BusinessUnit` are not seeded here; they come from
`seed demo`.)
## Step 5: Pick plugins to enable
The image bundles thirteen plugins (computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, usb, warranty). Only enabled plugins are loaded.
```bash
docker compose exec api flask plugin list
docker compose exec api flask plugin install computers
docker compose exec api flask plugin install machines
# ... repeat for each plugin the site tracks
```
To install a sister-site or third-party plugin (per ADR-003), drop its directory into `<repo>/plugins/<name>/` (the docker-compose mounts this read-only into the container) and run `flask plugin install <name>`.
## Step 6: Create the first admin (setup wizard)
The primary path is the first-run setup wizard. Once the stack is up and the DB
is seeded, browse to the site (through the reverse proxy configured in Step 7,
or directly at the API port during bring-up) and go to `/setup`. The wizard
creates the first admin account and captures site identity (facility name,
optional logo). It runs only while `setup_complete` is false; after it finishes
the route redirects to the app.
Headless alternative (no browser, e.g. automated provisioning):
```bash
docker compose exec api flask seed admin --username admin --email admin@facility.example.com
# Password is generated and printed once. Store in your password manager.
```
Subsequent users are managed through the UI.
## Step 7: Front the API with TLS
The Flask container listens on `5001/tcp` over plain HTTP. Production exposure must go through a reverse proxy that terminates TLS:
```nginx
server {
listen 443 ssl;
server_name shopdb.facility-a.example.com;
ssl_certificate /etc/ssl/certs/shopdb.crt;
ssl_certificate_key /etc/ssl/private/shopdb.key;
location / {
proxy_pass http://localhost:5001;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
```
The framework reads `X-Forwarded-For` for audit logging.
## Step 8: Backups
Per-site MySQL backups are the site's responsibility. Recommended: nightly `mysqldump` to offsite storage with 14-day retention.
```bash
docker compose exec -T db mysqldump -u root -p"${MYSQL_ROOT_PASSWORD}" shopdb_flask | gzip > backup-$(date +%F).sql.gz
```
Verify a restore quarterly. Back up the `instance/` directory alongside the DB;
it holds uploaded floor plans, branding, `plugins.json`, and tokens that are not
in MySQL. Under compose it is the `instance_data` named volume:
```bash
docker compose run --rm -v "$PWD:/backup" api tar czf /backup/instance-$(date +%F).tar.gz -C /app/instance .
```
See [docs/BACKUP-RESTORE.md](BACKUP-RESTORE.md) for the full backup and
restore procedure.
## Step 9: Updates
```bash
git pull origin main
docker compose build api
docker compose up -d api
docker compose exec api flask db upgrade
docker compose exec api flask plugin upgrade-all
docker compose exec api flask db current # must match `flask db heads`
```
`plugin upgrade-all` runs the per-plugin Alembic chains (ADR-008), which
`db upgrade` does NOT touch. The air-gap stack runs both in its one-shot
`migrate` service; a connected stack has to ask.
`up -d api` REPLACES the container. Everything the site has written lives in the
`instance_data` volume for exactly this reason: the enabled-plugin list
(`plugins.json`), uploaded floor plans and branding, model and application
images, employee photos, warranty proofs, slides, printed-part files, and the
Dell OAuth token. If your stack predates that volume, those files are in the old
container's writable layer and an update discards them. Move them across ONCE,
before the next rebuild:
```bash
docker compose cp api:/app/instance ./instance-rescued # BEFORE pulling new code
docker compose up -d api # creates the volume
docker compose cp ./instance-rescued/. api:/app/instance
docker compose exec -u root api chown -R shopdb:shopdb /app/instance
docker compose restart api
```
The `chown` is not optional. `docker compose cp` writes the files with the
copying user's numeric uid, which is only `shopdb` by coincidence if your host
account happens to be uid 1000. Get it wrong and the site reads its restored
files fine and cannot write new ones.
Then run the migrations and confirm the plugins came back:
```bash
docker compose exec api flask db upgrade
docker compose exec api flask plugin upgrade-all
docker compose exec api flask db current # must match `flask db heads`
docker compose exec api flask plugin list # the site's plugins, enabled
```
Restore `instance/` BEFORE `plugin upgrade-all`: that command works from
`plugins.json`, so running it against an empty instance directory upgrades
nothing and reports success.
The symptom of having missed this is a site that comes back with its plugins
disabled and image URLs that 404: the MySQL rows survived, the files did not.
Re-enabling by hand works, but `flask plugin apply-profile <profile.json>` puts
the same list back in one command and is the thing to keep in version control.
The framework's `__contract_version__` may have moved. Check `docs/adr/` for any new ADRs since the last update. If an ADR introduces a breaking change, the upgrade may require coordinated work; the ADR's "Consequences" section documents it. See [docs/UPGRADE.md](UPGRADE.md) for the full upgrade procedure, including re-seeding and the v0.5+ floor-plan note.
## Common issues
| Symptom | Cause | Fix |
|---------|-------|-----|
| `ConfigError: SECRET_KEY is required in production` | `.env` missing or blank | Set `SECRET_KEY` in `.env`, re-up |
| `ConfigError: CORS_ORIGINS must be a comma-separated allowlist` | `.env` has `*` | Set explicit origins |
| `PluginVersionError: requires core_version X but framework is Y` | Plugin pinned a too-narrow range | Update `manifest.json` `core_version` or pin framework version |
| 500s after `flask db upgrade` | Migration ran but app cached old schema | `docker compose restart api` |
| Cannot reach API after restart | Reverse proxy not pointing at the container's exposed port | Confirm `API_PORT` and proxy config |
## Health check
```bash
curl -s -X POST -H "Content-Type: application/json" \
-d '{}' http://localhost:5001/api/auth/login \
| jq .
# Expect: {"status": "error", "data": {"error": {"code": "VALIDATION_ERROR", ...}}}
```
If this returns a 500 or no JSON, the container is unhealthy. Check `docker compose logs api`.
## References
- [docs/adr/ADR-004-deployment-topology.md](adr/ADR-004-deployment-topology.md) - per-site instances rationale
- [docs/adr/ADR-003-plugin-distribution.md](adr/ADR-003-plugin-distribution.md) - bundled vs external plugins
- [docs/adr/ADR-006-collector-contract.md](adr/ADR-006-collector-contract.md) - per-plugin collector endpoints
- [docs/PLUGIN-QUICKSTART.md](PLUGIN-QUICKSTART.md) - building a custom plugin for your site
- [docs/CONFIG.md](CONFIG.md) - every environment variable and every Setting key
- [docs/UPGRADE.md](UPGRADE.md) - upgrade procedure for an existing site
- [docs/BACKUP-RESTORE.md](BACKUP-RESTORE.md) - backup and restore procedure
- [shopdb/config.py](../shopdb/config.py) - all the env-vars in one place
Reference in New Issue View Git Blame Copy Permalink