ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: one manual runbook, and ADR statuses that mean something

Browse Source 
DEPLOY-WINDOWS-IIS was a second copy of the manual IIS procedure that had
diverged from the first: a different MySQL version (8.0, which reached end of
life in April), a different port, a different plugin list, and a profile file
that does not exist. Two runbooks for one procedure means a reader follows
whichever they found, and one of them was wrong. INSTALL-WINDOWS-IIS covers
everything it did plus a preflight step and the subpath method, so the one
section it uniquely had - redeploying a hand-built server - is folded in there,
with the plugin-chain step it was missing and a note to back up first, and the
duplicate is gone. Everything that pointed at it now points at the survivor.

Three ADR statuses said something untrue.

ADR-013 said PROPOSED while half of it had shipped and ADR-014 had been accepted
on top of it. A decision that has been implemented and depended upon is not
proposed, and leaving one that way devalues every other status in the index. The
catalog half is still unbuilt, which is the ordinary state of an accepted
decision: accepted means settled, not delivered.

ADR-016 said ACCEPTED for a design where nothing is built - the endpoint and
permissions it describes do not exist, so a reader goes looking for them. The
status stands, because the decision does; the header now says so plainly and
points at where today's credentials actually live.

ADR-003 and ADR-004 were ACCEPTED with their own Decision lines still opening
"**PROPOSED:**", which reads as though the decision was never taken.

And the dashboard proposal carried Status: ACCEPTED, which belongs to a decision
record. A proposal is a proposal; the contract it produced is the ADR.
This commit is contained in:
cproudlock
2026-08-14 16:20:59 -04:00
parent 928a50c16e
commit 96df19702e
11 changed files with 60 additions and 240 deletions
Download Patch File Download Diff File Expand all files Collapse all files

223
docs/DEPLOY-WINDOWS-IIS.md
View File

@@ -1,223 +0,0 @@
# Deploy shopdb-flask to Windows IIS (MySQL 5.6)
> **Not the route for a new site.** Sister sites install from the Windows
> installer - one `.exe`, no manual IIS work: **[INSTALL-WINDOWS.md](INSTALL-WINDOWS.md)**.
>
> This is the **manual** procedure for the the reference site server, which was built
> by hand against its existing MySQL 5.6 and predates the installer. Keep it for
> that box.
Runbook for standing up a single-site instance on the production Windows Server
that already runs the classic ASP shopdb, using IIS + HttpPlatformHandler +
waitress, against the existing MySQL 5.6. This is the test-instance path; keep
developing on the Linux dev box and redeploy as needed.
The Docker path in `DEPLOY.md` does NOT apply on Windows (gunicorn is Linux
only, and there is no MySQL container here). This file replaces it for IIS.
Notation: `APP_ROOT` = the deploy folder, e.g. `C:\shopdb-flask`. The IIS site
physical path must be `APP_ROOT` (where `wsgi.py` lives).
## 0. Prerequisites on the box
- Python 3.14 (same minor as dev and CI). `py -3.14 --version` to confirm.
- IIS with the **HttpPlatformHandler** module:
https://www.iis.net/downloads/microsoft/httpplatformhandler
- **URL Rewrite** module (only for the optional real-client-IP rule).
- Network access to the MySQL 5.6 server.
- If the box is air-gapped, you cannot `pip install` live. On the dev box run
`pip download -r requirements.txt -d wheels\` (on a matching
Windows/Python target, or use `--platform` wheels), copy `wheels\` over, and
install with `pip install --no-index --find-links wheels\ ...`.
## 1. Copy the code
Copy the repo to `APP_ROOT`, INCLUDING `frontend/dist` (the built SPA the API
serves). Build it on dev first if stale:
```bash
# on the dev box
cd frontend && npm run build # produces frontend/dist
```
Ship `frontend/dist` with the code (Node is not needed on the prod box).
## 2. Python venv + dependencies
```powershell
cd C:\shopdb-flask
py -3.14 -m venv venv
venv\Scripts\python -m pip install --upgrade pip
venv\Scripts\pip install -r requirements.txt
```
The DB driver is `pymysql` (pure Python) so no C compiler / MySQL client libs
are needed. `waitress` is the WSGI server and ships in `requirements.txt`
(unlike gunicorn, which the Docker image installs separately).
## 3. Prepare MySQL 5.6 (the utf8mb4 gotcha)
MySQL 5.6 defaults cannot index utf8mb4 VARCHAR(255) columns (767-byte prefix
limit) and often defaults the server charset to latin1. The schema is utf8mb4,
so the server needs Barracuda + large-prefix, made durable in `my.ini` under
`[mysqld]`, then restart the MySQL service:
```ini
[mysqld]
innodb_file_per_table = 1
innodb_file_format = Barracuda
innodb_large_prefix = 1
```
Then create the database as utf8mb4 and a least-privilege app user:
```sql
CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'shopdb'@'%' IDENTIFIED BY 'CHANGE_ME';
GRANT ALL PRIVILEGES ON shopdb_flask.* TO 'shopdb'@'%';
FLUSH PRIVILEGES;
```
Without the `[mysqld]` flags, `flask db upgrade` fails with error 1071
("Specified key was too long"). The migration chain emits `ROW_FORMAT=DYNAMIC`
per table (see `migrations/env.py`), which fits the 3072-byte prefix those
flags unlock.
## 4. Configure secrets and connection (.env)
Create `APP_ROOT\.env` (loaded by `wsgi.py` via `load_dotenv()`). Keep secrets
here, not in `web.config`. Lock the file's ACLs to the IIS app-pool identity +
administrators.
```
FLASK_ENV=production
SECRET_KEY=<64+ random chars>
JWT_SECRET_KEY=<another 64+ random chars>
DATABASE_URL=mysql+pymysql://shopdb:CHANGE_ME@<mysql-host>:3306/shopdb_flask?charset=utf8mb4
CORS_ORIGINS=https://<the site's own hostname>
```
`ProductionConfig.validate()` refuses to boot if any of `SECRET_KEY`,
`JWT_SECRET_KEY`, `DATABASE_URL`, `CORS_ORIGINS` is missing or left at a dev
default. `CORS_ORIGINS` is the browser origin users hit (the IIS binding).
Generate a key: `venv\Scripts\python -c "import secrets;print(secrets.token_urlsafe(64))"`.
## 5. Initialize schema, data, plugins, admin
Run from `APP_ROOT` with the venv active and `.env` present:
```powershell
$env:FLASK_APP="shopdb"
venv\Scripts\flask db upgrade
venv\Scripts\flask seed reference-data
# Install the plugins this site tracks (registry lives in the gitignored
# instance/plugins.json, so a fresh box starts with none installed). Run
# `flask plugin list` to see the current bundled set; the the bundled plugins are
# computers, employees, geenforce, knowledgebase, machines, measuringtools,
# network, notifications, printedparts, printers, slides, usb, warranty. Install
# only the ones this site wants:
venv\Scripts\flask plugin list
venv\Scripts\flask plugin install machines
venv\Scripts\flask plugin install printers
venv\Scripts\flask plugin install computers
venv\Scripts\flask plugin install network
venv\Scripts\flask plugin install notifications
venv\Scripts\flask plugin install usb
venv\Scripts\flask plugin install knowledgebase
venv\Scripts\flask plugin install slides
venv\Scripts\flask plugin install employees
venv\Scripts\flask plugin upgrade-all
# First admin (password is generated and printed once):
venv\Scripts\flask seed admin --username admin --email admin@yourfacility.example.com
```
Cleaner than a hand list: declare the set once in a site profile and apply it:
```powershell
venv\Scripts\flask plugin apply-profile deploy\site-profile-universal.json # install + enable the chosen set, in dependency order
# deploy\site-profile.example.json is the annotated starting point to copy and trim.
venv\Scripts\flask plugin upgrade-all
venv\Scripts\flask plugin prune-schema --yes --force # FIRST PROVISIONING ONLY - see the warning below
```
(Alternatively copy the dev box's `instance/plugins.json` to `APP_ROOT\instance\`
to reproduce the exact set, then just run `flask plugin upgrade-all`.)
> **`prune-schema --force` is for first provisioning only.** It drops the tables
> of plugins this site did not install *even when they hold rows*. On a site that
> already has data, run `flask plugin prune-schema` with no flags first and read
> what it says it would drop. Re-running with `--force` after a feature has been
> used deletes that feature's records with no prompt and no backup.
## 6. Create the IIS site + web.config
This describes the own-site method (the app gets its own IIS site + port). To
mount the app at a subpath under an existing site instead (e.g.
`https://<host>/ops/` sharing the classic site's binding and cert), see
**docs/INSTALL-WINDOWS-IIS.md section 7b**: same web.config, but the site is a
`New-WebApplication` under the parent, `MOUNT_PATH=/ops` is set (web.config or
`.env`), and the frontend is built with `VITE_BASE_PATH=/ops/`.
1. In IIS Manager, add a new **Site** (separate from the classic ASP site):
- Physical path: `APP_ROOT`
- Binding: a free port or a dedicated hostname (e.g. `https` 443 with the
facility cert, or `http` on a test port like 8081 to start).
- App pool: No Managed Code, and an identity that can read `APP_ROOT`.
2. Copy `deploy\windows\web.config` to `APP_ROOT\web.config` and edit the paths
(`C:\shopdb-flask` -> your `APP_ROOT`). It launches
`waitress-serve --port=%HTTP_PLATFORM_PORT% wsgi:app` and sets
`FLASK_ENV=production` + `PYTHONPATH`.
3. Create `APP_ROOT\logs` for the HttpPlatform stdout log.
4. **Unlock the handler sections** (locked server-wide by default; without this
IIS returns **HTTP 500.19** "section cannot be used at this path"):
```powershell
%windir%\system32\inetsrv\appcmd unlock config /section:system.webServer/handlers
%windir%\system32\inetsrv\appcmd unlock config /section:system.webServer/httpPlatform
```
5. Grant the app-pool identity read/execute on `APP_ROOT` and modify on
`APP_ROOT\logs` (e.g. `icacls APP_ROOT /grant "IIS AppPool\<pool>:(OI)(CI)RX" /T`).
6. Recycle the app pool / restart the site.
TLS terminates at the IIS binding. The `X-Forwarded-For` URL Rewrite rule in the
web.config (real client IP for audit logs / kiosk visitor-location) is
**commented out by default** because it needs the URL Rewrite module - with it
active but URL Rewrite absent, IIS returns HTTP 500.19. Install URL Rewrite and
uncomment the `<rewrite>` block to enable it.
## 7. Smoke test
```powershell
# SPA loads:
curl.exe -k https://<host>/ # returns index.html
# API rejects an empty login with a validation error (health signal):
curl.exe -k -X POST https://<host>/api/auth/login -H "Content-Type: application/json" -d "{}"
# expect JSON containing VALIDATION_ERROR
```
Then log in through the browser as the admin from step 5 and confirm the
dashboard renders.
## 8. Redeploying as dev advances
Because this is a test instance you keep iterating on:
1. Pull/copy new code to `APP_ROOT` (rebuild `frontend/dist` on dev if the UI
changed).
2. `venv\Scripts\pip install -r requirements.txt` (if deps changed).
3. `venv\Scripts\flask db upgrade` (if new migrations).
4. Recycle the app pool.
## Troubleshooting
| Symptom | Cause / fix |
|---|---|
| Site 502 / process won't start | Check `APP_ROOT\logs\httpplatform*`. Usually a bad `processPath`, missing waitress, or `wsgi:app` not importable (set `PYTHONPATH`). |
| Boots but SQL echoes / debug on | `FLASK_ENV` not `production` (web.config env var or `.env`). |
| `flask db upgrade` error 1071 | MySQL 5.6 `[mysqld]` flags in step 3 not applied / server not restarted. |
| ConfigError on boot | A required var (SECRET_KEY / JWT_SECRET_KEY / DATABASE_URL / CORS_ORIGINS) missing or left at a dev default in `.env`. |
| Login works, CORS errors in browser | `CORS_ORIGINS` does not match the exact origin (scheme + host + port) the browser used. |
| Audit logs show 127.0.0.1 | Expected without the URL Rewrite X-Forwarded-For rule (step 6). |

17
docs/INSTALL-WINDOWS-IIS.md
View File

@@ -242,6 +242,23 @@ each gets its own site, app pool, port, and venv.
--- ---
## Redeploying a hand-built server
A hand-built server has no installer to run, so an update is done by hand in the
same order the installer would:
1. Copy the new code to the application root, rebuilding `frontend/dist` first
if the UI changed.
2. `venv\Scripts\pip install -r requirements.txt`, if dependencies changed.
3. `venv\Scripts\flask db upgrade` for the core chain, then
`venv\Scripts\flask plugin upgrade-all` for the plugin chains. Both, every
time - the second is the one people skip, and it surfaces days later as a
1054 "Unknown column".
4. Recycle the application pool.
Take a database backup before step 3. The installer does this automatically and
restores from it when a migration fails; by hand, it is yours to remember.
## Troubleshooting ## Troubleshooting
| Symptom | Cause / fix | | Symptom | Cause / fix |

2
docs/INSTALL-WINDOWS.md
View File

@@ -5,7 +5,7 @@ and get a working application. Nothing here needs an internet connection, and yo
do not need to know IIS, Python or MySQL. do not need to know IIS, Python or MySQL.
If you are looking after an existing hand-built server, see If you are looking after an existing hand-built server, see
[DEPLOY-WINDOWS-IIS.md](DEPLOY-WINDOWS-IIS.md) instead - that is the manual [INSTALL-WINDOWS-IIS.md](INSTALL-WINDOWS-IIS.md) instead - that is the manual
procedure, and the installer will not adopt a server it did not build. procedure, and the installer will not adopt a server it did not build.
--- ---

8
docs/PROJECT-MAP.md
View File

@@ -77,13 +77,13 @@ Manifest-less directories under `plugins/` are core frontend surface and always
| ADR-010-frontend-plugin-hooks.md | ADR-010: Frontend plugin hook contract | ACCEPTED | | ADR-010-frontend-plugin-hooks.md | ADR-010: Frontend plugin hook contract | ACCEPTED |
| ADR-011-machines-rename.md | ADR-011: Rename the equipment domain to machines; retype the models catalog with modeltypes | ACCEPTED | | ADR-011-machines-rename.md | ADR-011: Rename the equipment domain to machines; retype the models catalog with modeltypes | ACCEPTED |
| ADR-012-geenforce-manifest-ownership.md | ADR-012: GE-Enforce manifest ownership in shopdb | ACCEPTED | | ADR-012-geenforce-manifest-ownership.md | ADR-012: GE-Enforce manifest ownership in shopdb | ACCEPTED |
| ADR-013-plugin-catalog-and-lean-builds.md | ADR-013: Plugin Catalog, Curated Shelf, and Lean Per-Site Builds | PROPOSED | | ADR-013-plugin-catalog-and-lean-builds.md | ADR-013: Plugin Catalog, Curated Shelf, and Lean Per-Site Builds | ACCEPTED |
| ADR-014-schema-lean-per-site.md | ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables) | ACCEPTED | | ADR-014-schema-lean-per-site.md | ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables) | ACCEPTED |
| ADR-015-site-specific-configuration.md | ADR-015: Where a site's own data is allowed to live | ACCEPTED | | ADR-015-site-specific-configuration.md | ADR-015: Where a site's own data is allowed to live | ACCEPTED |
| ADR-016-credential-delivery.md | ADR-016: Credential delivery to the fleet | ACCEPTED | | ADR-016-credential-delivery.md | ADR-016: Credential delivery to the fleet | ACCEPTED (decided; NOT yet implemented - |
## Size ## Size
- test functions defined: **1036** (parametrised cases collect higher) - test functions defined: **1050** (parametrised cases collect higher)
- documented API paths: **245** (`docs/openapi.json`, regenerate with `scripts/gen_openapi.py`) - documented API paths: **265** (`docs/openapi.json`, regenerate with `scripts/gen_openapi.py`)

6
docs/START-HERE.md
View File

@@ -14,9 +14,9 @@ exists.
3. [CSV-IMPORT](CSV-IMPORT.md) if the site's data is in spreadsheets, or 3. [CSV-IMPORT](CSV-IMPORT.md) if the site's data is in spreadsheets, or
[IMPORT-API](IMPORT-API.md) if there is a source database to script against. [IMPORT-API](IMPORT-API.md) if there is a source database to script against.
**Do not** follow INSTALL-WINDOWS-IIS or DEPLOY-WINDOWS-IIS for a new site. **Do not** follow [INSTALL-WINDOWS-IIS](INSTALL-WINDOWS-IIS.md) for a new site.
Those are the manual procedure, kept for hand-built servers that predate the That is the manual procedure, kept for hand-built servers that predate the
installer, and following them produces a server the installer then refuses to installer, and following it produces a server the installer then refuses to
upgrade. upgrade.
## I am deploying the shop-floor tools ## I am deploying the shop-floor tools

2
docs/adr/ADR-003-plugin-distribution.md
View File

@@ -25,7 +25,7 @@ Three viable distribution models:
## Decision ## Decision
**PROPOSED:** Use a **hybrid model** with two clearly-labeled paths. Use a **hybrid model** with two clearly-labeled paths.
1. **Bundled plugins**: a small set of plugins ships with the framework, in-tree at `plugins/`. These are the reference implementations and the default install (printers, computers, network, equipment, usb, notifications). A site that wants only what's bundled needs no extra work. 1. **Bundled plugins**: a small set of plugins ships with the framework, in-tree at `plugins/`. These are the reference implementations and the default install (printers, computers, network, equipment, usb, notifications). A site that wants only what's bundled needs no extra work.
2. **External plugins**: sister sites or third parties build plugins in their own git repos. The site running the framework drops the plugin into `plugins/<name>/` (clone, submodule, or symlink) and runs `flask plugin install <name>`. No pip packaging required for v1. 2. **External plugins**: sister sites or third parties build plugins in their own git repos. The site running the framework drops the plugin into `plugins/<name>/` (clone, submodule, or symlink) and runs `flask plugin install <name>`. No pip packaging required for v1.

2
docs/adr/ADR-004-deployment-topology.md
View File

@@ -17,7 +17,7 @@ The codebase today is single-tenant per deployment. There is no `siteid` column,
## Decision ## Decision
**PROPOSED:** **Per-site instances.** Each adopting site runs its own dedicated stack. The framework does not support multi-tenancy. **Per-site instances.** Each adopting site runs its own dedicated stack. The framework does not support multi-tenancy.
Each site: Each site:

16
docs/adr/ADR-013-plugin-catalog-and-lean-builds.md
View File

@@ -1,6 +1,6 @@
# ADR-013: Plugin Catalog, Curated Shelf, and Lean Per-Site Builds # ADR-013: Plugin Catalog, Curated Shelf, and Lean Per-Site Builds
- Status: PROPOSED - Status: ACCEPTED
- Date: 2026-07-18 - Date: 2026-07-18
- Deciders: ShopDB maintainers - Deciders: ShopDB maintainers
- Relates to: ADR-002 (contract versioning), ADR-003 (plugin distribution), ADR-004 (per-site instances), ADR-008 (per-plugin migrations), ADR-009 (frontend plugin gating), ADR-010 (frontend hook contract) - Relates to: ADR-002 (contract versioning), ADR-003 (plugin distribution), ADR-004 (per-site instances), ADR-008 (per-plugin migrations), ADR-009 (frontend plugin gating), ADR-010 (frontend hook contract)
@@ -397,3 +397,17 @@ Deferred, each to its own future decision: schema-lean core-baseline re-org
(blocked on the installedapps -> machines FK question), pip/entry-point (blocked on the installedapps -> machines FK question), pip/entry-point
distribution (ADR-003 v2), hook-based search/report aggregation contract, and distribution (ADR-003 v2), hook-based search/report aggregation contract, and
any revisit of Path B. any revisit of Path B.
## Amendment, 2026-08-14
Status corrected from PROPOSED to ACCEPTED. The lean-build half of this decision
shipped some time ago - `scripts/build-site.sh`, `SITE_PLUGINS` staging,
`flask plugin prune-schema` and `default_enabled: false` are all live, and
ADR-014 was accepted on top of it - while the record still said the decision was
under consideration. A decision that has been implemented and depended upon is
not proposed, whatever the header says, and leaving it that way makes every
other status in the index worth less.
The catalog and signed-artifact parts of the decision remain unbuilt. That is
the ordinary state of an accepted decision: accepted means settled, not
delivered.

13
docs/adr/ADR-016-credential-delivery.md
View File

@@ -1,6 +1,6 @@
# ADR-016: Credential delivery to the fleet # ADR-016: Credential delivery to the fleet
- Status: ACCEPTED - Status: ACCEPTED (decided; NOT yet implemented - see Implementation status)
- Date: 2026-08-11 - Date: 2026-08-11
- Deciders: ShopDB maintainers - Deciders: ShopDB maintainers
- Relates to: ADR-006 (collector contract), ADR-012 (GE-Enforce manifest ownership), ADR-015 (site-specific configuration) - Relates to: ADR-006 (collector contract), ADR-012 (GE-Enforce manifest ownership), ADR-015 (site-specific configuration)
@@ -169,3 +169,14 @@ serves many machines, so the payload cannot be encrypted to its readers.
the secret off the share immediately and needs no new endpoint - but it leaves the secret off the share immediately and needs no new endpoint - but it leaves
provisioning per-bay by hand and offers no rotation. Recommended as tier one provisioning per-bay by hand and offers no rotation. Recommended as tier one
regardless, since the client helper is the same either way. regardless, since the client helper is the same either way.
## Implementation status, 2026-08-14
Nothing in this ADR is built yet. The fetch endpoint and the `credentials.*`
permissions it describes do not exist in the code, and a reader searching for
them will not find them.
Recorded here rather than by changing the status, because the decision itself
stands: this is how credential delivery WILL work, and a plugin author designing
against it is designing correctly. What credentials the fleet uses today, and
where they live, is in [FLEET-ARCHITECTURE](../FLEET-ARCHITECTURE.md).

8
docs/llms.txt
View File

@@ -24,10 +24,10 @@ collector API, and the EventSaver screensaver - should read
Neither tool is site-specific: the server URL, the API key and the targeting are Neither tool is site-specific: the server URL, the API key and the targeting are
inputs, not code. inputs, not code.
Do NOT walk someone through `docs/INSTALL-WINDOWS-IIS.md` or Do NOT walk someone through `docs/INSTALL-WINDOWS-IIS.md` for a new site. That
`docs/DEPLOY-WINDOWS-IIS.md` for a new site. Those are the MANUAL procedure, kept is the MANUAL procedure, kept only for hand-built servers that predate the
only for hand-built servers that predate the installer; following them produces a installer; following it produces a server the installer then refuses to
server the installer then refuses to upgrade. upgrade.
Day-2 operations all go through `shopdb-admin.ps1` in the install directory Day-2 operations all go through `shopdb-admin.ps1` in the install directory
(default `C:\shopdb-flask`): `status`, `restart`, `logs`, `check`, `verify`, (default `C:\shopdb-flask`): `status`, `restart`, `logs`, `check`, `verify`,

3
docs/proposals/dashboard-live-fleet.md
View File

@@ -1,6 +1,7 @@
# Proposal: a dashboard that shows the fleet, not the row count # Proposal: a dashboard that shows the fleet, not the row count
- Status: ACCEPTED - Status: IMPLEMENTED (a proposal, not a decision record - the
contract it introduced is ADR-010 and contract 0.19.0)
- Date: 2026-08-11 - Date: 2026-08-11
- Author: ShopDB maintainers - Author: ShopDB maintainers
- Relates to: ADR-010 (frontend plugin hooks), ADR-013 / ADR-014 (lean per-site builds), ADR-006 (collector contract), ADR-012 (GE-Enforce manifest ownership) - Relates to: ADR-010 (frontend plugin hooks), ADR-013 / ADR-014 (lean per-site builds), ADR-006 (collector contract), ADR-012 (GE-Enforce manifest ownership)