ADR-014 Phase 2: flask plugin prune-schema for lean per-site DBs

A lean site still gets every plugin's tables from the shared core Alembic
baseline. prune-schema drops the tables of plugins not installed on this
site, leaving core + chosen-plugin tables, with no edit to any released
migration (the relocate-into-plugin-baselines alternative would mean
rewriting ~15 released core migrations for a cosmetic gain - see ADR-014).

- shopdb/plugins/cli.py: prune-schema command. Dry-run by default; --yes to
  execute; refuses non-empty tables without --force. Drops by table name (no
  plugin import) so it works on a lean image. MySQL: private AUTOCOMMIT engine
  (db.engine's pooled connections sit idle-in-transaction in a CLI context and
  would deadlock the DROP on a metadata lock). SQLite: db.engine, restoring the
  prior foreign_keys pragma so the StaticPool connection is not left changed.
- tests/test_plugin_prune_schema.py: drop-only-not-installed, full no-op,
  refuse-non-empty, force-drops-non-empty.
- docs/DEPLOY.md: lean provisioning step after upgrade-all.
- ADR-014 ACCEPTED; index updated.

Verified on MySQL: full install then prune = no-op (86 tables); lean install
(machines+printers) then prune drops the other 19 plugin tables; second run
no-op. Full suite 1077 passed.
This commit is contained in:
cproudlock
2026-07-19 11:39:46 -04:00
parent 42ca8d75c3
commit c386e211df
6 changed files with 307 additions and 39 deletions

View File

@@ -1,6 +1,6 @@
# ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables)
- Status: PROPOSED
- Status: ACCEPTED
- Date: 2026-07-19
- Deciders: cproudlock
- Relates to: ADR-008 (per-plugin migration ownership), ADR-013 (plugin catalog + lean per-site builds), ADR-001 (asset model)
@@ -65,54 +65,78 @@ the pre-cutover core baseline. Correct and inert regardless of Phase 2 (no
current caller creates against a populated schema). Verified against the
plugin-migration suite.
### Phase 2 (load-bearing, dedicated pass): lift plugin tables into plugin baselines
### Phase 2 (executed): prune not-installed plugin tables after upgrade
With no cross-plugin FKs remaining, each plugin's tables can be created
independently. One coherent baseline edit:
Two mechanisms were weighed to make a lean site's database carry only
core + chosen-plugin tables:
- Remove the ~30 plugin-owned `create_table` blocks from the core baseline
(68b3947ae14f), plus the four dead-object blocks (machinerelationships,
printerdata, installedapps, communications.machineid) it creates only for
later migrations to drop. Core baseline then creates only core tables.
- Change each of the 14 plugin 0001 anchors from stamp-only `pass` to
`create_plugin_tables(<name>)` / `drop_plugin_tables(<name>)` (idempotent, per
above).
- **Relocate** (rejected): pull every plugin-table create/alter out of the core
chain into the plugin baselines, so the core chain never creates a
not-installed plugin's table. Measurement killed this: plugin tables are
created and altered across ~15 released core migrations (baseline plus 7c04,
7d05, 7d08, 7d13, 7d15, 7d16, 7d17, ...), not just the baseline. Because the
whole core chain runs before any plugin chain, removing a table's create from
core while a later core migration still alters it breaks FULL installs too, so
relocation means surgically rewriting ~15 released migrations - the highest
blast radius in the project - for a purely cosmetic gain (the omitted tables
are empty and the lean CODE build already never loads the plugin).
A fresh lean install then creates core tables plus only the chosen plugins'
tables. A fresh full install creates the identical table set it does today.
- **Prune-after-upgrade** (chosen): leave the entire core chain untouched. Add
`flask plugin prune-schema`, which drops the tables of every plugin in
PLUGIN_TABLE_OWNERS that is not installed on this site. Run once at deploy,
after `flask db upgrade` and `flask plugin upgrade-all`. Same end state
(core + chosen tables) with near-zero blast radius: no released migration is
edited, and an existing full site is unaffected because it never runs the
command.
Existing-database safety: an existing database is stamped past the baseline and
past each plugin's old stamp-anchor, so neither re-runs; it keeps its tables.
Editing the baseline's content only changes what a FRESH install creates. This
is the highest-blast-radius edit in the project (the released baseline every
site's DB derives from), so it is staged as its own pass gated on the full
verification matrix: fresh-full (== current schema), fresh-lean (strict subset),
existing-DB (no re-run, unchanged), and the migrations-mysql CI (fresh upgrade
from empty + per-plugin install + second-run no-op).
`prune-schema` drops by table name (no plugin-code import), so it works on a
lean image where the omitted plugin's directory is absent. It is a dry-run by
default and refuses to drop a table that holds rows unless `--force`, so a
misfire on a populated site cannot silently delete data. Because the core chain
seeds a few plugin reference tables (e.g. 7d05 inserts default access
protocols), initial lean provisioning uses `--force` - at that point the tables
hold only migration-seeded defaults, before any site data exists.
The idempotent `create_plugin_tables` (enabling change above) is what lets a
lean site later ADD an omitted plugin: its anchor recreates the pruned tables.
Verified end to end on MySQL: fresh full install (86 tables) then prune is a
no-op; fresh lean install (machines + printers) then prune drops the other 19
plugin tables, leaving core + chosen; second prune is a no-op; the non-empty
guard refuses without `--force`. Four SQLite regression tests pin the behavior
(tests/test_plugin_prune_schema.py), running in the backend CI job via the real
CLI runner: drop-only-not-installed, full-site no-op, refuse-non-empty, and
force-drops-non-empty.
## Consequences
### Positive
- Removes every cross-plugin foreign key; plugin schemas become independent, as
ADR-013 requires.
- Deletes dead legacy tables/columns every database has carried since the
cutover (real cleanup, not just lean).
- After Phase 2, a lean site's database contains only core + chosen-plugin
tables.
- A lean site's database contains only core + chosen-plugin tables, with no edit
to any released migration (near-zero blast radius).
- The cross-plugin FK blocker ADR-013 cited is gone (dead cruft, dropped by
existing migrations), so plugin schemas are already FK-independent.
- Adding an omitted plugin to a lean site later just works: the idempotent
anchor recreates its tables.
### Negative / risk
- Phase 2 edits the released baseline's content. It is safe because existing
databases never re-run a stamped revision, but it demands the full fresh +
existing + CI verification and is therefore staged separately.
- Dropping tables is destructive; the migration downgrade recreates them empty
(structure only) - acceptable because they hold no live data.
- prune-schema is destructive by nature; the row-count guard + dry-run default +
required `--force` for non-empty tables contain that. It is a deploy-time
provisioning step, not something to run casually on a live populated site.
- A lean fresh install still transiently creates then drops the omitted plugins'
tables (the core chain builds them, prune removes them). Harmless and one-time
at provisioning; the trade for not touching the released baseline.
## Implementation
- Phase 1: core migration `7d27_retire_legacy_machine_fk_tables` + drop the dead
`communications.machineid` column from the model. Verified: fresh upgrade,
idempotent re-run, and a scratch database that had the tables drops them.
- Phase 2: baseline edit + 14 plugin anchor rewrites + idempotent-create guards,
gated by the migrations-mysql CI, in a dedicated pass.
- Phase 1: nothing to do - the dead cross-boundary FK objects were already
dropped by existing migrations `7a01_adr001_position_contract` and
`7c01_drop_legacy_machine`; verified absent on a fresh full MySQL upgrade.
- Enabling change: `create_plugin_tables` made idempotent
(`shopdb/plugins/alembic_template.py`).
- Phase 2: `flask plugin prune-schema` (`shopdb/plugins/cli.py`), dry-run by
default, `--yes` to execute, `--force` for non-empty tables. Deploy order:
`flask db upgrade` -> `flask plugin upgrade-all` -> `flask plugin prune-schema
--yes --force`. Regression tests in `tests/test_plugin_prune_schema.py` (run in
the backend CI job).

View File

@@ -26,7 +26,7 @@ Each ADR captures a single architectural decision: the context, the decision its
| [011](ADR-011-machines-rename.md) | Machines rename + modeltypes retyping | ACCEPTED |
| [012](ADR-012-geenforce-manifest-ownership.md) | GE-Enforce manifest ownership in shopdb | ACCEPTED |
| [013](ADR-013-plugin-catalog-and-lean-builds.md) | Plugin catalog, curated shelf, and lean per-site builds | PROPOSED |
| [014](ADR-014-schema-lean-per-site.md) | Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables) | PROPOSED |
| [014](ADR-014-schema-lean-per-site.md) | Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) | ACCEPTED |
## Authoring