ADR-014 Phase 2: flask plugin prune-schema for lean per-site DBs
A lean site still gets every plugin's tables from the shared core Alembic baseline. prune-schema drops the tables of plugins not installed on this site, leaving core + chosen-plugin tables, with no edit to any released migration (the relocate-into-plugin-baselines alternative would mean rewriting ~15 released core migrations for a cosmetic gain - see ADR-014). - shopdb/plugins/cli.py: prune-schema command. Dry-run by default; --yes to execute; refuses non-empty tables without --force. Drops by table name (no plugin import) so it works on a lean image. MySQL: private AUTOCOMMIT engine (db.engine's pooled connections sit idle-in-transaction in a CLI context and would deadlock the DROP on a metadata lock). SQLite: db.engine, restoring the prior foreign_keys pragma so the StaticPool connection is not left changed. - tests/test_plugin_prune_schema.py: drop-only-not-installed, full no-op, refuse-non-empty, force-drops-non-empty. - docs/DEPLOY.md: lean provisioning step after upgrade-all. - ADR-014 ACCEPTED; index updated. Verified on MySQL: full install then prune = no-op (86 tables); lean install (machines+printers) then prune drops the other 19 plugin tables; second run no-op. Full suite 1077 passed.
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables)
|
||||
|
||||
- Status: PROPOSED
|
||||
- Status: ACCEPTED
|
||||
- Date: 2026-07-19
|
||||
- Deciders: cproudlock
|
||||
- Relates to: ADR-008 (per-plugin migration ownership), ADR-013 (plugin catalog + lean per-site builds), ADR-001 (asset model)
|
||||
@@ -65,54 +65,78 @@ the pre-cutover core baseline. Correct and inert regardless of Phase 2 (no
|
||||
current caller creates against a populated schema). Verified against the
|
||||
plugin-migration suite.
|
||||
|
||||
### Phase 2 (load-bearing, dedicated pass): lift plugin tables into plugin baselines
|
||||
### Phase 2 (executed): prune not-installed plugin tables after upgrade
|
||||
|
||||
With no cross-plugin FKs remaining, each plugin's tables can be created
|
||||
independently. One coherent baseline edit:
|
||||
Two mechanisms were weighed to make a lean site's database carry only
|
||||
core + chosen-plugin tables:
|
||||
|
||||
- Remove the ~30 plugin-owned `create_table` blocks from the core baseline
|
||||
(68b3947ae14f), plus the four dead-object blocks (machinerelationships,
|
||||
printerdata, installedapps, communications.machineid) it creates only for
|
||||
later migrations to drop. Core baseline then creates only core tables.
|
||||
- Change each of the 14 plugin 0001 anchors from stamp-only `pass` to
|
||||
`create_plugin_tables(<name>)` / `drop_plugin_tables(<name>)` (idempotent, per
|
||||
above).
|
||||
- **Relocate** (rejected): pull every plugin-table create/alter out of the core
|
||||
chain into the plugin baselines, so the core chain never creates a
|
||||
not-installed plugin's table. Measurement killed this: plugin tables are
|
||||
created and altered across ~15 released core migrations (baseline plus 7c04,
|
||||
7d05, 7d08, 7d13, 7d15, 7d16, 7d17, ...), not just the baseline. Because the
|
||||
whole core chain runs before any plugin chain, removing a table's create from
|
||||
core while a later core migration still alters it breaks FULL installs too, so
|
||||
relocation means surgically rewriting ~15 released migrations - the highest
|
||||
blast radius in the project - for a purely cosmetic gain (the omitted tables
|
||||
are empty and the lean CODE build already never loads the plugin).
|
||||
|
||||
A fresh lean install then creates core tables plus only the chosen plugins'
|
||||
tables. A fresh full install creates the identical table set it does today.
|
||||
- **Prune-after-upgrade** (chosen): leave the entire core chain untouched. Add
|
||||
`flask plugin prune-schema`, which drops the tables of every plugin in
|
||||
PLUGIN_TABLE_OWNERS that is not installed on this site. Run once at deploy,
|
||||
after `flask db upgrade` and `flask plugin upgrade-all`. Same end state
|
||||
(core + chosen tables) with near-zero blast radius: no released migration is
|
||||
edited, and an existing full site is unaffected because it never runs the
|
||||
command.
|
||||
|
||||
Existing-database safety: an existing database is stamped past the baseline and
|
||||
past each plugin's old stamp-anchor, so neither re-runs; it keeps its tables.
|
||||
Editing the baseline's content only changes what a FRESH install creates. This
|
||||
is the highest-blast-radius edit in the project (the released baseline every
|
||||
site's DB derives from), so it is staged as its own pass gated on the full
|
||||
verification matrix: fresh-full (== current schema), fresh-lean (strict subset),
|
||||
existing-DB (no re-run, unchanged), and the migrations-mysql CI (fresh upgrade
|
||||
from empty + per-plugin install + second-run no-op).
|
||||
`prune-schema` drops by table name (no plugin-code import), so it works on a
|
||||
lean image where the omitted plugin's directory is absent. It is a dry-run by
|
||||
default and refuses to drop a table that holds rows unless `--force`, so a
|
||||
misfire on a populated site cannot silently delete data. Because the core chain
|
||||
seeds a few plugin reference tables (e.g. 7d05 inserts default access
|
||||
protocols), initial lean provisioning uses `--force` - at that point the tables
|
||||
hold only migration-seeded defaults, before any site data exists.
|
||||
|
||||
The idempotent `create_plugin_tables` (enabling change above) is what lets a
|
||||
lean site later ADD an omitted plugin: its anchor recreates the pruned tables.
|
||||
|
||||
Verified end to end on MySQL: fresh full install (86 tables) then prune is a
|
||||
no-op; fresh lean install (machines + printers) then prune drops the other 19
|
||||
plugin tables, leaving core + chosen; second prune is a no-op; the non-empty
|
||||
guard refuses without `--force`. Four SQLite regression tests pin the behavior
|
||||
(tests/test_plugin_prune_schema.py), running in the backend CI job via the real
|
||||
CLI runner: drop-only-not-installed, full-site no-op, refuse-non-empty, and
|
||||
force-drops-non-empty.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Removes every cross-plugin foreign key; plugin schemas become independent, as
|
||||
ADR-013 requires.
|
||||
- Deletes dead legacy tables/columns every database has carried since the
|
||||
cutover (real cleanup, not just lean).
|
||||
- After Phase 2, a lean site's database contains only core + chosen-plugin
|
||||
tables.
|
||||
- A lean site's database contains only core + chosen-plugin tables, with no edit
|
||||
to any released migration (near-zero blast radius).
|
||||
- The cross-plugin FK blocker ADR-013 cited is gone (dead cruft, dropped by
|
||||
existing migrations), so plugin schemas are already FK-independent.
|
||||
- Adding an omitted plugin to a lean site later just works: the idempotent
|
||||
anchor recreates its tables.
|
||||
|
||||
### Negative / risk
|
||||
|
||||
- Phase 2 edits the released baseline's content. It is safe because existing
|
||||
databases never re-run a stamped revision, but it demands the full fresh +
|
||||
existing + CI verification and is therefore staged separately.
|
||||
- Dropping tables is destructive; the migration downgrade recreates them empty
|
||||
(structure only) - acceptable because they hold no live data.
|
||||
- prune-schema is destructive by nature; the row-count guard + dry-run default +
|
||||
required `--force` for non-empty tables contain that. It is a deploy-time
|
||||
provisioning step, not something to run casually on a live populated site.
|
||||
- A lean fresh install still transiently creates then drops the omitted plugins'
|
||||
tables (the core chain builds them, prune removes them). Harmless and one-time
|
||||
at provisioning; the trade for not touching the released baseline.
|
||||
|
||||
## Implementation
|
||||
|
||||
- Phase 1: core migration `7d27_retire_legacy_machine_fk_tables` + drop the dead
|
||||
`communications.machineid` column from the model. Verified: fresh upgrade,
|
||||
idempotent re-run, and a scratch database that had the tables drops them.
|
||||
- Phase 2: baseline edit + 14 plugin anchor rewrites + idempotent-create guards,
|
||||
gated by the migrations-mysql CI, in a dedicated pass.
|
||||
- Phase 1: nothing to do - the dead cross-boundary FK objects were already
|
||||
dropped by existing migrations `7a01_adr001_position_contract` and
|
||||
`7c01_drop_legacy_machine`; verified absent on a fresh full MySQL upgrade.
|
||||
- Enabling change: `create_plugin_tables` made idempotent
|
||||
(`shopdb/plugins/alembic_template.py`).
|
||||
- Phase 2: `flask plugin prune-schema` (`shopdb/plugins/cli.py`), dry-run by
|
||||
default, `--yes` to execute, `--force` for non-empty tables. Deploy order:
|
||||
`flask db upgrade` -> `flask plugin upgrade-all` -> `flask plugin prune-schema
|
||||
--yes --force`. Regression tests in `tests/test_plugin_prune_schema.py` (run in
|
||||
the backend CI job).
|
||||
|
||||
@@ -26,7 +26,7 @@ Each ADR captures a single architectural decision: the context, the decision its
|
||||
| [011](ADR-011-machines-rename.md) | Machines rename + modeltypes retyping | ACCEPTED |
|
||||
| [012](ADR-012-geenforce-manifest-ownership.md) | GE-Enforce manifest ownership in shopdb | ACCEPTED |
|
||||
| [013](ADR-013-plugin-catalog-and-lean-builds.md) | Plugin catalog, curated shelf, and lean per-site builds | PROPOSED |
|
||||
| [014](ADR-014-schema-lean-per-site.md) | Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables) | PROPOSED |
|
||||
| [014](ADR-014-schema-lean-per-site.md) | Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) | ACCEPTED |
|
||||
|
||||
## Authoring
|
||||
|
||||
|
||||
Reference in New Issue
Block a user