ADR-014 Phase 2: flask plugin prune-schema for lean per-site DBs
A lean site still gets every plugin's tables from the shared core Alembic baseline. prune-schema drops the tables of plugins not installed on this site, leaving core + chosen-plugin tables, with no edit to any released migration (the relocate-into-plugin-baselines alternative would mean rewriting ~15 released core migrations for a cosmetic gain - see ADR-014). - shopdb/plugins/cli.py: prune-schema command. Dry-run by default; --yes to execute; refuses non-empty tables without --force. Drops by table name (no plugin import) so it works on a lean image. MySQL: private AUTOCOMMIT engine (db.engine's pooled connections sit idle-in-transaction in a CLI context and would deadlock the DROP on a metadata lock). SQLite: db.engine, restoring the prior foreign_keys pragma so the StaticPool connection is not left changed. - tests/test_plugin_prune_schema.py: drop-only-not-installed, full no-op, refuse-non-empty, force-drops-non-empty. - docs/DEPLOY.md: lean provisioning step after upgrade-all. - ADR-014 ACCEPTED; index updated. Verified on MySQL: full install then prune = no-op (86 tables); lean install (machines+printers) then prune drops the other 19 plugin tables; second run no-op. Full suite 1077 passed.
This commit is contained in:
@@ -73,6 +73,24 @@ any plugin-specific migrations added after the ownership cutover. Both commands
|
||||
are idempotent, so re-running them is safe. See ADR-008 for why plugin schema
|
||||
splits into per-plugin chains from the cutover forward.
|
||||
|
||||
**Lean sites (ADR-014):** the core chain creates every bundled plugin's tables,
|
||||
so a site that ships only some plugins still has the others' (empty) tables. To
|
||||
carry only core + chosen-plugin tables, prune the rest once, at initial
|
||||
provisioning, after the two commands above:
|
||||
|
||||
```bash
|
||||
docker compose exec api flask plugin prune-schema # dry-run, review
|
||||
docker compose exec api flask plugin prune-schema --yes --force
|
||||
```
|
||||
|
||||
It drops the tables of every plugin not installed on this site. `--force` is
|
||||
needed because the core chain seeds a few plugin reference tables (default
|
||||
access protocols, etc.); at first provisioning those hold only seeded defaults,
|
||||
before any site data. It refuses to drop a table that holds rows without
|
||||
`--force`, so it is safe to leave out of routine upgrades - run it only when
|
||||
provisioning a lean site or after deliberately removing a plugin. Installing a
|
||||
pruned plugin later recreates its tables automatically.
|
||||
|
||||
**Charset:** the schema is utf8mb4 (`utf8mb4_unicode_ci`). The docker-compose `db` service sets `--character-set-server=utf8mb4`, so the auto-created `shopdb_flask` database is utf8mb4. If you point at an external MySQL instead of the bundled container, create the database as utf8mb4 first, or it inherits the server default (often latin1) and the schema silently drifts:
|
||||
|
||||
```sql
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables)
|
||||
|
||||
- Status: PROPOSED
|
||||
- Status: ACCEPTED
|
||||
- Date: 2026-07-19
|
||||
- Deciders: cproudlock
|
||||
- Relates to: ADR-008 (per-plugin migration ownership), ADR-013 (plugin catalog + lean per-site builds), ADR-001 (asset model)
|
||||
@@ -65,54 +65,78 @@ the pre-cutover core baseline. Correct and inert regardless of Phase 2 (no
|
||||
current caller creates against a populated schema). Verified against the
|
||||
plugin-migration suite.
|
||||
|
||||
### Phase 2 (load-bearing, dedicated pass): lift plugin tables into plugin baselines
|
||||
### Phase 2 (executed): prune not-installed plugin tables after upgrade
|
||||
|
||||
With no cross-plugin FKs remaining, each plugin's tables can be created
|
||||
independently. One coherent baseline edit:
|
||||
Two mechanisms were weighed to make a lean site's database carry only
|
||||
core + chosen-plugin tables:
|
||||
|
||||
- Remove the ~30 plugin-owned `create_table` blocks from the core baseline
|
||||
(68b3947ae14f), plus the four dead-object blocks (machinerelationships,
|
||||
printerdata, installedapps, communications.machineid) it creates only for
|
||||
later migrations to drop. Core baseline then creates only core tables.
|
||||
- Change each of the 14 plugin 0001 anchors from stamp-only `pass` to
|
||||
`create_plugin_tables(<name>)` / `drop_plugin_tables(<name>)` (idempotent, per
|
||||
above).
|
||||
- **Relocate** (rejected): pull every plugin-table create/alter out of the core
|
||||
chain into the plugin baselines, so the core chain never creates a
|
||||
not-installed plugin's table. Measurement killed this: plugin tables are
|
||||
created and altered across ~15 released core migrations (baseline plus 7c04,
|
||||
7d05, 7d08, 7d13, 7d15, 7d16, 7d17, ...), not just the baseline. Because the
|
||||
whole core chain runs before any plugin chain, removing a table's create from
|
||||
core while a later core migration still alters it breaks FULL installs too, so
|
||||
relocation means surgically rewriting ~15 released migrations - the highest
|
||||
blast radius in the project - for a purely cosmetic gain (the omitted tables
|
||||
are empty and the lean CODE build already never loads the plugin).
|
||||
|
||||
A fresh lean install then creates core tables plus only the chosen plugins'
|
||||
tables. A fresh full install creates the identical table set it does today.
|
||||
- **Prune-after-upgrade** (chosen): leave the entire core chain untouched. Add
|
||||
`flask plugin prune-schema`, which drops the tables of every plugin in
|
||||
PLUGIN_TABLE_OWNERS that is not installed on this site. Run once at deploy,
|
||||
after `flask db upgrade` and `flask plugin upgrade-all`. Same end state
|
||||
(core + chosen tables) with near-zero blast radius: no released migration is
|
||||
edited, and an existing full site is unaffected because it never runs the
|
||||
command.
|
||||
|
||||
Existing-database safety: an existing database is stamped past the baseline and
|
||||
past each plugin's old stamp-anchor, so neither re-runs; it keeps its tables.
|
||||
Editing the baseline's content only changes what a FRESH install creates. This
|
||||
is the highest-blast-radius edit in the project (the released baseline every
|
||||
site's DB derives from), so it is staged as its own pass gated on the full
|
||||
verification matrix: fresh-full (== current schema), fresh-lean (strict subset),
|
||||
existing-DB (no re-run, unchanged), and the migrations-mysql CI (fresh upgrade
|
||||
from empty + per-plugin install + second-run no-op).
|
||||
`prune-schema` drops by table name (no plugin-code import), so it works on a
|
||||
lean image where the omitted plugin's directory is absent. It is a dry-run by
|
||||
default and refuses to drop a table that holds rows unless `--force`, so a
|
||||
misfire on a populated site cannot silently delete data. Because the core chain
|
||||
seeds a few plugin reference tables (e.g. 7d05 inserts default access
|
||||
protocols), initial lean provisioning uses `--force` - at that point the tables
|
||||
hold only migration-seeded defaults, before any site data exists.
|
||||
|
||||
The idempotent `create_plugin_tables` (enabling change above) is what lets a
|
||||
lean site later ADD an omitted plugin: its anchor recreates the pruned tables.
|
||||
|
||||
Verified end to end on MySQL: fresh full install (86 tables) then prune is a
|
||||
no-op; fresh lean install (machines + printers) then prune drops the other 19
|
||||
plugin tables, leaving core + chosen; second prune is a no-op; the non-empty
|
||||
guard refuses without `--force`. Four SQLite regression tests pin the behavior
|
||||
(tests/test_plugin_prune_schema.py), running in the backend CI job via the real
|
||||
CLI runner: drop-only-not-installed, full-site no-op, refuse-non-empty, and
|
||||
force-drops-non-empty.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Removes every cross-plugin foreign key; plugin schemas become independent, as
|
||||
ADR-013 requires.
|
||||
- Deletes dead legacy tables/columns every database has carried since the
|
||||
cutover (real cleanup, not just lean).
|
||||
- After Phase 2, a lean site's database contains only core + chosen-plugin
|
||||
tables.
|
||||
- A lean site's database contains only core + chosen-plugin tables, with no edit
|
||||
to any released migration (near-zero blast radius).
|
||||
- The cross-plugin FK blocker ADR-013 cited is gone (dead cruft, dropped by
|
||||
existing migrations), so plugin schemas are already FK-independent.
|
||||
- Adding an omitted plugin to a lean site later just works: the idempotent
|
||||
anchor recreates its tables.
|
||||
|
||||
### Negative / risk
|
||||
|
||||
- Phase 2 edits the released baseline's content. It is safe because existing
|
||||
databases never re-run a stamped revision, but it demands the full fresh +
|
||||
existing + CI verification and is therefore staged separately.
|
||||
- Dropping tables is destructive; the migration downgrade recreates them empty
|
||||
(structure only) - acceptable because they hold no live data.
|
||||
- prune-schema is destructive by nature; the row-count guard + dry-run default +
|
||||
required `--force` for non-empty tables contain that. It is a deploy-time
|
||||
provisioning step, not something to run casually on a live populated site.
|
||||
- A lean fresh install still transiently creates then drops the omitted plugins'
|
||||
tables (the core chain builds them, prune removes them). Harmless and one-time
|
||||
at provisioning; the trade for not touching the released baseline.
|
||||
|
||||
## Implementation
|
||||
|
||||
- Phase 1: core migration `7d27_retire_legacy_machine_fk_tables` + drop the dead
|
||||
`communications.machineid` column from the model. Verified: fresh upgrade,
|
||||
idempotent re-run, and a scratch database that had the tables drops them.
|
||||
- Phase 2: baseline edit + 14 plugin anchor rewrites + idempotent-create guards,
|
||||
gated by the migrations-mysql CI, in a dedicated pass.
|
||||
- Phase 1: nothing to do - the dead cross-boundary FK objects were already
|
||||
dropped by existing migrations `7a01_adr001_position_contract` and
|
||||
`7c01_drop_legacy_machine`; verified absent on a fresh full MySQL upgrade.
|
||||
- Enabling change: `create_plugin_tables` made idempotent
|
||||
(`shopdb/plugins/alembic_template.py`).
|
||||
- Phase 2: `flask plugin prune-schema` (`shopdb/plugins/cli.py`), dry-run by
|
||||
default, `--yes` to execute, `--force` for non-empty tables. Deploy order:
|
||||
`flask db upgrade` -> `flask plugin upgrade-all` -> `flask plugin prune-schema
|
||||
--yes --force`. Regression tests in `tests/test_plugin_prune_schema.py` (run in
|
||||
the backend CI job).
|
||||
|
||||
@@ -26,7 +26,7 @@ Each ADR captures a single architectural decision: the context, the decision its
|
||||
| [011](ADR-011-machines-rename.md) | Machines rename + modeltypes retyping | ACCEPTED |
|
||||
| [012](ADR-012-geenforce-manifest-ownership.md) | GE-Enforce manifest ownership in shopdb | ACCEPTED |
|
||||
| [013](ADR-013-plugin-catalog-and-lean-builds.md) | Plugin catalog, curated shelf, and lean per-site builds | PROPOSED |
|
||||
| [014](ADR-014-schema-lean-per-site.md) | Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables) | PROPOSED |
|
||||
| [014](ADR-014-schema-lean-per-site.md) | Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) | ACCEPTED |
|
||||
|
||||
## Authoring
|
||||
|
||||
|
||||
Reference in New Issue
Block a user