Files

cproudlock 534f179ee6 Make main the lean product a site receives

main was 16 days and 259 commits behind the line the product is actually built
on, at contract 0.30.0 against 0.35.0, with no per-PC credential code at all. It
was neither current nor curated - just stale. This brings it forward and, at the
same time, makes it mean something: main is what a receiving site installs, and
nothing else.

WHAT IT CARRIES. Twelve plugins - backups, computers, endpoints, guides,
knowledgebase, machines, network, notifications, printers, slides, tools,
warranty - plus applications, which is manifest-less core and always ships. The
only manifest dependency inside that set is endpoints on computers, and both are
in it, so the closure is self-contained.

WHAT IT DOES NOT. employees, geenforce, measuringtools, printedparts and usb
stay on the site line. So do the Windows installer tree, the VM rehearsal
helpers, the agent tooling config, one site's data loader, the internal audit
and findings records, and the planner spreadsheet. 1,135 tracked files against
1,438.

A REMOVAL THAT BREAKS THE APP IS NOT A REMOVAL, and two nearly shipped:

deploy/site-profile-universal.json still named all seventeen plugins. It is the
file docs/DEPLOY.md tells a site to provision from, so a fresh install would
have failed at `flask plugin apply-profile` with PluginError - the first thing a
receiving site does, broken by the branch meant to serve them. It now names the
twelve that exist, and its comment says the two must agree.

agent/ was never refreshed. Whoever populated this branch copied shopdb/,
plugins/, frontend/ and tests/ from the site line and left the agent tree at its
September state: 22 C# and PowerShell sources missing, including AgentCredentials,
EnrollmentService, AgentHttpFactory and the fleet-alert client, plus one file the
site line had deleted still present. The old tree is internally consistent, so it
compiles, and nothing in CI builds it - every gate was green over a server that
speaks enrollment and per-PC credentials to a bundled agent source that cannot.
Resynced whole.

DECISIONS THIS ENCODES, rather than defers:

- The plugin-authoring lab builds printedparts, whose manifest depends on
  employees, so it could not stay. The lab remains on the site line and main
  ships no build-along tutorial. That is a real loss for an adopting developer
  and is recorded here rather than discovered later.
- Recognition, Training and Recertification notifications are no longer seeded.
  Each splits one notification into a card per employee and renders a photo,
  which needs a plugin this branch does not bundle. Seeding them would offer an
  author a type that cannot resolve anybody. A site that installs employees gets
  them from that plugin's own seed.
- Windows/IIS is documented as the legacy path throughout, and the README leads
  with Docker, because that is what a receiving site gets.

The API description is regenerated for THIS branch - 347 paths and 483
operations, down from 415 and 573 - so the published contract describes what is
here rather than what the site line serves. docs/CONFIG.md lost the two
measuring-tool label keys the code no longer declares, which two lanes had
decided opposite ways.

Suite: 3124 passed, 6 skipped. Naming and style gates pass. The docs gates run
rather than skip, and every index line count is derived from the file it names.

2026-09-19 11:24:40 -04:00

7.5 KiB

Raw Blame History

ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables)

Status: ACCEPTED
Date: 2026-07-19
Deciders: ShopDB maintainers
Relates to: ADR-008 (per-plugin migration ownership), ADR-013 (plugin catalog + lean per-site builds), ADR-001 (asset model)

Context

ADR-013 delivered lean per-site builds for plugin CODE (backend tree + frontend bundle carry only chosen plugins). One residual was left, explicitly deferred: the DATABASE. The core Alembic baseline (68b3947ae14f) creates EVERY table, including ~30 plugin-owned tables (PLUGIN_TABLE_OWNERS). Each plugin's own baseline is a stamp-only no-op (the core chain already made its tables, per ADR-008). So a lean site that omits a plugin still creates that plugin's tables, empty and unused.

The deferral cited a blocker: "the computers-owned installedapps table FKs machines.machineid while computers declares no dependency on machines; reversing the cutover would introduce undeclared hard deps or drop FKs; neither is decided." Investigation refined this:

The cross-boundary foreign keys into the machines plugin table are ALL held by DEAD legacy tables/columns that predate the asset model (ADR-001) and the per-plugin cutover (ADR-008), and are queried nowhere in the codebase:

machinerelationships (child/parentmachineid -> machines) - superseded by assetrelationships. No model, no query.
printerdata (machineid -> machines) - the pre-cutover printers table, superseded by the printers plugin. No model, no query.
installedapps (machineid -> machines) - a standalone machine-app link table; the live relationship is computerinstalledapps (FK to computers only). The standalone table has no model, no query.
communications.machineid (-> machines) - a legacy column on the core communications table (which is now assetid-based). Not read anywhere.

No LIVE plugin table hard-FKs another plugin's table. computerinstalledapps FKs only computers.computerid (intra-plugin). So the blocker is dead cruft, not live design.

Decision

Two phases, both leaving existing databases correct.

Phase 1: retire the dead cross-boundary cruft - ALREADY DONE

Investigation found this is already accomplished by existing migrations: 7a01_adr001_position_contract and 7c01_drop_legacy_machine drop machinerelationships, printerdata, installedapps, and communications.machineid (with its FK). The current schema (verified on the dev database) has none of them. So the cross-plugin FK blocker ADR-013 cited no longer exists in the live schema - only in the baseline's transient create-then-later-drop. No new migration is needed for Phase 1.

Precedent: ADR-001 dropped a cross-plugin FK the same way (usbcheckouts.machineid -> machines became a soft sentinel).

Enabling change (executed now): idempotent create_plugin_tables

shopdb/plugins/alembic_template.py:create_plugin_tables now skips any table that already exists (inspects the bind first) instead of raising. This is the mechanism Phase 2 needs: a plugin anchor can create its tables on a fresh lean install AND be a safe no-op on an existing database that already has them from the pre-cutover core baseline. Correct and inert regardless of Phase 2 (no current caller creates against a populated schema). Verified against the plugin-migration suite.

Phase 2 (executed): prune not-installed plugin tables after upgrade

Two mechanisms were weighed to make a lean site's database carry only core + chosen-plugin tables:

Relocate (rejected): pull every plugin-table create/alter out of the core chain into the plugin baselines, so the core chain never creates a not-installed plugin's table. Measurement killed this: plugin tables are created and altered across ~15 released core migrations (baseline plus 7c04, 7d05, 7d08, 7d13, 7d15, 7d16, 7d17, ...), not just the baseline. Because the whole core chain runs before any plugin chain, removing a table's create from core while a later core migration still alters it breaks FULL installs too, so relocation means surgically rewriting ~15 released migrations - the highest blast radius in the project - for a purely cosmetic gain (the omitted tables are empty and the lean CODE build already never loads the plugin).
Prune-after-upgrade (chosen): leave the entire core chain untouched. Add flask plugin prune-schema, which drops the tables of every plugin in PLUGIN_TABLE_OWNERS that is not installed on this site. Run once at deploy, after flask db upgrade and flask plugin upgrade-all. Same end state (core + chosen tables) with near-zero blast radius: no released migration is edited, and an existing full site is unaffected because it never runs the command.

prune-schema drops by table name (no plugin-code import), so it works on a lean image where the omitted plugin's directory is absent. It is a dry-run by default and refuses to drop a table that holds rows unless --force, so a misfire on a populated site cannot silently delete data. Because the core chain seeds a few plugin reference tables (e.g. 7d05 inserts default access protocols), initial lean provisioning uses --force - at that point the tables hold only migration-seeded defaults, before any site data exists.

The idempotent create_plugin_tables (enabling change above) is what lets a lean site later ADD an omitted plugin: its anchor recreates the pruned tables.

Verified end to end on MySQL: fresh full install (86 tables) then prune is a no-op; fresh lean install (machines + printers) then prune drops the other 19 plugin tables, leaving core + chosen; second prune is a no-op; the non-empty guard refuses without --force. Four SQLite regression tests pin the behavior (tests/test_plugin_prune_schema.py), running in the backend CI job via the real CLI runner: drop-only-not-installed, full-site no-op, refuse-non-empty, and force-drops-non-empty.

Consequences

Positive

A lean site's database contains only core + chosen-plugin tables, with no edit to any released migration (near-zero blast radius).
The cross-plugin FK blocker ADR-013 cited is gone (dead cruft, dropped by existing migrations), so plugin schemas are already FK-independent.
Adding an omitted plugin to a lean site later just works: the idempotent anchor recreates its tables.

Negative / risk

prune-schema is destructive by nature; the row-count guard + dry-run default + required --force for non-empty tables contain that. It is a deploy-time provisioning step, not something to run casually on a live populated site.
A lean fresh install still transiently creates then drops the omitted plugins' tables (the core chain builds them, prune removes them). Harmless and one-time at provisioning; the trade for not touching the released baseline.

Implementation

Phase 1: nothing to do - the dead cross-boundary FK objects were already dropped by existing migrations 7a01_adr001_position_contract and 7c01_drop_legacy_machine; verified absent on a fresh full MySQL upgrade.
Enabling change: create_plugin_tables made idempotent (shopdb/plugins/alembic_template.py).
Phase 2: flask plugin prune-schema (shopdb/plugins/cli.py), dry-run by default, --yes to execute, --force for non-empty tables. Deploy order: flask db upgrade -> flask plugin upgrade-all -> flask plugin prune-schema --yes --force. Regression tests in tests/test_plugin_prune_schema.py (run in the backend CI job).

7.5 KiB Raw Blame History