f9d6b57f0f
The docs lived on one lineage and the fleet code on another. That is why six pages described a shared fleet key as how an agent authenticates, why COLLECTOR-AUTH had to warn that the per-PC credential code was not in the repository it sat in, and why ADR-019 recorded a decision whose implementation was somewhere else. Documentation that cannot see the code it documents goes stale by construction, and no amount of editing fixes that. So the corpus moves here: 84 files, 56 pages, 20 ADRs, the proposals, INDEX.md, llms.txt and both gates. Filenames are unchanged, because a wiki page name is derived from the filename and a rename breaks every link anybody saved. THE POINT OF THE MOVE IS THE CORRECTION IT ENABLES. Every caveat that said "not in this tree" was false the moment the files landed beside the code, so each was rewritten from what is actually here: - COLLECTOR-AUTH gained the real enroll request body, the two bounds on the unauthenticated route (per-IP rate limit, pending cap with stale rows dropped before a real PC is refused), and the rejected pending reason that binds to the endpoint identity rather than the nonce. - ADR-019 now describes an implementation that is present, including its tests, and says plainly that what remains is the FLEET migration, not the code. - proposals/device-identity claimed phase 1 was unbuilt. plugins/computers declares deviceid on the collector schema and prefers it over the older fallbacks. Corrected in the page, its summary and the index. WINDOWS/IIS IS NOW DOCUMENTED AS THE LEGACY PATH, per the product decision, and carefully: it is what production runs today. Every Windows page opens by saying it is supported, not for a new site, and where a new site should go instead. No procedure was cut - an operator on an IIS box at 2am needs these pages to work exactly as well tomorrow. MIGRATE-TOPOLOGY stops being a footnote and becomes the off-ramp, which is why it nearly doubled: what was there did not match how the Docker stack really works here. The Docker material that already lived on this line was kept and given ownership rather than buried under the ported Windows-centric prose: DEPLOY now describes the four services, the migrate service that runs the upgrade chain before api starts, TLS terminated by nginx, and the plugin count corrected from thirteen to seventeen - a number both lineages had wrong. ROADMAP is rewritten from evidence rather than memory: what has shipped, read out of the changelog, the ADRs and the manifests, and ahead of it exactly two items - finalize a distributed release and polish the setup wizard. PROJECT-MAP is generated and was stale; the GENERATOR was what was wrong. --check passes now. API-REFERENCE is retired to a pointer. /api/docs serves 573 operations with derived auth, query parameters and harvested examples, and a hand-written page competing with that is a liability. The file stays so its wiki page and its inbound links survive. AND THE MOVE TURNED ON A SCRUB THAT HAD NEVER RUN HERE. test_docs_publishable keys on docs/*.md, which this line did not have, so it skipped - its own comment predicted what would surface. Porting the corpus activated it and it found 48 lines that must not reach a published wiki: branch names abbreviating the site, assistant-tooling paths, the site name in two deployment pages, and an employee ID in a Windows path that the scrub does not even look for.
151 lines
7.5 KiB
Markdown
151 lines
7.5 KiB
Markdown
<!-- docmeta
|
|
purpose: explanation
|
|
audience: developer, adopter
|
|
status: current
|
|
summary: Why a lean site stops creating database tables for plugins it never installed.
|
|
loadwith: none
|
|
-->
|
|
|
|
# ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables)
|
|
|
|
- Status: ACCEPTED
|
|
- Date: 2026-07-19
|
|
- Deciders: ShopDB maintainers
|
|
- Relates to: ADR-008 (per-plugin migration ownership), ADR-013 (plugin catalog + lean per-site builds), ADR-001 (asset model)
|
|
|
|
## Context
|
|
|
|
ADR-013 delivered lean per-site builds for plugin CODE (backend tree + frontend
|
|
bundle carry only chosen plugins). One residual was left, explicitly deferred:
|
|
the DATABASE. The core Alembic baseline (68b3947ae14f) creates EVERY table,
|
|
including ~30 plugin-owned tables (PLUGIN_TABLE_OWNERS). Each plugin's own
|
|
baseline is a stamp-only no-op (the core chain already made its tables, per
|
|
ADR-008). So a lean site that omits a plugin still creates that plugin's tables,
|
|
empty and unused.
|
|
|
|
The deferral cited a blocker: "the computers-owned installedapps table FKs
|
|
machines.machineid while computers declares no dependency on machines; reversing
|
|
the cutover would introduce undeclared hard deps or drop FKs; neither is
|
|
decided." Investigation refined this:
|
|
|
|
The cross-boundary foreign keys into the machines plugin table are ALL held by
|
|
DEAD legacy tables/columns that predate the asset model (ADR-001) and the
|
|
per-plugin cutover (ADR-008), and are queried nowhere in the codebase:
|
|
|
|
- `machinerelationships` (child/parentmachineid -> machines) - superseded by
|
|
`assetrelationships`. No model, no query.
|
|
- `printerdata` (machineid -> machines) - the pre-cutover printers table,
|
|
superseded by the printers plugin. No model, no query.
|
|
- `installedapps` (machineid -> machines) - a standalone machine-app link table;
|
|
the live relationship is `computerinstalledapps` (FK to computers only). The
|
|
standalone table has no model, no query.
|
|
- `communications.machineid` (-> machines) - a legacy column on the core
|
|
communications table (which is now assetid-based). Not read anywhere.
|
|
|
|
No LIVE plugin table hard-FKs another plugin's table. computerinstalledapps FKs
|
|
only computers.computerid (intra-plugin). So the blocker is dead cruft, not
|
|
live design.
|
|
|
|
## Decision
|
|
|
|
Two phases, both leaving existing databases correct.
|
|
|
|
### Phase 1: retire the dead cross-boundary cruft - ALREADY DONE
|
|
|
|
Investigation found this is already accomplished by existing migrations:
|
|
`7a01_adr001_position_contract` and `7c01_drop_legacy_machine` drop
|
|
`machinerelationships`, `printerdata`, `installedapps`, and
|
|
`communications.machineid` (with its FK). The current schema (verified on the
|
|
dev database) has none of them. So the cross-plugin FK blocker ADR-013 cited no
|
|
longer exists in the live schema - only in the baseline's transient
|
|
create-then-later-drop. No new migration is needed for Phase 1.
|
|
|
|
Precedent: ADR-001 dropped a cross-plugin FK the same way
|
|
(usbcheckouts.machineid -> machines became a soft sentinel).
|
|
|
|
### Enabling change (executed now): idempotent create_plugin_tables
|
|
|
|
`shopdb/plugins/alembic_template.py:create_plugin_tables` now skips any table
|
|
that already exists (inspects the bind first) instead of raising. This is the
|
|
mechanism Phase 2 needs: a plugin anchor can create its tables on a fresh lean
|
|
install AND be a safe no-op on an existing database that already has them from
|
|
the pre-cutover core baseline. Correct and inert regardless of Phase 2 (no
|
|
current caller creates against a populated schema). Verified against the
|
|
plugin-migration suite.
|
|
|
|
### Phase 2 (executed): prune not-installed plugin tables after upgrade
|
|
|
|
Two mechanisms were weighed to make a lean site's database carry only
|
|
core + chosen-plugin tables:
|
|
|
|
- **Relocate** (rejected): pull every plugin-table create/alter out of the core
|
|
chain into the plugin baselines, so the core chain never creates a
|
|
not-installed plugin's table. Measurement killed this: plugin tables are
|
|
created and altered across ~15 released core migrations (baseline plus 7c04,
|
|
7d05, 7d08, 7d13, 7d15, 7d16, 7d17, ...), not just the baseline. Because the
|
|
whole core chain runs before any plugin chain, removing a table's create from
|
|
core while a later core migration still alters it breaks FULL installs too, so
|
|
relocation means surgically rewriting ~15 released migrations - the highest
|
|
blast radius in the project - for a purely cosmetic gain (the omitted tables
|
|
are empty and the lean CODE build already never loads the plugin).
|
|
|
|
- **Prune-after-upgrade** (chosen): leave the entire core chain untouched. Add
|
|
`flask plugin prune-schema`, which drops the tables of every plugin in
|
|
PLUGIN_TABLE_OWNERS that is not installed on this site. Run once at deploy,
|
|
after `flask db upgrade` and `flask plugin upgrade-all`. Same end state
|
|
(core + chosen tables) with near-zero blast radius: no released migration is
|
|
edited, and an existing full site is unaffected because it never runs the
|
|
command.
|
|
|
|
`prune-schema` drops by table name (no plugin-code import), so it works on a
|
|
lean image where the omitted plugin's directory is absent. It is a dry-run by
|
|
default and refuses to drop a table that holds rows unless `--force`, so a
|
|
misfire on a populated site cannot silently delete data. Because the core chain
|
|
seeds a few plugin reference tables (e.g. 7d05 inserts default access
|
|
protocols), initial lean provisioning uses `--force` - at that point the tables
|
|
hold only migration-seeded defaults, before any site data exists.
|
|
|
|
The idempotent `create_plugin_tables` (enabling change above) is what lets a
|
|
lean site later ADD an omitted plugin: its anchor recreates the pruned tables.
|
|
|
|
Verified end to end on MySQL: fresh full install (86 tables) then prune is a
|
|
no-op; fresh lean install (machines + printers) then prune drops the other 19
|
|
plugin tables, leaving core + chosen; second prune is a no-op; the non-empty
|
|
guard refuses without `--force`. Four SQLite regression tests pin the behavior
|
|
(tests/test_plugin_prune_schema.py), running in the backend CI job via the real
|
|
CLI runner: drop-only-not-installed, full-site no-op, refuse-non-empty, and
|
|
force-drops-non-empty.
|
|
|
|
## Consequences
|
|
|
|
### Positive
|
|
|
|
- A lean site's database contains only core + chosen-plugin tables, with no edit
|
|
to any released migration (near-zero blast radius).
|
|
- The cross-plugin FK blocker ADR-013 cited is gone (dead cruft, dropped by
|
|
existing migrations), so plugin schemas are already FK-independent.
|
|
- Adding an omitted plugin to a lean site later just works: the idempotent
|
|
anchor recreates its tables.
|
|
|
|
### Negative / risk
|
|
|
|
- prune-schema is destructive by nature; the row-count guard + dry-run default +
|
|
required `--force` for non-empty tables contain that. It is a deploy-time
|
|
provisioning step, not something to run casually on a live populated site.
|
|
- A lean fresh install still transiently creates then drops the omitted plugins'
|
|
tables (the core chain builds them, prune removes them). Harmless and one-time
|
|
at provisioning; the trade for not touching the released baseline.
|
|
|
|
## Implementation
|
|
|
|
- Phase 1: nothing to do - the dead cross-boundary FK objects were already
|
|
dropped by existing migrations `7a01_adr001_position_contract` and
|
|
`7c01_drop_legacy_machine`; verified absent on a fresh full MySQL upgrade.
|
|
- Enabling change: `create_plugin_tables` made idempotent
|
|
(`shopdb/plugins/alembic_template.py`).
|
|
- Phase 2: `flask plugin prune-schema` (`shopdb/plugins/cli.py`), dry-run by
|
|
default, `--yes` to execute, `--force` for non-empty tables. Deploy order:
|
|
`flask db upgrade` -> `flask plugin upgrade-all` -> `flask plugin prune-schema
|
|
--yes --force`. Regression tests in `tests/test_plugin_prune_schema.py` (run in
|
|
the backend CI job).
|