Files

cproudlock f9d6b57f0f Move the documentation to the line that ships, and correct it against the code

The docs lived on one lineage and the fleet code on another. That is why six
pages described a shared fleet key as how an agent authenticates, why
COLLECTOR-AUTH had to warn that the per-PC credential code was not in the
repository it sat in, and why ADR-019 recorded a decision whose implementation
was somewhere else. Documentation that cannot see the code it documents goes
stale by construction, and no amount of editing fixes that.

So the corpus moves here: 84 files, 56 pages, 20 ADRs, the proposals, INDEX.md,
llms.txt and both gates. Filenames are unchanged, because a wiki page name is
derived from the filename and a rename breaks every link anybody saved.

THE POINT OF THE MOVE IS THE CORRECTION IT ENABLES. Every caveat that said "not
in this tree" was false the moment the files landed beside the code, so each was
rewritten from what is actually here:

- COLLECTOR-AUTH gained the real enroll request body, the two bounds on the
  unauthenticated route (per-IP rate limit, pending cap with stale rows dropped
  before a real PC is refused), and the rejected pending reason that binds to
  the endpoint identity rather than the nonce.
- ADR-019 now describes an implementation that is present, including its tests,
  and says plainly that what remains is the FLEET migration, not the code.
- proposals/device-identity claimed phase 1 was unbuilt. plugins/computers
  declares deviceid on the collector schema and prefers it over the older
  fallbacks. Corrected in the page, its summary and the index.

WINDOWS/IIS IS NOW DOCUMENTED AS THE LEGACY PATH, per the product decision, and
carefully: it is what production runs today. Every Windows page opens by saying
it is supported, not for a new site, and where a new site should go instead. No
procedure was cut - an operator on an IIS box at 2am needs these pages to work
exactly as well tomorrow. MIGRATE-TOPOLOGY stops being a footnote and becomes
the off-ramp, which is why it nearly doubled: what was there did not match how
the Docker stack really works here.

The Docker material that already lived on this line was kept and given
ownership rather than buried under the ported Windows-centric prose: DEPLOY now
describes the four services, the migrate service that runs the upgrade chain
before api starts, TLS terminated by nginx, and the plugin count corrected from
thirteen to seventeen - a number both lineages had wrong.

ROADMAP is rewritten from evidence rather than memory: what has shipped, read
out of the changelog, the ADRs and the manifests, and ahead of it exactly two
items - finalize a distributed release and polish the setup wizard.

PROJECT-MAP is generated and was stale; the GENERATOR was what was wrong.
--check passes now.

API-REFERENCE is retired to a pointer. /api/docs serves 573 operations with
derived auth, query parameters and harvested examples, and a hand-written page
competing with that is a liability. The file stays so its wiki page and its
inbound links survive.

AND THE MOVE TURNED ON A SCRUB THAT HAD NEVER RUN HERE. test_docs_publishable
keys on docs/*.md, which this line did not have, so it skipped - its own comment
predicted what would surface. Porting the corpus activated it and it found 48
lines that must not reach a published wiki: branch names abbreviating the site,
assistant-tooling paths, the site name in two deployment pages, and an employee
ID in a Windows path that the scrub does not even look for.

2026-09-19 09:10:15 -04:00

6.4 KiB

Raw Blame History

ADR-008: Plugin migration ownership (per-plugin chains from the cutover)

Status: ACCEPTED
Date: 2026-07-10
Deciders: ShopDB maintainers
Supersedes: the "Migration strategy (resolved)" section of ADR-004

Context

ADR-004 resolved a Phase 7B footgun by folding every bundled plugin's tables into the single core Alembic chain (migration 7c04_fold_plugin_schema), and later core migrations (7d04..7d16) kept adding plugin schema directly to the core chain. At the time this was the safe choice: bundled-plugin baselines and the core baseline had both been creating the same tables, so flask plugin upgrade-all would collide with flask db upgrade.

That resolution left the per-plugin Alembic engine (shopdb/plugins/migrations.py, shopdb/plugins/alembic_template.py, the alembic_version_<plugin> version tables, and flask plugin upgrade-all) fully built but unused for the bundled plugins. The framework is the product (per the project's charter); a plugin that cannot own its own schema is not really a plugin. Sister sites that adopt or fork a plugin need its schema history to travel with the plugin, not be entangled in the host's core chain. Keeping every future plugin table change in the core chain also means the core chain grows without bound and a plugin can never be cleanly removed.

The blocker ADR-004 worried about (double table creation) only exists while a plugin's own migration tries to CREATE tables the core chain already created. That is avoidable: history is immutable, so the tables already built by the core chain stay owned by the core chain; only NEW schema needs a new home.

Decision

Ownership splits at a fixed cutover, the current core-chain head.

Core chain owns history through its head. The core Alembic chain (baseline 68b3947ae14f .. head 7d16_directoryemployees) remains authoritative for every table that exists at the cutover, including the bundled-plugin tables it created. Those migrations are immutable and are not rewritten. flask db upgrade continues to reproduce the full schema.
Plugin chains own plugin schema going forward. From the cutover forward, any change to a plugin's schema lands as plugins/<name>/migrations/versions/000N_*.py in that plugin's own chain, never in the core chain. The core chain is reserved for core tables.
Every table-owning bundled plugin gets a 0001 anchor. Each such plugin carries a migration chain whose first revision is a stamp-only no-op: upgrade() does nothing because the core chain already created the tables. The anchor exists so the plugin chain has a base that flask plugin upgrade-all can stamp into the per-plugin version table alembic_version_<plugin>. Blueprint-only plugins that own no tables get no chain.
Deploy and upgrade sequence. A deploy runs flask db upgrade (core chain, creates everything through the head) then flask plugin upgrade-all (stamps every plugin anchor and applies any later per-plugin migrations). The same two commands upgrade an existing install; both are idempotent. The registry (instance/plugins.json) tracks each plugin's applied revisions in migrations_applied.
Table-ownership registry. PLUGIN_TABLE_OWNERS in shopdb/plugins/alembic_template.py is the explicit map of which tables each plugin owns; it is kept in sync with the plugins' __tablename__ declarations and pinned by tests/test_plugin_migrations.py.

External (out-of-tree) plugins per ADR-003 already shipped their own chains; this ADR brings the bundled plugins onto the same model, so there is one rule for all plugins.

Consequences

Positive

A plugin's schema history travels with the plugin. Adopting or forking sites get the plugin's migrations, not a slice of someone else's core chain.
The core chain stops accreting plugin schema; it stays about core tables.
A plugin can be evolved (or, with its own downgrade, removed) independently.
The long-built per-plugin Alembic engine is finally exercised on every deploy, so it cannot silently rot.

Negative / cost

Two migrate commands per deploy instead of one. Documented in docs/DEPLOY.md and docs/UPGRADE.md; both are idempotent so the cost is one extra safe call.
A plugin author must now put new tables in the plugin chain and register them in PLUGIN_TABLE_OWNERS, rather than autogenerating into the core chain. The flask plugin new guidance and this ADR spell that out.
The cutover is a discontinuity: tables created before it are core-owned, tables created after it are plugin-owned. The line is the core-chain head at this ADR's date, recorded here so it is unambiguous.

Neutral

No schema changes and no data migration: the anchors are no-ops. A fresh install and an existing install converge to the same state.

Alternatives considered

Keep everything in the core chain (status quo per ADR-004). Simplest operationally but defeats the plugin-as-product goal: plugin schema cannot travel, the core chain grows without bound, and plugins can never be cleanly removed. Rejected.
Rewrite history so plugin tables move out of the core chain into plugin 0001 CREATE migrations. Would make each plugin chain self-contained from empty, but breaks the immutability rule, forces every existing site to re-run a rewritten chain, and re-introduces the exact double-creation footgun ADR-004 fixed. Rejected.
Anchor that CREATEs tables with IF NOT EXISTS guards. Lets a from-empty install build plugin tables from the plugin chain, but then two chains both claim the same tables and drift can diverge silently. The no-op anchor keeps a single authoritative creator (the core chain) for cutover-era tables. Rejected.

References

ADR-003 (plugin distribution; external plugins already ship chains)
ADR-004 (deployment topology; this ADR supersedes its migration-strategy note)
shopdb/plugins/alembic_template.py (PLUGIN_TABLE_OWNERS, shared env runner)
shopdb/plugins/migrations.py, shopdb/plugins/cli.py (upgrade-all)
plugins/<name>/migrations/ (per-plugin chains and 0001 anchors)
tests/test_plugin_migrations.py (ownership + chain + idempotency guards)
docs/DEPLOY.md, docs/UPGRADE.md, docs/PLUGINS.md (deploy sequence)

6.4 KiB Raw Blame History