Add ADR-013: plugin catalog, curated shelf, lean per-site builds

Design record for distributing optional plugins across GE sites: a small
mandatory core plus a catalog of optional plugins, packaged as signed
versioned artifacts, served from a transport-agnostic read-only shelf (a
SharePoint-synced or sneakernet folder - untrusted either way because every
decision-bearing byte is signed), verified at adopt AND at every load and
migrate. Lean per-site builds stage only chosen plugins into the backend
image and SPA bundle.

Status PROPOSED. Grounds the design in the real loader/contract/migration/
frontend code and records defects to fix along the way (upgrade-all
migrating unadopted folders, enable-without-install, reverse-dep checks
blind to unloaded plugins, missing cycle detection and dependency closure,
hardcoded plugin imports in the SPA router). Honest on scope: the frontend
re-org is the long pole (one core-router change plus per-plugin relocation),
not a mechanical move. Phased 0-5 with schema-lean and runtime-JS delivery
explicitly deferred.
This commit is contained in:
cproudlock
2026-07-18 17:40:29 -04:00
parent 603872ff76
commit 3ac5ed2580
3 changed files with 401 additions and 0 deletions

View File

@@ -22,6 +22,7 @@ Architecture decisions live in `docs/adr/`. Read those before making schema or c
- ADR-010: Frontend plugin hook contract - ACCEPTED
- ADR-011: Machines rename + modeltypes retyping - ACCEPTED
- ADR-012: GE-Enforce manifest ownership in shopdb - ACCEPTED
- ADR-013: Plugin catalog, curated shelf, and lean per-site builds - PROPOSED
## Coding convention

View File

@@ -0,0 +1,399 @@
# ADR-013: Plugin Catalog, Curated Shelf, and Lean Per-Site Builds
- Status: PROPOSED
- Date: 2026-07-18
- Deciders: cproudlock
- Relates to: ADR-002 (contract versioning), ADR-003 (plugin distribution), ADR-004 (per-site instances), ADR-008 (per-plugin migrations), ADR-009 (frontend plugin gating), ADR-010 (frontend hook contract)
## Context
Every site today ships identical code. The backend image contains all 13 plugin
directories (Dockerfile COPY at line 55; the header comment listing "eleven core
plugins" is stale), and the SPA compiles every plugin's routes and views via a
static glob (frontend/src/router/index.js:11) plus hardcoded imports. A site's
"chosen set" exists only as runtime enable flags in instance/plugins.json.
Disabled is not absent: a site that never wants printedparts/usb/network still
ships, and can execute, that code.
Distribution per ADR-003 is "drop a directory into plugins/ by hand". There is
no artifact format, no signing, no catalog, no validate gate, and the loader
trusts whatever it finds on disk (loader.py:61-73 discovers any folder with a
plugin.py; loader.py:185-224 loads it; migrations.py:29-62 runs its DDL with
full DB privileges). Plugins run in-process with the full shopdb.api surface
including db, so the only tenable security model on air-gapped GE networks is
curation plus cryptographic provenance, enforced everywhere code can execute,
not sandboxing.
Known defects this ADR also resolves:
- upgrade_all_plugins checks hasattr(registry, 'list_installed') which never
exists (registry has only get_all/get_enabled_plugins), so it always falls
back to migrating every folder on disk, adopted or not (__init__.py:94).
- PILOT-DEPLOY.md enables plugins that were never installed; enable refuses.
There is no declarative "apply this chosen set" operation.
- Reverse-dependency checks on uninstall/disable read only LOADED plugin
instances, so an installed-but-unloaded dependent is invisible.
- Dependency install/enable is check-only; nothing computes a closure, and
install-a-dependent can fail at its own load step because the dependency was
installed disabled (default_enabled=false on employees).
- The dependency sort has no cycle detection.
- Soft couplings (geenforce -> computers, notifications -> employees) are
invisible to the manifest graph.
Frontend reality check (this drove the design below): plugin UI is NOT one
folder per plugin. Routes live in routes/<plugin>.js (slides has none), inside
the shared core.js (computers report, printers toner report, employees detail,
slides settings), and as six hardcoded top-level imports in index.js itself
(/parts-kiosk, /tv, /print/printer-qr x2, /print/usb-labels,
/print/printedparts-labels). View dirs mismatch plugin names (computers ->
views/pcs). Plugin settings cards sit in shared views/settings/
(DellWarrantySettings, ZabbixSettings, SlideManager, EmployeeDirectory,
MeasuringToolTypesList, PrintedPartsSettings), plugin print views in shared
views/print/, and some views span plugins (AssetLabel.vue serves five asset
types; PCDetail imports WarrantyPanel). Any lean-frontend design that only
moves routes/*.js and views/<plugin>/ fails the build the moment a plugin is
pruned. This ADR scopes that work honestly instead of calling it mechanical.
## Decision
### 1. Tiering: mandatory core is the core package; all plugins are catalog-optional
- The mandatory core is the non-plugin shopdb/core/ package (auth, users,
assets, locations, vendors, models, settings, audit, dashboard, search,
reports, plugin management). It already survives every plugin being absent
via hasattr/lazy-import guards. No plugin is promoted into it.
- New optional manifest field `tier: "core" | "optional"`, default "optional".
All 13 existing manifests are unchanged and unchanged in meaning. The
lifecycle gains a guard: uninstall_plugin and disable_plugin refuse a
tier:core plugin (alongside the reverse-dependency checks at
__init__.py:269-278 and :351-360). No plugin ships tier:core initially; the
field and guard exist so a future curation decision is a manifest edit, not a
framework change.
- Per-site mandates live in the site profile (section 5): a `locked` list the
profile applier refuses to remove. This preserves ADR-004 site autonomy: a
wing site can mandate usb without the framework mandating it fleet-wide.
- New manifest field `optional_dependencies: []` (names only, loader-ignored).
Declared for the verified soft couplings: geenforce lists computers
(service.py:32-43 loses app-detection gates without it), notifications lists
employees (routes.py:151-171 loses name/photo enrichment). Catalog listing
and adopt WARN on unmet optional deps; nothing blocks.
- Hard `dependencies` gains optional PEP440 ranges ("employees>=1.1").
validate/adopt honor ranges; the runtime loader keeps name-only semantics
(specifier stripped) so no loader behavior changes. The single existing hard
edge printedparts -> employees stays as-is; whether it can relax to optional
(badges.py has an external HR fallback) is a follow-up product question, not
blocked on this ADR.
- Dependency plumbing fixes: _sort_by_dependencies gains cycle detection
(raise PluginDependencyError on a back edge); reverse-dependency checks read
manifests of ALL installed plugins from disk, not loaded instances.
### 2. Packaging: signed, versioned artifacts
Artifact: `<name>-<version>.shopdbplugin` (a zip of the plugin directory:
manifest.json, plugin.py, api/, models/, migrations/, and frontend/ once
section 6 lands) plus two members generated at pack time:
- `PROVENANCE.json`: plugin name, version, publisher id, build timestamp, and
a sorted map of every packaged file path to its SHA-256. PROVENANCE.json is
not listed in its own map, so there is no circular-hash problem and no zip
canonicalization needed; determinism comes from sorted per-file hashes.
- `PROVENANCE.sig`: detached ed25519 signature over the exact PROVENANCE.json
bytes.
New CLI:
- `flask plugin pack <name> --key <path>` (producer side): runs validate on the
directory, then emits the artifact.
- `flask plugin validate <dir|artifact>` (the missing pre-publish gate),
fail-closed pipeline: signature (artifact mode) -> per-file hashes -> manifest
against a new docs/plugin-manifest.schema.json -> name == directory ->
core_version parses as a specifier and admits the target contract version ->
static import-surface scan reusing tests/test_plugin_contract.py logic ->
alembic versions parse. The schema types the known fields (name, version,
description, dependencies, optional_dependencies, tier, core_version,
api_prefix, display_name, default_enabled, provides, settings) and PERMITS
additional properties, so all 13 existing manifests pass unmodified.
The import-surface scan is documented as a lint, not a security control; it is
trivially bypassed by dynamic import. The security control is human review
before signing (section 4).
### 3. The shelf: a read-only folder, transport-agnostic by design
- One config knob: `PLUGIN_SHELF_DIR`. The app only ever reads this folder. It
never speaks SharePoint, OneDrive, or any network protocol.
- Transport is explicitly out of scope and explicitly untrusted. On networks
that can reach corporate M365, a SharePoint document library sync populates
the folder. On strictly air-gapped floors where no sync agent can run, the
folder is populated by robocopy/USB. Both are equally supported and equally
untrusted, because every decision-bearing byte is signed: swapping transport
changes nothing about the trust model.
- Layout: `<shelf>/<name>/<name>-<version>.shopdbplugin` plus
`shelf-index.json` and `shelf-index.sig`.
- The index is SIGNED with the same publisher key and carries a monotonically
increasing `serial` plus a `revoked` list of name-version pairs. Each site
records the last-seen serial in instance state and refuses an index with a
lower serial (anti-rollback of the catalog itself). The index also carries
per-entry version/tier/core_version so `flask plugin shelf list` can display
compatibility without unpacking, but the index is a BROWSE layer only:
adopt reads dependencies, tier, and core_version from the signed manifest
inside the verified artifact, never from the index.
- Trusted keys: `PLUGIN_TRUSTED_KEYS` is a list of pinned public keys delivered
out-of-band in the site's deployed config/image. Keys are NEVER read from the
shelf; a folder that can be written by an attacker must not also carry the
keys that authenticate it. Multiple pinned keys allow overlap rotation.
Revocation of an artifact rides the signed index `revoked` list; a
`flask plugin audit` command warns when an installed version appears there.
- Partial-sync robustness: adopt copies the artifact to a temp location,
verifies signature and every file hash there, then unpacks to
plugins/.staging/<name> and renames into place atomically. OneDrive
placeholder stubs, zero-byte files, or an index referencing not-yet-synced
artifacts all fail closed with a clear "artifact not fully synced/verified"
error.
- `flask plugin adopt <name>[==version]`: resolve version from the shelf,
verify, compute the hard-dependency closure from signed manifests, then for
each closure member in topological order: unpack, INSTALL, and ENABLE (not
install-only; the load gate at loader.py:201-206 checks is_enabled, so an
install-only closure with default_enabled=false deps would fail its own
load). Migrations run via the unchanged per-plugin chain (ADR-008). Refuses
to adopt a version lower than the installed one unless
`--force-downgrade` is given interactively. Prints the restart notice.
- Adopt/install/uninstall remain CLI-only. The admin HTTP surface stays a
read-only catalog view plus the existing enable/disable toggle; because
Flask cannot register blueprints after the first request, any adopt or
enable takes full effect only on restart, and the UI says so. There is no
"install button" that pretends otherwise.
### 4. Trust model: verify at adopt AND at every load and migrate
Signing that gates only adoption is bypassable through every other write path
into plugins/ (git clone, symlink, USB drop) and defeated by post-adoption
tampering. Therefore verification is enforced where code executes:
- Adoption leaves PROVENANCE.json and PROVENANCE.sig inside plugins/<name>/
and records publisher + artifact hash in the registry entry.
- load_plugin verifies the signature against PLUGIN_TRUSTED_KEYS and re-hashes
the plugin tree against the provenance file map BEFORE importing plugin.py
(new step ahead of loader.py:185). Missing or invalid provenance is a
fail-closed refusal in production.
- run_plugin_migrations performs the same verification before executing any
revision, so a routine `flask plugin upgrade-all` can never run DDL from an
unverified folder.
- upgrade_all_plugins iterates registry.get_all() (fixing the phantom
list_installed fallback at __init__.py:94), so unadopted on-disk folders are
never migrated as a side effect of deploys.
- Development and the ADR-003 external-repo/symlink workflow (including
scripts/test-external-plugin.sh) are preserved via `PLUGIN_DEV_TRUST_DIRS`,
honored ONLY when DEBUG or TESTING is set. Production ignores it.
- Cost: hashing 13 small plugin trees at boot is milliseconds; accepted.
What signing does NOT claim: a valid signature proves the artifact is exactly
what a curator reviewed and signed, nothing more. Plugins remain in-process
Python with full DB access. The actual safety control is the human review
before signing; the signature makes that review's verdict tamper-evident all
the way to execution.
### 5. Declarative site profiles and lean backend builds
- `site-profile.json` per site (kept in the site's deploy config): site name,
list of chosen plugins, optional `locked` list. `flask plugin apply-profile
<file>` resolves the closure, installs AND enables in dependency order, runs
migrations, reports which changes need a restart. This replaces the
imperative CLI sequences in DEPLOY.md/PILOT-DEPLOY.md and fixes the
enable-without-install bug.
- Lean backend image: `scripts/build-site.sh` reads the profile and stages
only core + chosen plugin directories into the Docker build context
(correcting the Dockerfile COPY and its stale header comment). Discovery
needs no change; it already scans whatever exists.
- Prerequisite the naive version misses: core hardcodes plugin imports.
shopdb/core/api/search.py (~15 sites), reports.py, assets.py, collector.py,
applications.py, auditlogs.py, and shopdb/cli/__init__.py import
plugins.<name>.* lazily. Some already guard ImportError; ALL must, with
graceful degradation, before any site prunes a folder. This is audited and
enforced by a new CI job that deletes one plugin directory and runs the full
test suite (repeated per plugin). Longer term these aggregators should move
to registry-driven contract hooks (get_search_providers/get_report_sources)
so a new catalog plugin can join search/reports without core edits; that is
scoped as follow-up work, not a blocker for lean builds.
- Schema-lean is DEFERRED to its own ADR. The core baseline 68b3947ae14f
unconditionally creates the 10 pre-cutover plugins' tables, and lifting them
into plugin baselines collides with cross-plugin foreign keys (the
computers-owned installedapps table FKs machines.machineid while computers
declares no dependency on machines). Reversing the cutover would either
introduce undeclared hard deps or drop FKs; neither is decided here. A lean
site therefore carries a handful of empty pre-cutover tables. Accepted.
### 6. Frontend delivery: Path C for rich UIs, Path A for simple ones, Path B rejected
Path B (runtime-loaded JS / module federation) is REJECTED: it moves executable
UI delivery from a signed, statically auditable build artifact to runtime
fetching, which is exactly the wrong direction for an air-gapped,
review-then-sign posture, for zero benefit given restarts are already required.
Path A (declarative JSON UI over generic renderers) is COMMITTED and scheduled
EARLY: the three unwired ADR-010 endpoints (pluginui.py asset-panels:62,
map-overlays:88, asset-presentation:100) get generic core renderers, joining
the already-consumed settings-cards. After this, a simple plugin ships JSON-only
UI with zero frontend build involvement. Sequencing this before the relocation
gives every plugin an escape hatch during the migration instead of after it.
Path C (self-contained plugin frontend) is the primary mechanism, scoped
against the real code, not the idealized layout:
- Canonical home: plugins/<name>/frontend/ containing routes.js (the plugin's
complete route array, INCLUDING routes currently embedded in index.js and
core.js), views/, and settings views.
- A pre-Vite staging step (scripts/stage-frontend.mjs, run by build-site.sh
and the dev script) copies the CHOSEN plugins' frontend/ into
frontend/src/.plugins-staged/<name>/ (gitignored) and generates two files
inside the Vite root: routes.gen.js (aggregated plugin routes) and
meta.gen.js (plugin-supplied icon names, title spellings, settings-standalone
flags, replacing the hardcoded iconMap/TITLE_SPELLINGS/SETTINGS_STANDALONE in
AppLayout.vue, settingsCatalog.js, and index.js). This exists because
import.meta.glob requires a static literal inside the project root and
cannot select a per-site subset by itself.
- ONE-TIME core-router surgery, done first and called what it is: the six
hardcoded plugin-view imports in index.js (PartsKiosk, TVDashboard,
PrinterQRBatch/Single, USBLabelBatch, PrintedPartsLabels) and the plugin
routes embedded in core.js move into their owning plugins' routes.js. Without
this, pruning slides/printers/usb/printedparts fails the Vite build on
unresolvable imports; no amount of glob work fixes it.
- Per-plugin relocation PRs (13), each REAL WORK, not a file move: carve routes
out of shared files, move views (handling name mismatches like computers ->
views/pcs), move the plugin's settings views out of shared views/settings/,
and rewrite relative ../../ imports of core shared code to the @/ alias
(relative paths break at the staged depth). A lint rule enforces alias-only
core imports in plugin frontend code from then on.
- Shared plugin-aware code STAYS CORE and ships to every site: AssetLabel.vue
(spans five asset types), views/print helpers (assetLabel.js, qrLogo.js),
MachineBadge.vue, and cross-plugin panels like WarrantyPanel used by
PCDetail. These already null-guard or gate via isPluginEnabled and must keep
degrading when a peer plugin is absent; over time they migrate to ADR-010
asset-panels so the data becomes plugin-supplied. Lean v1 therefore prunes
plugin-EXCLUSIVE code; a small plugin-aware core remainder is accepted and
shrinks as Path A absorbs it.
- Dual-location transition: the staging step unions legacy locations
(routes/*.js glob, views/<plugin>/) with plugins/<name>/frontend/ until each
plugin has moved. The SPA builds green at every commit; each plugin's move is
independently revertable until the legacy glob is removed at the end.
- Nav and settings cards are already server-driven (dashboardApi.navigation,
settings-cards); the remaining hardcoded plugin entries in settingsNav.js
(/settings/zabbix, /settings/dellwarranty) move to those plugins'
get_settings_cards so pruning leaves no dead links.
### 7. Effect on the 13 existing plugins
- Backend: ZERO code changes required. tier/optional_dependencies/provenance
are additive; pack zips the directory as-is; all plugins keep passing
tests/test_plugin_contract.py. Bundled plugins in a site's image get
provenance stamped at build time by pack, so verify-at-load applies to them
identically.
- Frontend: one relocation PR each, of the honest scope above. Until a
plugin's PR lands it keeps working from its legacy location.
- Operationally nothing changes for a site that does nothing: default builds
remain all-plugins, apply-profile is opt-in, and enable/disable semantics
(including the restart requirement) are unchanged.
## Consequences
### Positive
- A real catalog: sites declare their set in site-profile.json and apply it in
one idempotent command; the chosen set drives backend image, SPA bundle, and
runtime state from one source of truth.
- Curated marketplace with end-to-end provenance: review -> sign -> any
transport -> verify at adopt, at load, and at migrate. Transport (SharePoint
sync or sneakernet) is untrusted and interchangeable, which is exactly right
for air-gapped sites.
- Lean per-site builds: unchosen plugins exist in neither the image nor the
bundle, shrinking attack surface and download size.
- Fixes shipped along the way: upgrade-all migrating unadopted folders,
PILOT-DEPLOY install/enable ordering, reverse-dep checks blind to unloaded
plugins, missing cycle detection, missing dependency closure, hardcoded
frontend plugin metadata.
- Path A completion makes simple plugins UI-capable with no build glue, which
is the cheapest possible marketplace onboarding.
### Negative
- Key management is a per-site operational burden: pinned keys delivered
out-of-band, rotation is a config change everywhere. Accepted as the price of
not trusting the distribution folder.
- The frontend re-org is the long pole: one core-router surgery plus 13
non-trivial PRs. It is sequenced to be always-green and per-plugin
revertable, but it is weeks of work, not a rename.
- Schema is not lean: pre-cutover plugin tables still appear at every site
until the deferred baseline re-org ADR.
- Restarts remain required after adopt/enable (Flask blueprint constraint);
the marketplace UX is honest about it rather than working around it.
- Boot adds a signature + tree-hash check per enabled plugin (milliseconds,
but nonzero).
### Risks
- Key compromise or curation failure: a signature proves provenance, not
safety; a compromised pinned key or a rubber-stamp review signs malware that
every gate will happily pass. Mitigations: multi-key pinning with overlap
rotation, signed revocation list with monotonic index serial, and keeping the
signing key offline with the curator. The static import scan is a lint and
must never be presented as a boundary.
- Rollback/downgrade: mitigated three ways: index serial monotonicity, adopt
refusing version downgrades without interactive --force-downgrade, and the
signed revoked list. Residual risk: a site that never syncs a newer index
cannot learn of revocations; `flask plugin audit` at deploy time narrows the
window.
- Version skew across ADR-004 sites: one shelf serves sites at different
contract versions. Adopt checks core_version from the signed manifest against
the site's own __contract_version__ (authoritative); shelf list shows an
advisory compatibility column from the index. Incompatible artifacts are
listable but not adoptable.
- Partial/placeholder sync files: fail closed on hash verification; the error
message distinguishes "not fully synced" from "tampered" only by wording,
intentionally, since the app cannot tell.
- Dev-trust misuse: PLUGIN_DEV_TRUST_DIRS silently ignored outside
DEBUG/TESTING; a prod config carrying it gets a startup warning.
- Frontend closure drift: plugin views importing cross-plugin components is a
graph the manifest does not model. The lint rule (plugin frontend may import
core @/ paths and its own tree only, never another plugin's) prevents new
edges; existing shared plugin-aware code is explicitly core-owned.
## Implementation phases
- Phase 0, groundwork (small, days): upgrade_all_plugins uses
registry.get_all(); reverse-dep checks read installed manifests from disk;
cycle detection in _sort_by_dependencies; docs/plugin-manifest.schema.json +
`flask plugin validate` (directory mode); `flask plugin apply-profile` with
install+enable closure ordering; fix Dockerfile stale comment. All additive,
zero risk to running sites.
- Phase 1, packaging and signing (medium, about a week): PROVENANCE format,
`flask plugin pack`, validate artifact mode, PLUGIN_TRUSTED_KEYS config,
ed25519 signing tooling and curator docs. No runtime behavior change yet.
- Phase 2, shelf and enforcement (medium-large, one to two weeks):
PLUGIN_SHELF_DIR, signed shelf-index with serial + revoked list,
`flask plugin shelf list` / `adopt` / `audit` with atomic verified unpack;
verify-at-load in load_plugin and verify-at-migrate in
run_plugin_migrations, fail-closed in prod; PLUGIN_DEV_TRUST_DIRS for
dev/test and the external-repo harness; tier:core lifecycle guard;
provenance stamping of bundled plugins at build. This phase completes the
security model; everything after it is delivery optimization.
- Phase 3, Path A completion (medium, one to two weeks): generic renderers for
asset-panels, map-overlays, asset-presentation; migrate settingsNav.js
hardcoded plugin cards to get_settings_cards. Done BEFORE relocation so
JSON-only UI is available during the migration.
- Phase 4, frontend re-org (large, the long pole, several weeks elapsed):
stage-frontend.mjs staging + routes.gen.js/meta.gen.js codegen; ONE core PR
moving the six index.js hardcoded plugin imports and the core.js-embedded
plugin routes into plugin route files; then 13 per-plugin relocation PRs
(views, settings views, name-mismatch dirs, @/ alias rewrite) under the
dual-location union; lint rule for plugin frontend imports. Always-green,
per-plugin revertable.
- Phase 5, lean builds end to end (medium, about a week after Phase 4):
build-site.sh staging backend dirs + frontend staging from site-profile.json;
core lazy-import guard audit finished, enforced by the delete-a-plugin CI
matrix; remove the legacy glob; pilot one real lean site (a location without
printedparts/usb/network) and diff its image and bundle against a full build.
Deferred, each to its own future decision: schema-lean core-baseline re-org
(blocked on the installedapps -> machines FK question), pip/entry-point
distribution (ADR-003 v2), hook-based search/report aggregation contract, and
any revisit of Path B.

View File

@@ -25,6 +25,7 @@ Each ADR captures a single architectural decision: the context, the decision its
| [010](ADR-010-frontend-plugin-hooks.md) | Frontend plugin hook contract | ACCEPTED |
| [011](ADR-011-machines-rename.md) | Machines rename + modeltypes retyping | ACCEPTED |
| [012](ADR-012-geenforce-manifest-ownership.md) | GE-Enforce manifest ownership in shopdb | ACCEPTED |
| [013](ADR-013-plugin-catalog-and-lean-builds.md) | Plugin catalog, curated shelf, and lean per-site builds | PROPOSED |
## Authoring