A lean site still gets every plugin's tables from the shared core Alembic
baseline. prune-schema drops the tables of plugins not installed on this
site, leaving core + chosen-plugin tables, with no edit to any released
migration (the relocate-into-plugin-baselines alternative would mean
rewriting ~15 released core migrations for a cosmetic gain - see ADR-014).
- shopdb/plugins/cli.py: prune-schema command. Dry-run by default; --yes to
execute; refuses non-empty tables without --force. Drops by table name (no
plugin import) so it works on a lean image. MySQL: private AUTOCOMMIT engine
(db.engine's pooled connections sit idle-in-transaction in a CLI context and
would deadlock the DROP on a metadata lock). SQLite: db.engine, restoring the
prior foreign_keys pragma so the StaticPool connection is not left changed.
- tests/test_plugin_prune_schema.py: drop-only-not-installed, full no-op,
refuse-non-empty, force-drops-non-empty.
- docs/DEPLOY.md: lean provisioning step after upgrade-all.
- ADR-014 ACCEPTED; index updated.
Verified on MySQL: full install then prune = no-op (86 tables); lean install
(machines+printers) then prune drops the other 19 plugin tables; second run
no-op. Full suite 1077 passed.
The lean-build endgame: a site ships carrying only the plugins it chose.
- scripts/build-site.sh: reads a site profile, resolves the hard-dependency
closure from manifests, builds the frontend with SITE_PLUGINS (stage-frontend
carries only those plugins), and stages a backend tree of core + only the
chosen plugin dirs. An unchosen plugin is in neither the bundle nor the tree.
- Core lazy-import guard: `flask seed demo` hard-imported the 5 asset subtype
models, which would crash a lean build missing any of those plugins. Now
guarded (a missing model skips its demo section).
- test_lean_build_guards.py: statically asserts NO core (shopdb/core, shopdb/cli)
import of a plugin is unguarded - a lean build omitting that plugin would
otherwise crash. 0 unguarded today.
Pilot verified: a lean build (machines + printers) carries only machines +
printers code - PartsKiosk / ManifestEditor / USBLabelBatch / KnowledgeBaseDetail
/ EmployeeDirectory are absent from the bundle, and only machines/printers plugin
dirs stage into the backend. (Sidebar labels for absent plugins remain - the
accepted small plugin-aware core remainder.) Guard test + naming green.
Wires the ADR-010 get_map_overlays hook into the floor map so a plugin decorates
markers as JSON, no map code. ShopFloorMap fetches /api/pluginui/map-overlays,
then each overlay's endpoint (per-asset [{assetid, color, label}]), joins by
assetid, and draws a ring or badge circleMarker on matching markers plus a
legend entry - all as extra Leaflet layers cleared and redrawn with the markers.
Aligned the measuringtools calibration overlay endpoint to the documented
contract: it now returns {assetid, color, label} (was {calibrationstatus,
statuscolor}) and only decorates due/overdue tools.
Additive + guarded (assetid null check, per-endpoint try/catch, cleanup on
re-render), so the map degrades to no decorations on any failure. Verified: the
overlay endpoint serves the contract shape, the map renders without error, and
the frontend builds. A populated badge needs a site that actually places
measuring tools on its map (this dataset places none). 38 measuringtools/pluginui
tests, 58 vitest, build + naming green.
Global-search rows built the plugin detail URL from a hardcoded url_map of
plugin routes in core. Now core prefers a plugin's declared
get_asset_presentation route (ADR-010), substituting the core assetid via the
plugin's by-asset resolver; types that have not declared fall back to the legacy
id-keyed map, so nothing breaks. Measuring tools (which declare the route) link
through it now; machines/PCs/printers/network migrate off the hardcode as they
add a by-asset route + declaration. Presentation map is collected once per
search (cached on flask.g). 2 consumer tests; 26 search tests green.
Wires the ADR-010 get_asset_panels hook to a generic frontend renderer so a
plugin adds detail-page UI as JSON, no Vue. This is the Path A foundation that
lets simple plugins ship UI without a frontend build.
- components/PluginAssetPanels.vue + pluginAssetPanels.js: fetches
/api/pluginui/asset-panels for an asset, then each panel's data endpoint, and
renders by mode: list (title + status badge + meta lines via a field map),
keyvalue, table (declared or inferred columns), badge. Pure mapping logic is
in the .js module and unit tested (9 specs), same pattern as entryForm.js.
- New 'list' render mode with a declarative field map (title/badge/meta),
documented on the hook in base.py.
- Warranty migrated to it: get_asset_panels now declares a 'list' panel + map
that reproduces WarrantyPanel's output (vendor title, status badge with color
+ label map, servicelevel/ends/tag meta, manage link) with zero
warranty-specific frontend code.
- MachineDetail swapped from <WarrantyPanel> to <PluginAssetPanels> (pilot); the
hero warranty badge is unchanged. Verified end to end: the API serves the list
panel + map and the warranty rows; the page renders without error.
Rollout of the other 4 detail pages (PCDetail, PrinterDetail, NetworkDeviceDetail,
MeasuringToolDetail) and the map-overlays / asset-presentation renderers are
follow-up Phase 3 commits. 58 vitest, build clean, 1067 backend pass, naming green.
Fourth review found the last import-path bypass: the is_dir() branch returned
None for a name whose dir has no __init__.py, without checking a same-name
sibling file. FileFinder loads a file over an init-less namespace dir, so an
attacker could overwrite a signed foo.py with malicious bytes, mkdir an empty
foo/ next to it (PROVENANCE untouched, still verifies), and any import of that
name ran the unverified foo.py - RCE with only plugins/ write access.
Fix: the dir-with-no-__init__.py branch no longer returns early; it falls
through to the leaf .py hash gate and the non-source refuse check. Invariant:
find_spec returns None for a plugins.* name ONLY where FileFinder would also
find nothing on the same __path__.
Everything else was confirmed sound this round: the owned plugins root, exec of
exact verified bytes (never .pyc/.so), the extension/bytecode refusal, plugin.py
read-once, the provenance signature gate, dev-exemption scoping, and #3/#4.
Symlink, suffix-ordering, cache-lifecycle, and loader-internal angles cleared.
2 regression tests (tampered .py + sibling dir; unsigned .py + sibling dir). All
13 bundled plugins still load under enforcement; 1067 pass, naming green.
Third review found the meta_path guard leaked exactly where it delegated to the
stdlib import system:
1. Non-.py submodules (CRITICAL). When a name had no dir and no .py, find_spec
returned None and the stdlib loaded a planted .so (ExtensionFileLoader) or a
sourceless .pyc unverified - an attacker deletes a signed .py and drops a
same-named .so with arbitrary init code, run on a normal request via core's
`from plugins.<name>.models import ...`. The guard now refuses any name for
which a non-source importable candidate (EXTENSION_SUFFIXES + BYTECODE_
SUFFIXES) exists on disk; None is reserved for genuinely-absent modules.
2. Top-level plugins/__init__.py (CRITICAL). It is in no plugin's provenance,
is attacker-writable, and Python runs it before any guarded submodule. The
guard now owns `plugins`: it execs an EMPTY package body (search points at
the plugins dir), so an overwritten plugins/__init__.py never runs.
Also: specs are built with spec_from_file_location so loaded modules get
__file__/__path__ (Flask blueprint root paths need it) while the loader still
execs the verified in-memory bytes - never re-reading the file.
Verified end to end: under PLUGIN_REQUIRE_SIGNED with all 13 bundled plugins
stamped, the app boots and loads every plugin through the guard; a tampered
plugin file is refused at load. 4 new guard tests (planted .so, sourceless
.pyc, absent-module defer, neutralized package root). Prior fixes #3/#4
confirmed still sound by the review. 1065 pass, naming green.
A re-review showed the previous "single import choke point" claim was wrong:
`plugins` is a normal importable package, so core request handlers that do
`from plugins.<name>.models import ...` never passed through the loader and ran
unverified - an attacker who dropped a file into plugins/<name>/ got arbitrary
in-process code execution on an ordinary HTTP request (and a planted .pyc ran
from cache). Gating load_plugin_class covered only plugin.py, one path of many.
Fix: importguard.py installs a sys.meta_path finder (under enforcement) that
intercepts EVERY plugins.<name>.* import, verifies the plugin's signed
provenance once, then verifies each module file against it and execs the exact
bytes it hashed - read once, compiled, exec'd, never a .pyc, never a re-opened
file. This closes the submodule bypass and the planted-bytecode read, and the
read-once exec closes the verify-vs-exec TOCTOU on the import path. The import
system, not one method, is the real choke point.
- init_app installs the guard when PLUGIN_REQUIRE_SIGNED, clears it otherwise.
- load_plugin_class now verifies plugin.py from a single read and execs that
buffer (finding #3 on that file); its submodule imports flow through the guard.
- docs: stamp-bundled must cover every plugin dir present (a disabled plugin's
module can be imported by core); recommend a read-only plugins/ owned by the
deploy user as defense in depth (closes the residual migrate-time race an
attacker with concurrent write could otherwise attempt).
Earlier review's fixes#3 (migrate code paths) and #4 (shelf content binding)
were confirmed sound and are unchanged. 7 import-guard tests (submodule verify,
tamper, unsigned refused, planted .pyc ignored, real import through the guard,
install/uninstall). 1061 pass, naming green.
An adversarial security review of the Phase 2 trust model found four real
bypasses (two remote-triggerable to in-process code execution). Root cause for
three: the set of bytes verification covered was smaller than the set that
determined execution. Fixes:
1. Bytecode-cache blind spot (CRITICAL). verify_dir excluded __pycache__/.pyc,
so a planted cache ran while escaping the hash map. verify_dir now flags any
bytecode as an unexpected file; the loader strips bytecode before verify and
imports under sys.dont_write_bytecode, so only verified source executes.
2. Unauthenticated verify-at-load bypass (CRITICAL). load_plugin_class imported
plugin.py with no gate, reachable via discover_available / an anonymous GET
/api/plugins. The verify+strip gate moved INTO load_plugin_class - the single
import choke point every path flows through - so an unsigned/tampered plugin
is never imported. discover_available skips a refused plugin instead of 500.
3. Ungated migration entrypoints (HIGH). downgrade_plugin and get_current_head
(ScriptDirectory imports version modules) ran plugin code with no check. All
alembic-invoking methods now pass through _verify_ok (strip + verify) first
and run under no-bytecode.
4. Revocation/content bypass (HIGH). The signed index bound a filename, not
content; adopt did not bind the delivered bytes to the resolved version, so
revoked bytes could be served under a live filename. The index now records a
per-artifact SHA-256; adopt verifies the on-disk digest and requires the
artifact's own signed manifest version to equal the resolved version.
Enforcement stays default-off; strip/no-bytecode run only under enforcement, so
the unsigned path is unchanged. 6 regression tests (planted bytecode, the
discover import path, downgrade gate, version-swap). 1054 pass, naming green.
Completes the marketplace security model. Verification stops being advisory:
a plugin only loads or migrates when its tree matches a trusted signature, and
plugins are pulled from a signed shelf with anti-rollback and revocation.
Enforcement (default OFF - existing deploys unchanged):
- verification.py PluginVerifier, shared by the loader (verify-at-load, before
plugin.py is imported) and the migration manager (verify-at-migrate, before
any DDL). Fail-closed: an unsigned/tampered/wrong-key plugin does not run.
- Gated by PLUGIN_REQUIRE_SIGNED. PLUGIN_DEV_TRUST_DIRS exempts named dirs but
only under DEBUG/TESTING; production ignores it.
- flask plugin stamp-bundled writes provenance into in-tree plugins so
verify-at-load applies to bundled plugins too (image build step).
- tier:core manifest guard: uninstall/disable refuse a core-tier plugin.
Shelf (shelf.py):
- Signed shelf-index.json (+ .sig): monotonic serial (a site refuses an older
index - anti-rollback), revoked list carried across builds, per-entry
version/tier/core_version for browse. Index is a browse layer only; adopt
reads security-bearing fields from the verified artifact.
- flask plugin shelf-build / shelf-list / adopt / audit. adopt verifies index +
artifact (signature + every file hash), unpacks to staging, re-verifies, then
atomically moves into place and installs+enables the closure. Refuses a
downgrade without --force-downgrade. Anti-rollback serial stored in
instance/shelf-state.json.
- config PLUGIN_SHELF_DIR; the app only reads the folder, never speaks a
network. .env.example + docs/PLUGIN-SIGNING.md document the flow.
22 tests: verifier policy (off / no-keys / signed / tampered / wrong-key /
dev-exempt), verify-at-load + verify-at-migrate integration, tier guard, index
sign/verify + tamper/wrong-key, serial state, revocation, version resolution,
verified atomic unpack + tamper refusal. Live-smoked keygen->pack->shelf-build
->list->adopt->audit + serial guard. 1050 pass, naming green.
Packaging + provenance for the plugin marketplace. No runtime behavior change
yet - verification is available on demand; enforcing it at plugin load/migrate
and pulling from a shelf are Phase 2.
- signing.py: ed25519 key pairs + provenance. Provenance is a sorted per-file
SHA-256 map plus metadata; the detached signature covers the exact
serialized provenance bytes, so verifying is re-hash files, re-serialize,
check signature. verify() accepts any of several trusted keys (rotation).
Uses cryptography (already a dependency).
- packaging.py: pack() builds a signed <name>-<version>.shopdbplugin (zip +
PROVENANCE.json + PROVENANCE.sig). verify_artifact()/verify_dir() re-hash
and check the signature, and flag a tampered file, an unexpected file, a
wrong/absent key - all fail closed.
- CLI: `flask plugin keygen` (publisher key pair), `flask plugin pack <name>
--key` (validates then signs), and `flask plugin validate` extended to a
signed artifact by path (--pubkey, else PLUGIN_TRUSTED_KEYS).
- config PLUGIN_TRUSTED_KEYS: os.pathsep-separated public-key PEM paths,
delivered with the site config, never read from the shelf. .env.example
documents it.
- docs/PLUGIN-SIGNING.md: curator flow (keygen offline, review, pack, publish,
pin keys, rotate).
The signature proves an artifact is exactly what a curator signed, not that the
code is safe - human review before signing is the control. 11 tests: sign/verify
round trip, wrong key, provenance excludes noise, serialize determinism, pack +
verify, tamper -> hash mismatch, extra file, no-key fail-closed, verify_dir.
1028 pass, naming green.
Additive, zero-risk-to-running-sites prep for the plugin catalog. No
distribution or lean-build behavior yet; fixes latent bugs and adds the
declarative + validate tooling later phases build on.
Fixes:
- upgrade_all_plugins iterates registry.get_all(); only adopted plugins are
migrated. Removes the phantom hasattr(registry, 'list_installed') probe
that always fell through to migrating every folder on disk (unadopted DDL
ran with full DB rights on every deploy).
- Reverse-dependency checks on uninstall/disable read dependencies from the
manifest on disk via _installed_dependents, so an installed-but-unloaded or
disabled dependent is counted. Uninstall blocks on any installed dependent;
disable blocks on an enabled dependent.
- _sort_by_dependencies detects a dependency cycle (back edge in the DFS) and
raises PluginDependencyError instead of looping or dropping a plugin.
New:
- flask plugin validate <name>: manifest loads + name match, manifest-schema
check, core_version admits the framework contract, declared dependencies
exist on disk. No new dependency (lightweight checker); schema ships in the
package at shopdb/plugins/manifest_schema.json (docs/ is stripped on
publish). The check caught that provides is an object, not an array.
- flask plugin apply-profile <file>: declarative install AND enable of a
chosen plugin set plus its hard-dependency closure, in dependency order,
idempotent. Replaces the hand-ordered runbook sequences that could enable a
plugin that was never installed. deploy/site-profile.example.json template.
- Dockerfile header corrected (all 13 catalog plugins, not "eleven core").
10 new lifecycle tests (reverse-deps from disk, cycle detection, upgrade-all
scope, profile closure, schema, all 13 manifests match schema). 1018 pass,
naming green.
The gage lab assigns real WJRP asset numbers, so identity splits: the
internal itemcode stays auto-minted and a new optional unique
gagelabtag (migration 0003) carries the lab's number - settable on
create/edit, searchable, and resolved by the kiosk for scans and bare
keypad digits against the numeric tail of either identifier
(unique-match only). The print-files table becomes stacked revision
cards - filename with rev/current badges, one meta line, delete pinned
right - ending the horizontal scroll in that column.
Photos already resolved through the directory at read time, but names
only came from the stored employeename column - empty after a
shopdb-only import, so recertification/recognition cards showed bare
SSOs. New resolve_employee_display_name in the employees plugin
(mode-aware: self-hosted table or external HR) backs a fallback in
both the single-card and split-per-employee paths; stored names still
win when present.
Deleting any user who owned an API token or appeared in the audit log
hit the users FK and 500ed - the import's 'importer' account being the
guaranteed case (its PAT plus every audit row the import wrote).
Tokens are revoked outright; audit history is kept but detached
(userid NULL), so the trail survives the account.
Browsing the catalog (item list, detail, file listings) now requires
authentication plus the view permission, and the /printedparts pages
and the label print page require login. Still deliberately open: the
kiosk endpoints per the decision record, the image serve and file
download (img tags and anchor downloads cannot carry a JWT), and the
reports (product-wide jwt-optional convention). Grant
printedparts.view to the roles that should see the catalog.
printeditemfiles lands as the plugin's first incremental migration
(0002 on the plugin chain - the ADR-008 payoff). Revisions are
append-only per item: upload assigns the next number, records the
uploader from the JWT, enforces an extension allowlist and a 100 MB
cap; download serves the original filename; a permission-gated delete
covers wrong-file mistakes. The detail page gains the revision table
with a current badge. Unique storedfilename is sized 191 so the index
fits MySQL's 767-byte prefix - the per-plugin chain does not apply the
core env's ROW_FORMAT hook.
Alert recipients gain roles: Role joins the 0.13.0 surface, a role
picker on the settings page, and every active member of the selected
roles is folded into the deduped recipient list.
Retire button with confirmation on the detail page (item leaves the
storefront and the kiosk rejects its code; ledger history and label
survive), Restore on retired items, and an Include-retired list toggle
with a badge. Restore is its own permission-gated POST - the generic
update still cannot flip isactive. New codes mint as WJRP0042 style
without the dash; existing codes are immutable bin labels and keep
their form.
Contract 0.13.0 puts the User model on the plugin surface. The
settings page gains a checkbox picker over the user list; selected
users receive low-stock alerts at their account email, merged and
deduped with the free-text address list, inactive accounts skipped,
site alert_recipients still the fallback when both are empty.
Contract 0.12.0: send_email/send_alert join the plugin surface (the
mailer was core-only), PLUGIN-HOOKS and status docs updated, manifest
pins the new floor. The alert fires inside _ledger_write only when a
decrement CROSSES the item's threshold - one alert per depletion,
rearmed by restocking above - and is best-effort after the commit so
mail trouble can never fail a take. Recipients come from
printedparts_alert_email, falling back to the site alert_recipients.
on_enable re-seeds settings idempotently so existing installs pick up
new keys. Crossing/rearm semantics proven by test.
The lab is now a build-along mirroring what actually happened: ten
stages, each with the goal, the divergences, a see-it-work check, and
the errors genuinely hit while building (empty Migration error from a
broken model import, the migration-guard KeyError, the missing Lucide
icon, nested-app-context test writes, Decimal sums, and the authz
sweep catching the deliberately open kiosk take). That last one gets
its explicit EXEMPT_ENDPOINTS entry with a pointer to the decision
record - the net stays, the exception is reviewable. Full suite: 993
backend tests, 49 vitest, frontend build, naming hook, all green.
Two open endpoints: an item lookup by scanned code and the take POST -
the product's first unauthenticated write, held to the decision
record's bar (decrement-only, badge-attributed server-side, bounded,
physically rate-limited; justification in the plugin README). The
/parts-kiosk route is a full-screen no-auth view beside /shopfloor: a
hidden always-focused input consumes keyboard-wedge scans for
whichever step is active, TouchKeypad (net-new 3x4 grid) takes the
quantity, and a success screen resets after a few seconds. Manual
type-in fallbacks cover damaged labels. Kiosk test proves open access,
the over-take guard, the badge policy, and cache==ledger afterward.
get_permissions declares view/create/edit/delete/restock (seeded on
install/enable and by flask seed permissions); every write route adds
require_permission on top of jwt_required. New test proves
authentication alone is not authorization: a role-less member gets
403 where an admin succeeds.
Badge resolver copied from the USB contract (SSO digits, 0<digits>BZ
PayNo wrap) with names from the employees directory and the
unknown-badge policy setting; deliberately copied rather than
cross-imported so the contract test stays green. Restock and adjust
write the ledger row and move the cached quantity in one commit -
the single-commit invariant every write path must use. Adjust
requires a reason and refuses to drive stock below zero. Detail page
gains Restock/Adjust modals. Seven tests cover minting, the
cache==ledger invariant, badge shapes, policy toggle, and auth.
PrintedItem (catalog: code, name, image, cached quantityonhand,
per-item threshold, bin) and PrintedItemTransaction (the ledger:
signed quantity change attributed to a badge-resolved employee).
Both registered in PLUGIN_TABLE_OWNERS; 0001 is a post-cutover real
baseline. The migration-guard test learns the new expected head.
Routes are a placeholder ping until the next stage - the scaffold's
list route imported the deleted scaffold model, which surfaces as an
empty 'Migration error' because the alembic env imports the models
package.
Import-surface docs and docstrings describe the automation as a
migration script; status-doc references in CHANGELOG/ADR-009/ROADMAP
point at repo files. Screenshot/verify tools write to /tmp/shopdb-shots
(created on import) instead of a machine-specific directory.
The legacy-import surface (docs/IMPORT-API.md) is the schema-agnostic contract
every adopting site targets; these close two gaps found while mapping the WJ
classic import.
Network device IP: POST/PUT /api/network now accept an `ipaddress` and
materialize a primary Communication (mirroring the printer route), and GET
(list + detail + create/update result) surface it. Previously a network
device's IP - which lives in the communications table, not on the extension -
had no HTTP import path at all.
communicationtypes seed: `flask seed reference-data` now seeds the eight
canonical communication types (IP/Serial/Network_Interface/USB/Parallel/VNC/
FTP/DNC), which IMPORT-API.md already documents as a prerequisite. The IP type
must exist before any asset import so printer/network routes can attach an IP.
There is no CRUD endpoint for these, so seeding is the only path.
Tests: network create/update IP upsert + GET surfacing + seed creates IP type.
204 targeted tests pass; naming + pyflakes green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
From the database review (verdict: sound-with-minor-issues). Applies the
actionable findings.
Redundant indexes: five non-unique secondary indexes duplicated a named idx_*
or a unique index on the same column - ix_communications_assetid,
ix_computers_hostname, ix_networkdevices_hostname, ix_printers_hostname (each
shadowing an idx_*), and idx_usb_serial (shadowing the serialnumber unique
index). Removed the redundant index source from the models (column index=True /
the extra db.Index) and added core migration 7d25 dropping the live duplicates.
The unique ix_*_assetid indexes are kept (they enforce assetid uniqueness).
Dead column: usbcheckouts.machineid was a NOT NULL soft-ref to the retired
machines table storing sentinel 0 (ADR-001). Dropped from the model + the
machineid=0 literal in selfhosted checkout; usb plugin migration 0002 drops it
live (downgrade restores it default 0).
Index: notifications.businessunitid (filtered by the shopfloor feed) was
unindexed; added index=True + notifications migration 0002.
CI: new migrations-mysql job proves the real multi-site deploy path - fresh
`flask db upgrade` + per-plugin install on utf8mb4 MySQL from empty, asserting
table count + charset and a clean second-run no-op. The pytest suite only
exercises SQLite create_all(), so a regression in the Alembic chain on MySQL
would otherwise ship undetected.
Verified: fresh core upgrade on a scratch utf8mb4 MySQL builds clean + no-op on
rerun (redundant indexes absent, unique assetid kept); plugin migrations applied
+ verified on the dev DB (machineid gone, bu index present). 953 backend tests
pass; naming + pyflakes green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fills the review's highest-value coverage gaps in the framework's own core
feature.
New tests/test_plugin_lifecycle.py: manager enable/disable dependency guards
(beta depends on alpha - enable-beta-first refused, disable-alpha-while-beta-on
refused) via synthetic plugins; enable seeds a plugin's RBAC permissions
idempotently; registry disable-survives-reload, corrupt-file recovery, and the
equipment->machines rename migration; `flask seed settings` idempotency.
test_plugin_contract.py: BUNDLED_PLUGINS now covers all bundled plugins incl.
geenforce, measuringtools, warranty (was 9, contradicting the CLAUDE.md "all
bundled satisfy the contract" claim); the structural checks now run against them.
test_authz.py: account lockout auto-unlock path (expired lockeduntil -> correct
password logs in and clears the lock state), previously untested.
All new tests pass; naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Settings exposure (review medium): GET /api/settings and /api/settings/<key>
now return the full table only to an authenticated principal. Unauthenticated
callers (kiosk dashboards, print pages, login screen, setup router) get just a
public allowlist - categories branding + map plus a named set (site_base_url,
facility_name, printer_hostname_template, contact_email_domain,
servicenow_enabled, setup_complete). A non-public single-key GET returns 404 so
existence is not confirmed. Secrets stay masked in both cases. Closes the
unauthenticated enumeration of smtp_host / employee_db_host / zabbix_url /
servicenow URLs. Allowlist mirrors the keys siteSettings.js + mapConfig.js +
setupState.js read before login.
audit.view (review low): the three audit-read routes (list, entity-history,
stats) were jwt_required only despite a defined-but-unwired audit.view
permission; now gated by it (seeded to admin), so a role-less member or unscoped
PAT can no longer read the cross-user audit trail.
flask seed test-user (review low): refuses outside DEBUG/TESTING - it creates
the well-known admin/admin123; production sites use `flask seed admin`.
Tests: unauthenticated allowlist + authed-full-masked + private-key-404, and
member-403 / admin-200 on audit routes. 336 authz tests pass; naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add test_geenforce_ddl_parity to lock the manifest models against their
Alembic baseline (catches model/migration drift for a chain that is still
amendable pre-deploy).
Retire the "Collector PC Types" settings card: GE-Enforce scope
computertypeid supersedes the pctypemap editor UI (ADR-012). The collector
still reads pctype_mapping(), so the backend map stays; only the editor
surface is removed, with a deprecation note in pctypemap.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fleet-install compliance for app-linked manifest entries: new service
compliance_for_scope + GET /geenforce/scopes/<id>/compliance count active
ComputerInstalledApp rows by curated appid (null-safe when computers plugin
absent). ManifestEditor gains a compliance panel. Curated appid stays shopdb
metadata and never enters manifest JSON, so behavioral parity is unaffected.
Inline manifest payloads: store_inline_payload (sha256, 1MB cap,
payloadsource='inline') + POST/GET /geenforce/entries/<id>/payload; editor
gains an upload control. Entry payload metadata surfaced in _entry_payload.
Frontend test harness: extract the editor's entry-form logic into pure
entryForm.js (buildEntryPayload, describeEntry, availableEntryTypes, scope
gates, ...) and cover it with 45 vitest tests. ManifestEditor now imports
those helpers, so the tests exercise the shipped code path (no duplication).
908 backend tests pass; vitest 45 pass; frontend build green; naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The honest replacement for the backed-out auto-seeding: instead of scraping
manifest labels into duplicate Application rows, an entry can be LINKED to an
existing catalog Application, cross-referencing what shopdb already tracks.
- Model: manifestentries.appid (nullable soft ref to core applications; in the
0001 baseline). It is shopdb METADATA, deliberately NOT a manifest field - it
never appears in the rendered manifest JSON, so enforcement + parity are
unaffected (test asserts it stays out of the preview manifest).
- API: _entry_payload returns appid + resolved appname; create/update accept an
optional appid (validated, unknown id ignored, null unlinks) via _apply_app_link;
GET /geenforce/applications is the picker source (id + name).
- Editor: a "Tracked application (optional)" select in the entry modal, and the
entry summary line notes the linked app ("...; tracked: eDNC").
- Foundation for a future desired-vs-observed compliance view.
889 tests green (incl. the link test + parity/migration unaffected); build +
naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A deep Fable review of the real manifest corpus (READ-ONLY reference) showed the
manifests are an ENFORCEMENT PROGRAM, not an application inventory, and that
auto-seeding the Applications catalog from entry Type + Name was wrong:
- The catalog ALREADY tracks these apps from the classic-shopdb migration, with
version histories (PC - DMIS, UDC x11 versions, eMX / eDNC, CLM, CSF, Oracle
Database, FormTracePak). Seeding from manifest labels created DUPLICATES under
different names (PC-DMIS 2016 vs PC - DMIS; eDNC (bundles NTLARS) vs eMX / eDNC;
OpenText HostExplorer ShopFloor vs CSF). It also misclassified config drops
(eMxInfo.txt) as apps and could never match a PC's reported ARP name.
So the seed-applications command + service are removed. Properly linking
manifest entries to the EXISTING catalog is a curated feature, not label-scraping.
Two REAL bugs the review found are fixed and kept:
- Report status (R4): every healthy cycle runs Always/no-detection scripts the
engine counts as "installed", so keying self-heal off installed>0 marked the
common scope selfhealed forever and made 'ok' unreachable. Status now derives
from explicit per-entry self-heal flags only; the stored flag no longer infers
from action=='installed'; the client kit doc reflects it.
- PCTypesStrict (R5): the runtime engine has no strict handling (preinstall
runner only). filters.matches_pctype now applies strict only when phase ==
'preinstall'; simulate + parity thread the scope phase through; the strict test
uses a preinstall scope.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
seed-applications: a flask geenforce seed-applications command + service that
reads the imaging-PC-type manifests and creates a core Application for every
installer entry (MSI/EXE/CMD/BAT), so shopdb tracks what GE-Enforce actually
deploys. Idempotent, deduped by appname; File/Registry/PS1/INF config entries
are skipped. Run against the West Jefferson reference: 27 apps tracked (PC-DMIS
2016/2019/2026, eDNC, Oracle Client, Adobe Reader, HostExplorer, the VC++ redist
matrix, Keyence VR-6000, PowerShell, Display Kiosk, ...). 2 tests.
Editor: the CMM version gate (_CmmVersion) now only shows for CMM scopes - it is
metrology-specific, so a printer/common entry form no longer carries the
irrelevant field.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Consolidated fixes from a three-dimension adversarial review.
Data-loss (HIGH): the manifest entry editor stripped fields the form did not
expose, because PUT /entries is a full reset-then-apply. The form now captures
everything - InUseCheck processes as structured name/ExePath/timeout rows (not
just names), LogFile, and the three preinstall flags as checkboxes; the dead
payload-source control (never wired) is removed. New regression test proves an
edit preserves ExePath/timeout/LogFile/PreEnrollment/PCTypesStrict.
Update-entry crash (found by that regression test): replacing an entry's
one-to-one InUseCheck (unique entryid) collided with the old row mid-flush ->
IntegrityError -> 400. update_entry now frees the old InUseCheck (delete+flush)
before populate re-inserts it.
Export truncation (MEDIUM): export_scope_to_share used a plain truncating open,
so a failed/partial write left the live on-share manifest (every PC reads it)
empty. Now writes a temp file in the same dir and os.replace() atomically.
Report dedup case bug (MEDIUM, confirmed by scratch test): the iscurrent demote
matched hostname case-sensitively while the read path uses ilike, so a PC
reporting different casing left two iscurrent rows and double-counted. Demote is
now case-insensitive; regression test added.
Simulator fidelity (MEDIUM): PCTypesStrict was captured but ignored by the
filter mirror, so the simulator wrongly matched a collections-only strict entry
to a nocollections PC via the shared Standard alias group. matches_pctype now
honors PCTypesStrict (disables alias expansion); test added.
Hardening: removed the dead/unscoped GEENFORCE_API_KEY env fallback (never wired
into config; tokens are the only path); create/update entry return 400 on a
duplicate Name instead of 500; parity now asserts scope-level Version/Site; a
new test guards real-manifest field lengths against column limits (the DB-free
parity harness can't see truncation); error handling added to the previously
unguarded editor + reports API calls.
Full suite green; naming + frontend build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rounds out the Milestone 1 admin experience: author + publish in shopdb, push
to the share by a button, and see what the fleet actually did.
Export to share:
- GET/PUT /api/geenforce/config stores the on-share export root (Setting
geenforce_share_root); POST /scopes/<id>/export-share writes the current
published JSON to <shareroot>/<scope>/manifest.json (preinstall.json for the
preinstall phase), backing up the existing file to _meta/history first.
geenforce.publish gated. The engine and PCs are untouched - this is the safe
Milestone 1 push whose rollback is restoring the history backup.
- Editor: a share-root config row + an "Export to Share" button per scope.
- 3 tests (config roundtrip, export writes the file, second export backs up).
Fleet-compliance UI (Settings > Enforcement Reports):
- New page over GET /reports + /reports/<id>: latest report per PC with
received (applied vs latest published version), status (ok/selfhealed/failed),
and install/skip/fail counts; row detail shows per-entry outcomes with
self-heal flags, exit codes, and messages. Hostname/PC-type filters.
- ADR-010 settings card + ADR-009 plugin-gated route.
Full suite 883 green; frontend build + naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full HTTP admin surface behind the manifest editor (geenforce.manage for edits,
geenforce.publish for shipping):
- Scopes: POST/GET/PUT/DELETE /scopes[/<id>] (create imaging PC types, edit the
ComputerType/MeasuringToolType mapping + metadata, delete).
- Entries: POST /scopes/<id>/entries, PUT/DELETE /entries/<id>. Payloads use the
manifest Applications[] shape; populate_entry (refactored out of build_entry)
updates an entry in place, resetting omitted fields and replacing children.
- Reorder: PUT /scopes/<id>/entries/reorder enforces the ordering contract
(body must list exactly the scope's entry ids).
- Simulate: GET /scopes/<id>/simulate?pctype&subtype&hostname&machinenumber&
cmmversion returns which entries apply and which filter excluded the rest,
reusing the engine-mirror filters. The "what would this PC get" tool.
- Publish lifecycle: POST /scopes/<id>/publish (records publishedby from JWT),
GET /scopes/<id>/versions, GET .../versions/<n> (frozen manifest),
POST /scopes/<id>/rollback.
Entry type validated against ENTRY_TYPES; 8 CRUD tests. JWT+permission gated so
the authz sweep covers them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PCs now report enforcement results back to shopdb, closing the desired-vs-observed
loop.
- POST /api/geenforce/report (geenforce.report service token): each cycle a PC
posts the published version it applied, install/skip/fail/filtered counts, and
per-entry outcomes.
- Two tables: manifestenforcementreports (latest-per-host + history: applied
version, enforcer version, counts, derived status ok/selfhealed/failed) and
manifestenforcementresults (per entry: action installed/skipped/failed,
selfhealed flag, exit code, warning/error message).
- RECEIVED: reports carry the applied version; the admin view derives
receivedlatest by comparing it to the scope's current published version, so
the fleet view shows which PCs picked up an update.
- SELF-HEAL: per-entry action captures drift correction (installed when it
should already be present) vs skipped (already good) vs failed, with messages.
- Admin reads: GET /reports (fleet compliance rollup) and GET /reports/<id>
(per-entry detail). New geenforce.report permission.
- Tables added to the (undeployed) 0001 baseline; geenforce.post_report is a
service-token endpoint so it is exempt from the JWT authz sweep, like the
collector blueprint. 8 reporting tests; full suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First execution phases of docs/proposals/ge-enforce-plugin.md. The GE-Enforce
manifest becomes shopdb data.
P0 scaffold: new geenforce plugin (api_prefix /api/geenforce, default_enabled
false, core_version >=0.7.0). Registered in PLUGIN_TABLE_OWNERS (ADR-008); its
0001 baseline really creates the tables.
P1a model: one wide manifestentries table + entrytype discriminator (not STI,
not JSON blob), manifestscopes (UNIQUE scopename+phase), the three multi-value
filter child tables, inusechecks + processes, immutable manifestpublishedversions
(frozen rendered JSON), manifestpayloads (inline, capped), pctypealiases
(mirror of the engine lib's alias graph). regvalue stored as its raw JSON
literal so DWord typing survives.
P1c importer + exporter: parse common + gea-shopfloor-* + preinstall.json into
draft rows and rebuild the JSON verbatim from rows in sortorder.
P1d parity harness (GATE A): filters.py mirrors the engine's four filter
functions + alias graph; parity.py proves import+export is behaviorally lossless
(field-identical + same-entries-fire across 18 machine-profile fixtures) WITHOUT
byte-diffing. Verified PASS against all 11 real reference manifests (64 entries)
and a synthetic site-neutral fixture covering every type/filter (the CI gate).
First slice (gea-shopfloor-cmm shape): service layer (import/publish/rollback/
export-to-share), CLI (parity, import-share, publish, export-share), and the
client endpoint GET /api/geenforce/manifest serving the current published
snapshot (never the draft) with ETag/304. Split permissions
geenforce.manage/publish/fetch. Tests prove import->publish->serve, draft edits
never change served bytes, publish+rollback, and auth (401 unauth/wrong-scope).
Contract 0.11.0: added service_token_authorized(scope) to shopdb.api so plugin
service endpoints authorize a scoped managed token without importing core token
internals. Documented in PLUGIN-HOOKS.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Metrology PCs (CMM, Keyence, Genspect, wax-and-trace imaging pc-types) drive
an attached measuring instrument. The PC itself stays a shopfloor PC, but the
collector now models the instrument:
- New METROLOGY_TOOL_MAP (pctypemap.py) maps those pc-types to a
MeasuringToolType (CMM, Vision System, Genspect, Form Tracer).
- ComputersPlugin._sync_measuringtool_link creates the MeasuringTool asset
once and a directional PC->tool "controls" relationship, tagged
collector:measuringtool. Idempotent (re-push reuses, no duplicate asset) and
self-archiving (a PC re-imaged to a non-metrology type deactivates the link
but keeps the asset and any calibration history). Mirrors the printer-link
pattern. The MeasuringToolType is created on demand if not seeded.
- 4 tests: create+link, idempotent re-push, non-metrology skip, repurpose
archives. Non-metrology PCs never warn about a missing controls type.
Settings rail cleanup:
- Collapsible groups so the 13-group rail fits without scrolling (1511px ->
488px). The group containing the current page expands; the rest collapse.
CSS-drawn caret (ASCII source, no Unicode). Empty groups never render, in
both the rail and the landing page.
- Measuring Tools group placed with the other asset groups (right after
Machines) instead of appended last; empty placeholder positions the
plugin-contributed cards.
- Operating Systems moved from PCs to General Reference: OS is cross-asset
(PCs, machines, measuring tools, network devices all run one).
Plus docs/proposals/ge-enforce-plugin.md: a planning doc for refactoring
GE-Enforce/DSC into a shopdb plugin (manifest as shopdb data, payloads on
SMB/HTTP/inline), grounded in the real manifest schema.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collector: the computers collector schema gains defaultprinter and
printers; apply_collector_payload resolves each reported identifier to
a printer asset (windowsname/hostname/sharename/assetnumber/IP,
first-hit case-insensitive) and idempotently syncs relationships -
defaultprinter (directional) for the default, connectedto for the
rest. Collector-created rows are tagged so a re-report archives dropped
links while manual relationships are never touched; unresolved
identifiers warn instead of failing. Both PC and printer detail pages
show the links via the shared relationships card (no frontend change).
GE-Enforce Win32_Printer collection snippet documented.
Searchable custom fields: a per-field searchable flag (migration 7d24);
global search matches custom-field values on flagged active fields and
routes each hit to the asset detail page, reusing the existing
(type,id) dedupe and search_<type>_enabled domain filter. Searchable
toggle on the Custom Fields settings page.
822 tests pass; both verified live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The PC detail Installed Applications section 500d and vanished on any
real PC: ComputerInstalledApp had no to_dict, so the endpoint errored
and the v-if hid the section. Added the serializer (curated version
wins over the raw collected string, app name + description included)
and aligned PCDetail to the flat payload; regression test added.
Also: employee detail skips its USB panels when the usb plugin is
disabled (was firing 404s), and the shopfloor kiosk header is now
light-on-dark for readability.
810 tests pass; PC 259 installed apps verified live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Email: a stdlib SMTP mailer (settings-first config, graceful no-op when
unconfigured), a test-email endpoint wired to the Email settings page,
forced first-login password change (users.mustchangepassword, migration
7d23, /change-password flow), new-user welcome mail, and on-demand
report/alert delivery (POST /api/reports/email + Email Report buttons)
with an external-cron-with-a-scoped-PAT path documented for automation.
All tests patch smtplib - no network.
Labels: a shared /print/asset-label/<type>/<id> view any asset detail
page opens - card or plain style, QR or barcode, configurable encoding.
Per-type qr_target_* templates plus label_default_style/codetype/encodes
settings on the Printing page. Measuring-tool labels default to encoding
their inspection-operation code (derived from the location name, e.g.
0615), so every tool in an area shares the area code - verified by
decoding the rendered QR. Machine labels default to the machine number;
blank-serial handled gracefully.
808 tests pass; both features verified live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Support teams: contact management moved from a row expander to a modal
(Contacts (N) button per team); application detail Support card and the
modal show Email (mailto) and Teams chat buttons for contacts with an
SSO, derived as sso@ + a new contact_email_domain site setting
(default geaerospace.com, blank hides the buttons).
Audit log: hovering a user SSO shows the full name, resolved
best-effort from the employee directory in either mode.
Docs/hygiene from a standards review: CLAUDE.md active-state,
CONTRACT-STABILITY.md and README brought to contract 0.10.0 / 11
plugins / migration head 7d22; get_asset_panels endpoint path fixed in
the hook docstring; leftover debug console.logs removed.
781 tests pass; contacts modal, action-button hrefs, and the audit
tooltip verified live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plugins declare their own RBAC permissions instead of core accumulating
them: 36 permissions moved out of the core catalog into the 9 owning
plugins (core keeps the 19 its own blueprints enforce). The catalog is
resolved dynamically (core + enabled plugins) and feeds the roles grid,
the token scope picker and ceiling, and flask seed permissions;
installing or enabling a plugin seeds its permissions automatically. A
disabled plugin drops out of the assignable catalog while existing role
links keep working. New plugins - bundled or external - now bring their
permissions with zero core edits.
781 tests pass; live-verified with a machines.edit-scoped token.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A token scoped to the new collector.ingest permission is a collector
service token: the collector endpoints accept it via X-API-Key or
Bearer alongside the env fleet keys (which remain the fallback), giving
the fleet credential rotation, revocation, and last-used visibility
from the API Tokens page. Containment holds both ways: a collector
token authorizes nothing else, and no other credential gains collector
access. Shared token validation refactored out of the auth shim; a
Collector service token quick-preset in the create modal; integration
guide documents minting, rotation via site-config.json, and the
service-identity pattern.
765 tests pass; live acceptance matrix verified.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A token may carry a scopes list: it then grants only those permissions,
intersected with what the owner holds at use time, with the admin role
bypass suspended and role-gated routes denied - a scoped token from an
admin account is genuinely limited. Scope ceiling enforced at
create/update too (only permissions the owner holds; 400 lists
violations) and the picker only offers what you hold. Token management
itself now requires the new apitokens.create permission (admin by
default, grantable via roles). Unscoped tokens keep the exact prior
act-as-owner behavior; imports need an unscoped admin token.
Migration 7d22.
756 tests pass; live-verified scoped 201/403 matrix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
API tokens: any user mints named, optionally-expiring tokens
(shopdb_pat_..., sha256-stored, secret shown once) at Settings > API
Tokens; a before-request shim swaps a valid PAT for a request-scoped
JWT of its owner, so the entire existing auth/authz/import-mode stack
works unchanged and revoked/expired tokens 401 cleanly. Built for
long-running scripts - the legacy import no longer dies when a login
JWT expires. Migration 7d21_apitokens; create/revoke audit-logged.
Audited integration gaps fixed: Asset.to_dict serializes measuring
tools (typedata + pluginid - relationship links to tools resolve); map
subtype filter/colors and MapEditor include them; dashboard totals
count them; warranty links use a new by-asset route; the measuringtools
ADR-010 hooks are real (corrected presentation token, implemented
map-overlay endpoint); the login avatar resolves through the
employee-photo helper.
737 tests pass; naming green; frontend builds; both features verified
live end-to-end.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
