docs/ is excluded from the code bundle and its scrub gate, because it goes to
the GitHub wiki instead - via a generator that has no gate at all. So the one
part of the repository written in prose, by people, about internal
infrastructure, was the one part nothing checked.
What was reaching a public wiki: the internal git server's URL and hostname,
.gitea workflow paths, developer home directories in the GE-Enforce cutover
reference, and a dev database root password inside a copy-pasteable command in
the import guide.
All replaced with neutral equivalents. tests/test_docs_publishable.py is now the
gate, at the source, in CI - a wiki page cannot be un-published, so catching this
after the fact is not good enough.
PROJECT-REVIEW.md also referred to internal tooling by name throughout; those
references are generalised. It remains an internal candid assessment of this
project that is nonetheless published, which is worth a separate decision.
CLIENT IP / SPOOFABILITY. docs/geenforce-api-cutover.md claimed that removing the
IIS rewrite rule made the allowlist fail closed and that it does NOT become
spoofable. The opposite is true. IIS never sets X-Forwarded-For on its own; the
rule is the only thing that does. Remove it and IIS still forwards whatever
X-Forwarded-For the CALLER sent, waitress trusts it because it arrives from
127.0.0.1, and remote_addr becomes attacker-controlled - so a token-less caller
can fetch manifests from anywhere on the network. The document and the
_trusted_client_ip docstring now say so, waitress runs with
--trusted-proxy-count=1, and stage 5 checks the rule is actually live rather than
assuming it. The wizard question is rephrased to something an operator can verify
with their network team instead of guessing at.
NON-ASCII. The style gate only ever checked .py/.vue/.js/.ts, so documentation
accumulated em-dashes, arrows and box-drawing characters against this repo's own
convention - including in files added this week. Cleaned, and the gate now uses
INCLUDES_ALL so Markdown, JSON and YAML are covered.
PLUGIN DEFAULTS. The wizard pre-ticked measuringtools and printedparts, both of
which ship default_enabled=false, so every site taking the defaults installed and
enabled them against their manifests. Inno has no JSON parser so the list must be
hardcoded, but tests/test_installer_defaults.py now fails when it drifts.
UPGRADES. The payload copy merges, so a plugin dropped from a site's profile kept
its code forever - which defeats a lean build and leaves core's optional-import
guards succeeding for a plugin the site no longer has. Stale plugin directories
are now deregistered and removed before the copy.
add-plugin used 'plugin install', which for the five default_enabled=false
plugins left them installed but DISABLED - and printed a green success line
anyway. It now goes through apply-profile, and the success line is gated on the
exit code. Invoke-Flask records its own exit status, because $LASTEXITCODE keeps
a stale value when flask.exe is missing and no native command runs.
CHARSET. The utf8mb4 compiler hook lived inline in migrations/env.py, so it
covered the CORE chain only: plugin baselines inherited the server default, which
on a latin1 server means two charsets in one database. It is now
shopdb/utils/mysql_charset.py, imported by both, and preflight reports the
database's default charset.
BACKUP HONESTY. The dump was described as 'all of your asset data'. Uploaded
branding and floor-map images live in instance\ on disk, not in the database, so
a restore from the .sql alone comes back with no map. backup now archives
instance\ alongside it and says both are needed.
VERSIONING. AppVersion was hardcoded at 0.9.0 while the product, the frontend and
the newest tag said 0.7.0 - and 0.9.0 collides with a retired contract version.
Both builders now generate version.iss from shopdb/__init__.py.
Smaller: rollback overwrites .env before deleting it, as uninstall already did;
appcmd unlocks are scoped to this site's location rather than server-wide, with
the wide unlock as a fallback; DEVELOPMENT-SETUP says Python 3.14; the README
plugin list gains printedparts; prune-schema --force is documented as
first-provisioning-only; HTTPS is documented as not-the-default with the steps to
add it; the DBA SQL is on the wizard's database page; the features page says
unticking does not remove an installed feature; and the installer README states
that bundle-lock cannot vouch for the exe itself - that needs signing or an
out-of-band hash, neither of which is wired up.
An air-gapped site cannot be scanned from anywhere else, so when a CVE lands the
only way to answer 'is that component here, and at what version' was to RDP in
and go looking. The frontend was the real blind spot: nothing recorded which
version of leaflet, dompurify, jspdf or html2canvas ends up inside the compiled
SPA.
scripts/generate_sbom.py emits CycloneDX 1.6 covering both ecosystems - every pin
in requirements.txt with the sha256 the installer enforces, and every package in
package-lock.json. Build-only npm packages are marked scope 'excluded' rather
than dropped, so 'not here' stays distinguishable from 'not looked for'.
Dependency edges are real: uv's '# via' comments give the Python graph and
package-lock gives the npm one.
Hand-rolled rather than cyclonedx-py plus cyclonedx-npm because both inputs are
already pinned and committed - this is a format translation, not a scan - and
because the build box may be a work PC with nothing but Python and Node. It is
deterministic by construction: same inputs, byte-identical output, so
regenerating does not churn.
Staged into the application tree by both builders, so it installs onto the
server with the app. shopdb-admin.ps1 verify reports it and searches it by
component name, which is the question actually being asked.
Packages appearing at several depths in package-lock (node_modules/vite and
node_modules/vitest/node_modules/vite) are merged, and a copy reachable outside
the dev tree makes the component count as shipped. Emitting both produced
duplicate bom-refs, which CycloneDX forbids and scanners reject; getting the dev
merge backwards would have hidden a shipped package from a CVE search.
Not covered by bundle-lock.json on purpose: its provenance is git, not the
third-party payload.
The lock records what IS in the wheelhouse, not what the application NEEDS, so
an incomplete wheelhouse was locked, blessed and shipped - and only failed on an
air-gapped server.
That is not hypothetical. Assembling the wheelhouse anywhere other than Windows
silently omits colorama, a win32-only dependency of click, because pip evaluates
environment markers against the machine doing the downloading rather than the
machine being targeted. The bundle built here was short exactly that one wheel.
Both verifiers now cross-check wheels/ against the staged requirements.txt,
ignoring markers, since a requirement guarded by sys_platform == 'win32' is
precisely the one that must be present. Names are normalised to PEP 427 wheel
form, so mysql-connector-python matches mysql_connector_python.
bundle-lock.json is the first real lock: 42 files, cp314/win_amd64 - 39 wheels,
Python 3.14.6, HttpPlatformHandler 1.2 and URL Rewrite. MySQL is absent and
optional; a site choosing the bundled-database option adds it and re-locks.
The naming gate now skips the installer's build output. It contains a staged
copy of the application plus a second SPA build under dist-subpath, which
--exclude-dir=dist does not match, so a staged bundle failed the gate on
vendored minified JS nobody in this repository wrote.
The bundle carries ~40 wheels, a Python installer and two MSIs. All of them run
as SYSTEM on the target server, and nothing verified any of them. A missing
wheelhouse printed MISSING and the script still exited 0, so an empty bundle
compiled into a shippable installer and the failure surfaced on an air-gapped
server with no way to fix it.
bundle-lock.json now records that payload exactly - sha256 and byte size per
file - and verification is set equality: a missing file, an unexpected extra
file, or changed content all fail. Both builders check it and refuse to produce
an unverified bundle; the lock ships inside the bundle and shopdb-install.ps1
re-checks it on the server before running any of it.
This is deliberately a layer above requirements.txt hashes. pip lists every
artifact of a pinned version (cffi 2.1.0 alone has 100 hashes), so it proves a
wheel is genuine, not that it is the wheel this bundle was built and tested
with; it ignores extra files in the wheelhouse; and it covers none of the
executables.
refresh-bundle-lock.ps1 regenerates the lock but refuses to overwrite one until
the operator has seen the diff, because the commit is the review - it is the
only place a change to what runs as SYSTEM becomes visible to a human.
build-installer.ps1 is the whole build natively on Windows, so a work PC needs
no Bash. It shares the plugin closure resolver with build-site.sh.
Both builders now copy the installer scripts from the repository. They were
copied from a downloads folder, so the logic that shipped was not the logic that
was committed and the build worked on exactly one machine.
Two verifiers exist because PowerShell is the only thing guaranteed present on
the target server, while the Linux builder should not need pwsh.
tests/test_bundle_lock.py runs both against the same fixtures and fails if they
disagree.
Every asset list shows a Type column (and printers a Model, machines and
network a Vendor), but the search filters only looked at the asset number,
name, serial and hostname. Searching a type returned zero rows: 'Part Washer'
on machines, 'Standard' on PCs, 'Thermal' on printers.
Extend the search on machines, computers, printers, network devices,
measuring tools and the unified asset list to cover the type name plus the
vendor/model where the list shows them. Joins are outer joins so an asset
missing a type or vendor still matches on its own fields; the core list uses
a correlated EXISTS instead, since its type-name filter already joins
AssetType.
The Inno printer installers hand-parsed JSON in Pascal (brittle brace-counting).
Add ?format=text to install-list (one printer per line, pipe-delimited:
printerid|windowsname|vendorname|modelnumber|hostname|ipaddress|mapx|mapy) and
to pc-default (printerid|windowsname), so the installer side is a split() with
no JSON parser. The web map keeps the default JSON.
Also resolve install-list's vendorname via the model (as the batch already does),
since the import sets the model, not the printer's direct vendorid - otherwise
the installers' HP/Xerox/Brother filter drops every prod printer.
Prod had 331 relationships, 268 computers, 204 machines, but the report came
back empty. The query only matched computer(source) -> machine(target), while
the import stores the general machinerelationships as machine(source) ->
PC(target) (only the synthetic measuring-tool links are PC -> tool). So the real
shop-floor edges never matched.
Make the query direction-agnostic (UNION of both orientations); a PC-runs-machine
report is conceptually undirected. Also drop the comtypeid=1 filter so the IP is
taken from the primary communication regardless of its type.
Test: a machine(source) -> PC(target) edge now appears in the report.
Two fixes for the printer install-batch on prod data:
1. Vendor was read only from the printer's direct vendorid, which the legacy
import never sets (it sets the model; legacy resolved vendor through the
model). Every prod printer came back vendor "unknown", so all fell into the
manual group and the universal PrinterInstaller.exe block never emitted. Now
resolve vendor via the model's vendor when the printer has no direct one, as
the classic installprinter.asp did.
2. Harden the download base URL. Behind IIS the app sees http on a loopback
port and url_root drops the /shopdb mount, giving a broken download URL when
site_base_url is unset. Fall back to https + the forwarded Host + script_root.
Test: a printer with no vendorid but an HP/Xerox model now groups universal.
Rebuilds the classic printer-installer feature: pick printers on the shopfloor
map, download a .bat that installs them.
Backend (asset_routes.py): GET /api/printers/install-batch?printerids=1,2,3
returns a .bat attachment. Groups printers the way the classic installprinter.asp
did - HP/Xerox via the universal PrinterInstaller.exe /PRINTER="a,b,c", printers
with a .exe installpath via that installer /SILENT, and anything else (no
installpath, or a .zip) listed for manual install instead of being run blindly.
Download URLs derive from the site_base_url setting + the IIS-served /installers
folder (no hardcoded host). Reuses the existing install-list query shape.
Frontend: PrinterInstallerMap.vue - full-screen Leaflet shopfloor map (reuses
mapConfig), a marker per network printer at its mapx/mapy, click to toggle-select,
sidebar with the selection + an Install button that downloads the batch. Toplevel
route /printer-installer, printersApi.installList(), and an Installer Map button
on the printers list.
Tests: install-batch grouping (universal/specific/manual) + requires-ids.
IIS ARR sets X-Forwarded-For to clientip:port, and the port changes every
connection. Left in, the audit log showed IP:PORT, the dashboard IP fallback
never matched a stored (portless) DashboardDefault.ipaddress, and login rate
limiting keyed per-connection instead of per-host. Add an IPv6-safe
clientip.client_ip / strip_port helper and use it in the audit log, the
dashboard resolver, and the login rate-limit key.
The KB list search matched only shortdescription + keywords, so searching a
topic (e.g. "Spotfire", the Application name) surfaced just the one article
whose title/keywords contained the word, not the others tied to it by topic.
Match the topic too via an appid IN (apps named like the term) subquery - used
instead of a join so it does not collide with the sort=topic join, and articles
with no app still match on title/keywords.
The lobby-display and screensaver slide manager was admin-only. Add a shared
slides.manage permission so a curator can manage both surfaces without full
admin. Admins keep access via the require_permission admin bypass.
Backend:
- plugins/slides/api/routes.py: all 5 management routes require slides.manage
- plugins/slides/plugin.py: declare it via get_permissions(); nav item carries
the permission so the frontend can gate visibility
- shopdb/core/api/auth.py: login response now returns the user's permissions
(matches /me) so the frontend authStore has them on fresh login
Frontend:
- stores/auth.js: hasPermission(name) getter (admin true, else granted list)
- router/index.js: guard supports requiresPermission
- views/AppLayout.vue: hide nav items whose permission the user lacks
- plugins/slides/frontend/routes.js: slide manager gated requiresPermission
Tests: no-perm user 403, curator role with the perm 200 (+ login advertises
it), admin 200 via bypass.
Deploy: run `flask seed permissions` to create the row, then grant it to a
role in Settings > Users & Roles.
Global search did a single ilike('%CSF Roles%'), so any query with more than one
word required the exact contiguous phrase and usually returned nothing. Add
_word_match: split the query into words and AND them (OR across the searched
columns per word), so 'CSF Roles' matches a record with both words in any field,
any order. Applied across every domain (assets, applications, KB, employees
[selfhosted + external HR], notifications, hostnames, IP, custom fields,
vendor/model/type). External HR path uses a parameterized per-word LIKE.
Adds GET /api/computers/display-kiosks - the displays that reported in (Kiosk
type), each with its derived FQDN (F<serial>.<domain>). The Dashboard Defaults
form gets a kiosk dropdown that fills the FQDN so admins pick a display instead
of typing an IP; IP stays an optional manual field. Table shows FQDN or IP.
'Business Unit' label -> 'Location' on this page + the settings nav.
A display's DHCP IP can change; its FQDN (F<serial>.<domain>, domain from the
display_fqdn_domain setting) is stable and the collector already reports the
serial. Add a nullable unique fqdn column (varchar191 so the index fits utf8mb4
without innodb_large_prefix), make ipaddress nullable, and require fqdn OR ip.
visitor-location + display-role resolve by FQDN first, then IP; create/update
accept fqdn. Core migration 7d31, verified up/down/idempotent on MySQL 5.6.
'Business unit' wording -> 'location' in the validation messages.
- allowlist auth uses remote_addr, not the spoofable first X-Forwarded-For hop
(adds _trusted_client_ip + a regression test); rate-limit path unchanged
- client psm1: fix Set-StrictMode crashes reading absent keys in Get-ShopdbConfig
(token-less mode) and Resolve-ShopdbPayloads (no-payload entries); validate
the manifest response is JSON before overwriting the last-known-good cache
- runner: pass the engine its required -InstallerRoot/-LogFile; create the log
directory so enforce logging is not silently lost on a fresh kiosk
- display scope: dispatcher writes an all-users Startup shortcut instead of
Start-Process (SYSTEM cannot show a window in session 0), resolves the base
URL from HKLM, and adds an always-on power/no-lock entry; tests updated for
the 6-entry scope
Fleet PCs on a trusted (vaulted) network can now reach the GE-Enforce client
endpoints (manifest, payload, report) without a per-PC token: the auth path
accepts a valid geenforce.fetch/report token OR a source IP in the configured
allowlist (setting geenforce_allowed_cidrs). Fail-closed; an empty allowlist
means the token stays the only path, so existing deployments are unchanged.
Rationale: the client token lives in HKLM on every kiosk, so it does not
defend against a compromised kiosk anyway - network-perimeter trust is the
same practical strength with far less provisioning + no token-rotation churn
on a DB wipe. Documented in-UI that this is perimeter trust, not per-device
identity.
- _ip_allowlisted() (ipaddress, X-Forwarded-For-aware via _client_ip)
- /geenforce/config GET/PUT extended with allowedcidrs, server-validated +
normalized (bad CIDR -> 400)
- new GE-Enforce > Settings tab (GeEnforceSettings.vue) to edit the allowlist
in admin, no SQL
- 3 regression tests (allow by IP, reject outside list, empty = token required)
A geenforce.fetch token can now be pinned to specific manifest scopes so a
fleet-wide key (a display's, delivered by DSC or baked into the image) is not a
skeleton key for the whole content store. NULL binding = unrestricted, so every
existing service token keeps working.
Core:
- ApiToken.resourcescopes column + resourcescopelist property (migration
7d30_apitoken_resourcescopes; NULL = unrestricted).
- apitokens API create/update accept + persist an optional resourcescopes list
(a resource-name allowlist; not permission-catalog names).
- New contract helper authorized_service_token(scope): same check as
service_token_authorized but returns the ApiToken so a plugin can read its
binding. Contract 0.14.0 -> 0.15.0; also export SupportTeam.
GE-Enforce enforcement:
- get_manifest: a bound token requesting a scope outside its allowlist -> 403.
- get_payload: a bound token may only pull a blob its own scope(s) reference
(service.blob_referenced_by_scopes); anything else -> 404 (no hash probing).
- Decorator stashes the authorized token on g for the route to read.
Also fixes a pre-existing contract-surface violation: the printers/printedparts
alert helpers imported shopdb.core.models / shopdb.extensions directly; now
via shopdb.api (SupportTeam newly exported). Docs: GE-ENFORCE-DISPLAY.md
provisioning note, PLUGIN-HOOKS.md, CLAUDE.md.
9 new resource-binding tests; full suite 1131 passing.
Per decision: displays need none of the fleet-wide common scope's software, so
the gea-shopfloor-display scope carries everything it enforces and does not
inherit common. This avoids repackaging common's SMB-backed payloads for a
share-less display.
- Invert the client common-merge switch: -NoCommon (default-on) becomes
-IncludeCommon (default OFF). A scope now enforces alone unless opted in.
The capability stays for a future share-less non-display PC; displays omit it.
- Drop the common SMB-payload audit + inheritance sections from the display
seed comments and docs (GE-ENFORCE-DISPLAY.md); document self-sufficiency.
- GE-ENFORCE-CLIENT.md: common-scope inheritance is now opt-in.
Get GE-Enforce closer to running on credential-less Intune/Entra display PCs
that pull manifest + payloads over HTTPS instead of SMB.
Server (plugins/geenforce/api/routes.py):
- Rate-limit + 512MB served-size ceiling on GET /payload/<sha256> (reuses the
login limiter's cache pattern, config-overridable via GEENFORCE_PAYLOAD_*).
- New tests: payload hardening, manifestblobs model-vs-migration parity, and a
report-contract test locking the lowercase per-entry report keys.
PS client (plugins/geenforce/client/):
- Fix New-ShopdbReport per-entry key casing to lowercase (name/action/selfhealed/
exitcode/message) to match what the server reads; the engine emits PascalCase.
- Enforce TLS 1.2 in the network functions.
- Fetch + merge the fleet-wide common scope alongside the pctype scope
(pctype wins on conflict; -NoCommon opt-out).
- Normalize whatever the engine returns into a well-formed summary.
- Make the empty-cache fail-safe observable: event-log entry + report ping
instead of a silent exit 0.
Manifest (plugins/geenforce/seed_display_scope.py + docs/GE-ENFORCE-DISPLAY.md):
- Seed a gea-shopfloor-display scope: 4 Edge kiosk drift-heal registry entries
+ 1 data-driven dispatcher (Dashboard/Lobby/3DPrintRoom via display-type.txt).
Kiosk EXEs stay image-baked; the manifest heals policy/config drift only.
- Documents the common SMB-payload audit (entries needing http/inline before a
share-less display can inherit common).
Migration registry (shopdb/plugins/alembic_template.py + test):
- Register the pre-existing manifestblobs and the new printersupplyalerts tables
in PLUGIN_TABLE_OWNERS; update EXPECTED_HEAD_REVISION for geenforce (0002blobs),
printers (0002supplyalerts), and printedparts (0004txnrev) which had drifted.
Poll Zabbix for toner levels on a schedule and email/webhook on a downward
crossing. Warning fires at or below the warning threshold (default 5%),
critical at the critical threshold (default 0%); both thresholds are settings.
State lives in printersupplyalerts so an alert fires once per crossing and
re-arms after a refill.
Recipients mirror the printedparts pattern: plugin-scoped shopdb users +
roles + free-text emails (falling back to the site alert_recipients), and a
chosen support team's webhook (falling back to the site alert_webhook_url).
- PrinterSupplyAlert model + migration printers0002supplyalerts
- alerttier(remaining, warning, critical) + check_supplies poller
- flask printers check-toner-alerts CLI (run via scheduled task/cron)
- printers alert settings + Low-Toner Alerts settings page
- 7 tests: tier boundaries, once-per-crossing + re-arm, toner-only scope,
custom thresholds, support-team webhook routing
Support teams gain a webhookurl (migration 7d29 + API + settings-page field), so
a team is a notification target. send_webhook(url=) lets a caller override the
site default with a team's webhook. Printedparts gains a 'alert support team'
setting (printedparts_alert_supportteamid) + selector on its settings page;
low-stock alerts post to that team's webhook, falling back to the site
alert_webhook_url. Email leg unchanged. Same pattern extends to other alerting
plugins (printers low-toner next).
send_webhook(title,text) posts alerts to an optional webhook (Teams Incoming
Webhook / Workflow, or generic JSON) via alert_webhook_url + alert_webhook_format
settings; send_alert fans out to it alongside email; exposed on shopdb.api
(0.13.0->0.14.0, PLUGIN-HOOKS synced); low-stock posts on its custom-recipient
path too. Also: recent-transactions table shows the consumed print-file revision.
Label switches from CODE128 to a QR encoding 'TAG|rev' (gage lab tag + latest
print-file revision), so a physical part carries which revision it was printed
from - short payload stays low-version + reliable at 0.5in (margin quiet zone,
EC M, no logo). Item exposes latestrevision; kiosk strips the |rev to resolve
and records the scanned revision on the take (migration 0004 adds
printeditemtransactions.revision) for traceability of which rev was consumed.
Manual entry records a null revision.
Lets share-less (Intune/local-account) PCs pull installers the manifest
references over HTTPS instead of SMB - the general capability the whole fleet
migrates toward. New ManifestBlob registry (migration 0002) with bytes on disk
at instance/geenforce/payloads/<sha256> (deduped by content); service.store_blob
+ blob_path; client-facing GET /api/geenforce/payload/<sha256> (geenforce.fetch
token, ETag=hash, serves the blob store or an inline DB payload by hash). The
serializer now emits PayloadSource/PayloadSha256/PayloadRef for http/inline
entries only (smb entries round-trip unchanged - parity green). CLI
'flask geenforce add-payload <file>' registers a blob and prints its sha256.
This is the shopdb half (B1); the PS client/engine fetch is B2.
One 'display' image resolves what it shows from its own IP, like the existing
visitor-location BU mapping. Extend DashboardDefault with displayrole
(dashboard|lobby|partskiosk; migration 7d28, businessunitid now nullable since
only the dashboard role needs one) + a role->path map. New unauthenticated
GET /api/dashboarddefaults/display-role returns {role, path, businessunitid}
for the caller IP. Settings UI gains a Display selector, showing the business
unit only for the dashboard role.
Role badges rendered gray for everything except admin, with no way to tell
roles apart. Add an optional color per role, matching how statuses and types
carry one: new roles.color column (migration 7d27_roles_color), color threaded
through the role API and the user serializer, and a ColorSwatchPicker in the
role editor. Badges use the role's color with contrast-aware text and fall
back to the old admin/gray classes when unset.
A lean site still gets every plugin's tables from the shared core Alembic
baseline. prune-schema drops the tables of plugins not installed on this
site, leaving core + chosen-plugin tables, with no edit to any released
migration (the relocate-into-plugin-baselines alternative would mean
rewriting ~15 released core migrations for a cosmetic gain - see ADR-014).
- shopdb/plugins/cli.py: prune-schema command. Dry-run by default; --yes to
execute; refuses non-empty tables without --force. Drops by table name (no
plugin import) so it works on a lean image. MySQL: private AUTOCOMMIT engine
(db.engine's pooled connections sit idle-in-transaction in a CLI context and
would deadlock the DROP on a metadata lock). SQLite: db.engine, restoring the
prior foreign_keys pragma so the StaticPool connection is not left changed.
- tests/test_plugin_prune_schema.py: drop-only-not-installed, full no-op,
refuse-non-empty, force-drops-non-empty.
- docs/DEPLOY.md: lean provisioning step after upgrade-all.
- ADR-014 ACCEPTED; index updated.
Verified on MySQL: full install then prune = no-op (86 tables); lean install
(machines+printers) then prune drops the other 19 plugin tables; second run
no-op. Full suite 1077 passed.
The lean-build endgame: a site ships carrying only the plugins it chose.
- scripts/build-site.sh: reads a site profile, resolves the hard-dependency
closure from manifests, builds the frontend with SITE_PLUGINS (stage-frontend
carries only those plugins), and stages a backend tree of core + only the
chosen plugin dirs. An unchosen plugin is in neither the bundle nor the tree.
- Core lazy-import guard: `flask seed demo` hard-imported the 5 asset subtype
models, which would crash a lean build missing any of those plugins. Now
guarded (a missing model skips its demo section).
- test_lean_build_guards.py: statically asserts NO core (shopdb/core, shopdb/cli)
import of a plugin is unguarded - a lean build omitting that plugin would
otherwise crash. 0 unguarded today.
Pilot verified: a lean build (machines + printers) carries only machines +
printers code - PartsKiosk / ManifestEditor / USBLabelBatch / KnowledgeBaseDetail
/ EmployeeDirectory are absent from the bundle, and only machines/printers plugin
dirs stage into the backend. (Sidebar labels for absent plugins remain - the
accepted small plugin-aware core remainder.) Guard test + naming green.
Wires the ADR-010 get_map_overlays hook into the floor map so a plugin decorates
markers as JSON, no map code. ShopFloorMap fetches /api/pluginui/map-overlays,
then each overlay's endpoint (per-asset [{assetid, color, label}]), joins by
assetid, and draws a ring or badge circleMarker on matching markers plus a
legend entry - all as extra Leaflet layers cleared and redrawn with the markers.
Aligned the measuringtools calibration overlay endpoint to the documented
contract: it now returns {assetid, color, label} (was {calibrationstatus,
statuscolor}) and only decorates due/overdue tools.
Additive + guarded (assetid null check, per-endpoint try/catch, cleanup on
re-render), so the map degrades to no decorations on any failure. Verified: the
overlay endpoint serves the contract shape, the map renders without error, and
the frontend builds. A populated badge needs a site that actually places
measuring tools on its map (this dataset places none). 38 measuringtools/pluginui
tests, 58 vitest, build + naming green.
Global-search rows built the plugin detail URL from a hardcoded url_map of
plugin routes in core. Now core prefers a plugin's declared
get_asset_presentation route (ADR-010), substituting the core assetid via the
plugin's by-asset resolver; types that have not declared fall back to the legacy
id-keyed map, so nothing breaks. Measuring tools (which declare the route) link
through it now; machines/PCs/printers/network migrate off the hardcode as they
add a by-asset route + declaration. Presentation map is collected once per
search (cached on flask.g). 2 consumer tests; 26 search tests green.
Wires the ADR-010 get_asset_panels hook to a generic frontend renderer so a
plugin adds detail-page UI as JSON, no Vue. This is the Path A foundation that
lets simple plugins ship UI without a frontend build.
- components/PluginAssetPanels.vue + pluginAssetPanels.js: fetches
/api/pluginui/asset-panels for an asset, then each panel's data endpoint, and
renders by mode: list (title + status badge + meta lines via a field map),
keyvalue, table (declared or inferred columns), badge. Pure mapping logic is
in the .js module and unit tested (9 specs), same pattern as entryForm.js.
- New 'list' render mode with a declarative field map (title/badge/meta),
documented on the hook in base.py.
- Warranty migrated to it: get_asset_panels now declares a 'list' panel + map
that reproduces WarrantyPanel's output (vendor title, status badge with color
+ label map, servicelevel/ends/tag meta, manage link) with zero
warranty-specific frontend code.
- MachineDetail swapped from <WarrantyPanel> to <PluginAssetPanels> (pilot); the
hero warranty badge is unchanged. Verified end to end: the API serves the list
panel + map and the warranty rows; the page renders without error.
Rollout of the other 4 detail pages (PCDetail, PrinterDetail, NetworkDeviceDetail,
MeasuringToolDetail) and the map-overlays / asset-presentation renderers are
follow-up Phase 3 commits. 58 vitest, build clean, 1067 backend pass, naming green.
Fourth review found the last import-path bypass: the is_dir() branch returned
None for a name whose dir has no __init__.py, without checking a same-name
sibling file. FileFinder loads a file over an init-less namespace dir, so an
attacker could overwrite a signed foo.py with malicious bytes, mkdir an empty
foo/ next to it (PROVENANCE untouched, still verifies), and any import of that
name ran the unverified foo.py - RCE with only plugins/ write access.
Fix: the dir-with-no-__init__.py branch no longer returns early; it falls
through to the leaf .py hash gate and the non-source refuse check. Invariant:
find_spec returns None for a plugins.* name ONLY where FileFinder would also
find nothing on the same __path__.
Everything else was confirmed sound this round: the owned plugins root, exec of
exact verified bytes (never .pyc/.so), the extension/bytecode refusal, plugin.py
read-once, the provenance signature gate, dev-exemption scoping, and #3/#4.
Symlink, suffix-ordering, cache-lifecycle, and loader-internal angles cleared.
2 regression tests (tampered .py + sibling dir; unsigned .py + sibling dir). All
13 bundled plugins still load under enforcement; 1067 pass, naming green.
Third review found the meta_path guard leaked exactly where it delegated to the
stdlib import system:
1. Non-.py submodules (CRITICAL). When a name had no dir and no .py, find_spec
returned None and the stdlib loaded a planted .so (ExtensionFileLoader) or a
sourceless .pyc unverified - an attacker deletes a signed .py and drops a
same-named .so with arbitrary init code, run on a normal request via core's
`from plugins.<name>.models import ...`. The guard now refuses any name for
which a non-source importable candidate (EXTENSION_SUFFIXES + BYTECODE_
SUFFIXES) exists on disk; None is reserved for genuinely-absent modules.
2. Top-level plugins/__init__.py (CRITICAL). It is in no plugin's provenance,
is attacker-writable, and Python runs it before any guarded submodule. The
guard now owns `plugins`: it execs an EMPTY package body (search points at
the plugins dir), so an overwritten plugins/__init__.py never runs.
Also: specs are built with spec_from_file_location so loaded modules get
__file__/__path__ (Flask blueprint root paths need it) while the loader still
execs the verified in-memory bytes - never re-reading the file.
Verified end to end: under PLUGIN_REQUIRE_SIGNED with all 13 bundled plugins
stamped, the app boots and loads every plugin through the guard; a tampered
plugin file is refused at load. 4 new guard tests (planted .so, sourceless
.pyc, absent-module defer, neutralized package root). Prior fixes #3/#4
confirmed still sound by the review. 1065 pass, naming green.
A re-review showed the previous "single import choke point" claim was wrong:
`plugins` is a normal importable package, so core request handlers that do
`from plugins.<name>.models import ...` never passed through the loader and ran
unverified - an attacker who dropped a file into plugins/<name>/ got arbitrary
in-process code execution on an ordinary HTTP request (and a planted .pyc ran
from cache). Gating load_plugin_class covered only plugin.py, one path of many.
Fix: importguard.py installs a sys.meta_path finder (under enforcement) that
intercepts EVERY plugins.<name>.* import, verifies the plugin's signed
provenance once, then verifies each module file against it and execs the exact
bytes it hashed - read once, compiled, exec'd, never a .pyc, never a re-opened
file. This closes the submodule bypass and the planted-bytecode read, and the
read-once exec closes the verify-vs-exec TOCTOU on the import path. The import
system, not one method, is the real choke point.
- init_app installs the guard when PLUGIN_REQUIRE_SIGNED, clears it otherwise.
- load_plugin_class now verifies plugin.py from a single read and execs that
buffer (finding #3 on that file); its submodule imports flow through the guard.
- docs: stamp-bundled must cover every plugin dir present (a disabled plugin's
module can be imported by core); recommend a read-only plugins/ owned by the
deploy user as defense in depth (closes the residual migrate-time race an
attacker with concurrent write could otherwise attempt).
Earlier review's fixes#3 (migrate code paths) and #4 (shelf content binding)
were confirmed sound and are unchanged. 7 import-guard tests (submodule verify,
tamper, unsigned refused, planted .pyc ignored, real import through the guard,
install/uninstall). 1061 pass, naming green.
An adversarial security review of the Phase 2 trust model found four real
bypasses (two remote-triggerable to in-process code execution). Root cause for
three: the set of bytes verification covered was smaller than the set that
determined execution. Fixes:
1. Bytecode-cache blind spot (CRITICAL). verify_dir excluded __pycache__/.pyc,
so a planted cache ran while escaping the hash map. verify_dir now flags any
bytecode as an unexpected file; the loader strips bytecode before verify and
imports under sys.dont_write_bytecode, so only verified source executes.
2. Unauthenticated verify-at-load bypass (CRITICAL). load_plugin_class imported
plugin.py with no gate, reachable via discover_available / an anonymous GET
/api/plugins. The verify+strip gate moved INTO load_plugin_class - the single
import choke point every path flows through - so an unsigned/tampered plugin
is never imported. discover_available skips a refused plugin instead of 500.
3. Ungated migration entrypoints (HIGH). downgrade_plugin and get_current_head
(ScriptDirectory imports version modules) ran plugin code with no check. All
alembic-invoking methods now pass through _verify_ok (strip + verify) first
and run under no-bytecode.
4. Revocation/content bypass (HIGH). The signed index bound a filename, not
content; adopt did not bind the delivered bytes to the resolved version, so
revoked bytes could be served under a live filename. The index now records a
per-artifact SHA-256; adopt verifies the on-disk digest and requires the
artifact's own signed manifest version to equal the resolved version.
Enforcement stays default-off; strip/no-bytecode run only under enforcement, so
the unsigned path is unchanged. 6 regression tests (planted bytecode, the
discover import path, downgrade gate, version-swap). 1054 pass, naming green.
Completes the marketplace security model. Verification stops being advisory:
a plugin only loads or migrates when its tree matches a trusted signature, and
plugins are pulled from a signed shelf with anti-rollback and revocation.
Enforcement (default OFF - existing deploys unchanged):
- verification.py PluginVerifier, shared by the loader (verify-at-load, before
plugin.py is imported) and the migration manager (verify-at-migrate, before
any DDL). Fail-closed: an unsigned/tampered/wrong-key plugin does not run.
- Gated by PLUGIN_REQUIRE_SIGNED. PLUGIN_DEV_TRUST_DIRS exempts named dirs but
only under DEBUG/TESTING; production ignores it.
- flask plugin stamp-bundled writes provenance into in-tree plugins so
verify-at-load applies to bundled plugins too (image build step).
- tier:core manifest guard: uninstall/disable refuse a core-tier plugin.
Shelf (shelf.py):
- Signed shelf-index.json (+ .sig): monotonic serial (a site refuses an older
index - anti-rollback), revoked list carried across builds, per-entry
version/tier/core_version for browse. Index is a browse layer only; adopt
reads security-bearing fields from the verified artifact.
- flask plugin shelf-build / shelf-list / adopt / audit. adopt verifies index +
artifact (signature + every file hash), unpacks to staging, re-verifies, then
atomically moves into place and installs+enables the closure. Refuses a
downgrade without --force-downgrade. Anti-rollback serial stored in
instance/shelf-state.json.
- config PLUGIN_SHELF_DIR; the app only reads the folder, never speaks a
network. .env.example + docs/PLUGIN-SIGNING.md document the flow.
22 tests: verifier policy (off / no-keys / signed / tampered / wrong-key /
dev-exempt), verify-at-load + verify-at-migrate integration, tier guard, index
sign/verify + tamper/wrong-key, serial state, revocation, version resolution,
verified atomic unpack + tamper refusal. Live-smoked keygen->pack->shelf-build
->list->adopt->audit + serial guard. 1050 pass, naming green.
Packaging + provenance for the plugin marketplace. No runtime behavior change
yet - verification is available on demand; enforcing it at plugin load/migrate
and pulling from a shelf are Phase 2.
- signing.py: ed25519 key pairs + provenance. Provenance is a sorted per-file
SHA-256 map plus metadata; the detached signature covers the exact
serialized provenance bytes, so verifying is re-hash files, re-serialize,
check signature. verify() accepts any of several trusted keys (rotation).
Uses cryptography (already a dependency).
- packaging.py: pack() builds a signed <name>-<version>.shopdbplugin (zip +
PROVENANCE.json + PROVENANCE.sig). verify_artifact()/verify_dir() re-hash
and check the signature, and flag a tampered file, an unexpected file, a
wrong/absent key - all fail closed.
- CLI: `flask plugin keygen` (publisher key pair), `flask plugin pack <name>
--key` (validates then signs), and `flask plugin validate` extended to a
signed artifact by path (--pubkey, else PLUGIN_TRUSTED_KEYS).
- config PLUGIN_TRUSTED_KEYS: os.pathsep-separated public-key PEM paths,
delivered with the site config, never read from the shelf. .env.example
documents it.
- docs/PLUGIN-SIGNING.md: curator flow (keygen offline, review, pack, publish,
pin keys, rotate).
The signature proves an artifact is exactly what a curator signed, not that the
code is safe - human review before signing is the control. 11 tests: sign/verify
round trip, wrong key, provenance excludes noise, serialize determinism, pack +
verify, tamper -> hash mismatch, extra file, no-key fail-closed, verify_dir.
1028 pass, naming green.
Additive, zero-risk-to-running-sites prep for the plugin catalog. No
distribution or lean-build behavior yet; fixes latent bugs and adds the
declarative + validate tooling later phases build on.
Fixes:
- upgrade_all_plugins iterates registry.get_all(); only adopted plugins are
migrated. Removes the phantom hasattr(registry, 'list_installed') probe
that always fell through to migrating every folder on disk (unadopted DDL
ran with full DB rights on every deploy).
- Reverse-dependency checks on uninstall/disable read dependencies from the
manifest on disk via _installed_dependents, so an installed-but-unloaded or
disabled dependent is counted. Uninstall blocks on any installed dependent;
disable blocks on an enabled dependent.
- _sort_by_dependencies detects a dependency cycle (back edge in the DFS) and
raises PluginDependencyError instead of looping or dropping a plugin.
New:
- flask plugin validate <name>: manifest loads + name match, manifest-schema
check, core_version admits the framework contract, declared dependencies
exist on disk. No new dependency (lightweight checker); schema ships in the
package at shopdb/plugins/manifest_schema.json (docs/ is stripped on
publish). The check caught that provides is an object, not an array.
- flask plugin apply-profile <file>: declarative install AND enable of a
chosen plugin set plus its hard-dependency closure, in dependency order,
idempotent. Replaces the hand-ordered runbook sequences that could enable a
plugin that was never installed. deploy/site-profile.example.json template.
- Dockerfile header corrected (all 13 catalog plugins, not "eleven core").
10 new lifecycle tests (reverse-deps from disk, cycle detection, upgrade-all
scope, profile closure, schema, all 13 manifests match schema). 1018 pass,
naming green.
The gage lab assigns real WJRP asset numbers, so identity splits: the
internal itemcode stays auto-minted and a new optional unique
gagelabtag (migration 0003) carries the lab's number - settable on
create/edit, searchable, and resolved by the kiosk for scans and bare
keypad digits against the numeric tail of either identifier
(unique-match only). The print-files table becomes stacked revision
cards - filename with rev/current badges, one meta line, delete pinned
right - ending the horizontal scroll in that column.
Photos already resolved through the directory at read time, but names
only came from the stored employeename column - empty after a
shopdb-only import, so recertification/recognition cards showed bare
SSOs. New resolve_employee_display_name in the employees plugin
(mode-aware: self-hosted table or external HR) backs a fallback in
both the single-card and split-per-employee paths; stored names still
win when present.
Deleting any user who owned an API token or appeared in the audit log
hit the users FK and 500ed - the import's 'importer' account being the
guaranteed case (its PAT plus every audit row the import wrote).
Tokens are revoked outright; audit history is kept but detached
(userid NULL), so the trail survives the account.
Browsing the catalog (item list, detail, file listings) now requires
authentication plus the view permission, and the /printedparts pages
and the label print page require login. Still deliberately open: the
kiosk endpoints per the decision record, the image serve and file
download (img tags and anchor downloads cannot carry a JWT), and the
reports (product-wide jwt-optional convention). Grant
printedparts.view to the roles that should see the catalog.
printeditemfiles lands as the plugin's first incremental migration
(0002 on the plugin chain - the ADR-008 payoff). Revisions are
append-only per item: upload assigns the next number, records the
uploader from the JWT, enforces an extension allowlist and a 100 MB
cap; download serves the original filename; a permission-gated delete
covers wrong-file mistakes. The detail page gains the revision table
with a current badge. Unique storedfilename is sized 191 so the index
fits MySQL's 767-byte prefix - the per-plugin chain does not apply the
core env's ROW_FORMAT hook.
Alert recipients gain roles: Role joins the 0.13.0 surface, a role
picker on the settings page, and every active member of the selected
roles is folded into the deduped recipient list.
Retire button with confirmation on the detail page (item leaves the
storefront and the kiosk rejects its code; ledger history and label
survive), Restore on retired items, and an Include-retired list toggle
with a badge. Restore is its own permission-gated POST - the generic
update still cannot flip isactive. New codes mint as WJRP0042 style
without the dash; existing codes are immutable bin labels and keep
their form.
Contract 0.13.0 puts the User model on the plugin surface. The
settings page gains a checkbox picker over the user list; selected
users receive low-stock alerts at their account email, merged and
deduped with the free-text address list, inactive accounts skipped,
site alert_recipients still the fallback when both are empty.
Contract 0.12.0: send_email/send_alert join the plugin surface (the
mailer was core-only), PLUGIN-HOOKS and status docs updated, manifest
pins the new floor. The alert fires inside _ledger_write only when a
decrement CROSSES the item's threshold - one alert per depletion,
rearmed by restocking above - and is best-effort after the commit so
mail trouble can never fail a take. Recipients come from
printedparts_alert_email, falling back to the site alert_recipients.
on_enable re-seeds settings idempotently so existing installs pick up
new keys. Crossing/rearm semantics proven by test.
The lab is now a build-along mirroring what actually happened: ten
stages, each with the goal, the divergences, a see-it-work check, and
the errors genuinely hit while building (empty Migration error from a
broken model import, the migration-guard KeyError, the missing Lucide
icon, nested-app-context test writes, Decimal sums, and the authz
sweep catching the deliberately open kiosk take). That last one gets
its explicit EXEMPT_ENDPOINTS entry with a pointer to the decision
record - the net stays, the exception is reviewable. Full suite: 993
backend tests, 49 vitest, frontend build, naming hook, all green.
