ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Build the plugin set a real site runs, and write down what shipped

Browse Source 
The lean-build job proved the delete-a-plugin guarantee with two plugins and
four hand-picked marker components. It now builds a real twelve-plugin set, and
the assertions follow a rule instead of a hunch: one component from EVERY
omitted plugin must be absent, one from EVERY chosen plugin must be present, and
the backend tree is checked both ways. An omitted set that grows without its
check growing is how a leak ships.

CHANGELOG had not been touched across the whole range, so the largest
security-behaviour change in the product - per-PC endpoint credentials, the
shared-key refusal for enrolled PCs, server pinning, agent 1.20.1 and tray 1.7.1
- had no release record. An [Unreleased] section now carries it, written from
the commits rather than from memory.

CLAUDE.md and the agent quick-reference stated that ADR-016 bans storing
credentials in the registry. It does not, and this fleet depends on HKLM for its
collector key and server pins: the next person following that text removes them
and silently breaks reporting and pinning on every un-enrolled PC. Corrected,
along with two other statements that were false on this branch - the database is
MySQL, not PostgreSQL, and column names are lowercase concatenated, not
snake_case, which is the opposite of what the gate enforces.

The wiki drafts are updated against the code as it stands, each old finding
given a verdict with a file and line rather than left as an assertion from a
week ago, and a new page records the ADR conformance result - including that the
ADR set itself is not on this branch, so nothing here enforces it except the
naming and style script.

They are also scrubbed of the things that must not reach a published wiki: the
site name, the internal git server, developer home paths and the assistant
tooling. They now pass the repository's own publishability patterns, which is
what the scrub is for.
This commit is contained in:
cproudlock
2026-09-18 13:31:25 -04:00
parent 8fc6df8f0a
commit 69a7997dc4
10 changed files with 1946 additions and 746 deletions
Download Patch File Download Diff File Expand all files Collapse all files

155
.claude/QUICK-REFERENCE.md
View File

@@ -11,29 +11,58 @@ Fast lookup for architectural decisions and contracts.
- **ADR-008**: Plugin migration ownership - PLUGIN_TABLE_OWNERS, migration chains
- **ADR-013**: Plugin catalog - Lean builds, plugin manifest
- **ADR-015**: Site-specific configuration - Where site literals belong
- **ADR-016**: Credential delivery - Device tokens, kiosk auth
- **ADR-016**: Credential delivery to the fleet - cmdkey credentials for bays,
targeting and rotation. ACCEPTED but NOT built. It says nothing about device
tokens or kiosk auth, and nothing forbids the registry: it names registry
provisioning as a recommended first step
## Import Surface (stable API)
Every name below is in `shopdb/api/__init__.py` `__all__`. That list is the
surface; a name that is not in it is not importable from `shopdb.api`, whatever
else in the tree defines it.
```python
from shopdb.api import (
# Database
db,
# Core Models
Asset, Employee, User, Building, MapLevel,
# Plugin Base
BasePlugin,
# Auth/Authz
requires_role, requires_device_token,
# Core models
Asset, AssetType, AssetStatus, AssetRelationship,
Location, BusinessUnit, Model, OperatingSystem, Vendor,
User, Role, Setting,
# Auth / authz
require_permission, require_role, require_any_permission,
caller_holds, caller_is_role,
client_ip, ip_in_cidrs, request_ip_allowlisted,
# Responses and paging
success_response, error_response, ErrorCodes,
get_pagination_params, paginate_query, paginated_response,
# Utilities
badges, datevalues, queryargs
audit_log, cache,
normalise_badge, badge_kind, BadgeShapeError,
parse_date, parse_date_fields, DateValueError,
intarg, intargs, QueryArgError,
resolve_asset_position,
)
```
NOT on this surface. This block used to name all six, so the first import a new
agent copied raised ImportError:
- `BasePlugin`, `PluginMeta` - `from shopdb.plugins.base import ...`, which is
what every `plugins/*/plugin.py` in the tree does.
- `Building`, `MapLevel` - `shopdb.core.models.maplevel`. An asset's position
normally comes back through `resolve_asset_position`, which IS exported.
- `Employee` - no such model. The directory lives in a read-only external
database reached through `employee_connection`, and the plugin's own row type
is `DirectoryEmployee` in `plugins/employees/models/`.
- `badges`, `datevalues`, `queryargs` - module names under `shopdb/utils/`, not
exported. Import the functions listed above instead.
## Plugin Structure
```
@@ -55,9 +84,12 @@ plugins/{name}/
## Database Naming
- **Tables**: `printedpart`, `mappeddrives` (lowercase concatenated)
- **Columns**: `serial_number`, `last_seen` (snake_case)
- **Enforced**: `scripts/check-naming-and-style.sh`
- **Tables**: `machines`, `mappeddrives` (lowercase concatenated, plural)
- **Columns**: `machineid`, `lastzabbixsync`, `isactive` (lowercase concatenated,
singular). NOT snake_case
- **Indexes**: `idx_machines_locationid`. The only place an underscore is allowed
- **Enforced**: `scripts/check-naming-and-style.sh`, on model attributes AND on the
column names a migration creates
## Branch Model
@@ -132,19 +164,98 @@ PLUGIN_TABLE_OWNERS = {
}
```
## Device Tokens (ADR-016)
## Kiosk Access
Kiosks require device authentication:
Not ADR-016 - that ADR is about cmdkey credentials for bays and is not built.
A kiosk route admits a caller on EITHER ground, both gated on the kiosk's own
enable flag:
This is `plugins/usb/api/routes.py` as it stands, not a paraphrase:
```python
from shopdb.api import requires_device_token
from shopdb.api import Setting, client_ip, ip_in_cidrs
@bp.route('/kiosk/endpoint')
@requires_device_token(['usb_kiosk', 'parts_kiosk'])
def kiosk_view():
return render_template('kiosk.html')
KIOSK_NEVER_A_CLIENT = '127.0.0.0/8, ::1/128'
KIOSK_TRUSTED_PROXIES = 'usb_kiosk_trusted_proxies'
def _kiosk_client_ip():
return client_ip(request)
def _kiosk_address_is_infrastructure(address):
if ip_in_cidrs(address, KIOSK_NEVER_A_CLIENT):
return True
return ip_in_cidrs(address, Setting.get(KIOSK_TRUSTED_PROXIES))
def _kiosk_ip_allowlisted():
address = _kiosk_client_ip()
if _kiosk_address_is_infrastructure(address):
return False
return ip_in_cidrs(address, Setting.get('usb_kiosk_allowed_cidrs'))
def _kiosk_open():
if not _kiosk_enabled(): # the site has to turn it on first
return False
return _kiosk_token_presented() or _kiosk_ip_allowlisted()
```
THIS IS NOT `request_ip_allowlisted(request, cidrs)`. That helper is exactly
`ip_in_cidrs(client_ip(request_obj), rawcidrs)` and nothing else
(`shopdb/utils/ipallowlist.py`), so it has no infrastructure refusal. The USB
kiosk refuses an address that belongs to the path rather than to a caller BEFORE
it consults the allowlist, and that refusal is the whole point: see the topology
below. Writing the helper here instead of the real shape is how the two got
confused.
- The allowlists are per kiosk (`usb_kiosk_allowed_cidrs`,
`printedparts_kiosk_allowed_cidrs`) and ship EMPTY, so a site that enters no
ranges stays token-only.
- `ip_in_cidrs` fails closed on an empty setting, an unparseable caller address
and a malformed entry. Never copy a CIDR parser into a plugin.
- A refused caller gets 404, not 403: these routes do not advertise themselves.
- The two kiosks HAVE DIVERGED. `plugins/printedparts/api/routes.py:556` still
calls `request_ip_allowlisted(request, Setting.get('printedparts_kiosk_allowed_cidrs'))`
with no infrastructure check, so a site that allowlists loopback to work
around the topology below opens that kiosk to every caller. Read the plugin
you are editing; do not assume the other one matches.
### Why the infrastructure refusal exists (the shipped Windows path)
Trace it, do not guess. IIS hands the request to HttpPlatformHandler, which
connects to waitress on 127.0.0.1 with
`--trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for
--trusted-proxy-count=1` (`deploy/windows/web.config`). waitress, seeing the
trusted peer, overwrites `environ['REMOTE_ADDR']` from the LAST X-Forwarded-For
hop. `client_ip()` then returns `strip_port(request.remote_addr)`
(`shopdb/utils/clientip.py`) - it never reads the header itself.
The catch: IIS does not set X-Forwarded-For on its own, and the rewrite rule
that would is commented out in `web.config` - it SHIPS DISABLED because it needs
the URL Rewrite module. So on a default install there is no header, waitress has
nothing to promote, and every caller arrives as 127.0.0.1. `TRUSTED_PROXY_COUNT`
defaults to 0 (`shopdb/config.py`), so ProxyFix is not installed either, and it
would not help: waitress already did that job.
Consequences a kiosk change has to respect:
- Until someone enables the rule (`shopdb-install.ps1 -ClientIpSource direct`
installs URL Rewrite, allows `HTTP_X_FORWARDED_FOR` at server level and
uncomments it), a CIDR allowlist admits nobody and the kiosk is token-only.
That is the fail-closed direction and it is correct.
- Allowlisting 127.0.0.0/8 to "fix" that would admit everyone, which is why
`KIOSK_NEVER_A_CLIENT` refuses it whatever the setting says.
- `-ClientIpSource proxy` leaves the rule off ON PURPOSE: behind ARR or a load
balancer, REMOTE_ADDR is the proxy and the rule would destroy the real client
address. There the upstream proxy sets the header.
## Fleet Credentials
- Agent and GE-Enforce reporters run as SYSTEM, so there is nobody to prompt.
The collector key, base URL and server pins come from
`HKLM\SOFTWARE\GE\ShopDB`. That hive is the design, not a violation.
- An enrolled PC's own per-PC credential is DPAPI-protected in a SYSTEM-only
directory and sent as `X-Endpoint-Credential`; the shared fleet key still
works for a PC that has not enrolled and is refused for one that has.
## Wiki Location
Full docs: `~/Documents/github/shopdb-flask.wiki/`

44
.github/workflows/ci.yml vendored
View File

@@ -89,8 +89,19 @@ jobs:
lean-build:
# ADR-013 Phase 5: prove a per-site build carries only its chosen plugins.
# Builds a lean site (machines + printers) and asserts an omitted plugin's
# code is absent from the bundle - the delete-a-plugin guarantee in CI.
# The profile is the WJ plugin set, so this job proves the build this branch
# actually ships, not a two-plugin toy. The omission guarantee is what is
# under test, so the assertions below name a component from EVERY omitted
# plugin: a set that grows without its check growing is how a leak ships.
#
# Chosen (12): backups computers endpoints guides knowledgebase machines
# network notifications printers slides tools warranty
# Omitted (5): employees geenforce measuringtools printedparts usb
# 'applications' is manifest-less core and ships in every build (ADR-013).
# endpoints hard-depends on computers, which is already chosen, so the
# closure adds nothing - if that changes, resolve_plugin_closure.py pulls
# the dependency in and the absence assertions below start failing, which is
# the signal to update this list rather than to loosen the check.
runs-on: arc-runner-set
steps:
- uses: actions/checkout@v5
@@ -100,21 +111,28 @@ jobs:
cache: npm
cache-dependency-path: frontend/package-lock.json
- run: cd frontend && npm ci
- name: Build lean site (machines + printers)
- name: Build lean site (WJ plugin set)
run: |
printf '{ "site": "ci-lean", "plugins": ["machines", "printers"] }' \
printf '%s' '{ "site": "ci-lean", "plugins": ["backups", "computers", "endpoints", "guides", "knowledgebase", "machines", "network", "notifications", "printers", "slides", "tools", "warranty"] }' \
> /tmp/lean-profile.json
bash scripts/build-site.sh /tmp/lean-profile.json /tmp/leansite
- name: Assert omitted plugin code is absent, chosen present
run: |
assets=/tmp/leansite/frontend-dist/assets
for code in PartsKiosk ManifestEditor USBLabelBatch KnowledgeBaseDetail; do
# One component per OMITTED plugin: employees, geenforce,
# measuringtools, printedparts, usb.
for code in EmployeeDirectory ManifestEditor MeasuringToolsList PartsKiosk USBLabelBatch; do
if grep -rqoh "$code" "$assets"/*.js; then
echo "FAIL: omitted-plugin code '$code' leaked into the lean bundle"
exit 1
fi
done
for code in MachineDetail PrinterDetail; do
# One component per CHOSEN plugin, so a plugin silently dropped from
# the bundle fails here instead of on the floor.
for code in BackupHistory PCsList EndpointsList GuidesList \
KnowledgeBaseDetail MachineDetail NetworkDevicesList \
NotificationsList PrinterDetail SlideManager ToolsIndex \
WarrantiesList; do
grep -rqoh "$code" "$assets"/*.js || {
echo "FAIL: chosen-plugin code '$code' missing from the lean bundle"
exit 1; }
@@ -124,8 +142,18 @@ jobs:
grep -rqoh "ApplicationsList" "$assets"/*.js || {
echo "FAIL: core page 'ApplicationsList' missing from the lean bundle"
exit 1; }
test -d /tmp/leansite/plugins/machines
test ! -d /tmp/leansite/plugins/printedparts
for plugin in backups computers endpoints guides knowledgebase \
machines network notifications printers slides tools \
warranty; do
test -d "/tmp/leansite/plugins/$plugin" || {
echo "FAIL: chosen plugin '$plugin' missing from the backend tree"
exit 1; }
done
for plugin in employees geenforce measuringtools printedparts usb; do
test ! -d "/tmp/leansite/plugins/$plugin" || {
echo "FAIL: omitted plugin '$plugin' staged into the backend tree"
exit 1; }
done
echo "lean build verified: only chosen plugins present"
# NOTE: the MySQL-8 migration/seed job (fresh `flask db upgrade` + every

181
CHANGELOG.md
View File

@@ -702,6 +702,187 @@ ADR-007 and ADR-002.
`plugins/tools/frontend/labelPages.js`, where the column contract and the
two-sided page order are covered by tests.
### Added
- **Every PC gets its own endpoint credential, and enrolls to get it.** The
fleet authenticated with one shared collector key: every bay held the same
secret, one of them leaking meant reissuing all of them, and a PC could post
a check-in naming any asset it liked. A PC now enrolls once and is
authenticated as itself thereafter. `POST /api/endpoints/enroll` issues a
credential to a PC that matches exactly one computer on hostname and BIOS
serial and holds none yet; anything else - unknown hostname, a duplicate, a
serial that disagrees, a PC already enrolled - is recorded as a request an
administrator approves or rejects, and the agent's persisted nonce is what
binds an approved request back to the PC that asked. Enroll never refuses, so
a PC installed with no key at all still turns up in the queue instead of
vanishing. One auth helper serves every agent route: with
`X-Endpoint-Credential` the asset comes from the credential, so a request
naming a different PC is refused with `ASSET_MISMATCH` rather than believed,
and a revoked credential answers `CREDENTIAL_INVALID` so the agent
re-enrolls rather than going quiet. The core collector takes the credential
for the computers, printers and endpoints collectors through two hooks on
`shopdb.api` (contract 0.33.0).
Settings > Endpoints > Enrollment is where this is run. The Migration tab
shows readiness counts, each PC's state with the reasons it is blocked,
"Upgrade selected" and "Upgrade next 25", and live wave progress; a wave
packages the agent once and queues the existing per-PC upgrade. The
Credentials tab lists enrolled PCs with revoke, pending requests with
approve and reject, and the two switches. The fleet list gained a Credential
column and filter, and a PC's detail page shows its credential state, with
Revoke and Allow re-enrollment for an administrator. Migration 0023 creates
`endpointcredentials`, `endpointenrollrequests` and
`endpointmigrationwaves`.
- **The agent pins the server it talks to, and the registry owns the pins.**
An air-gapped site runs its own certificate, so "trusted chain" says very
little; the agent now checks the server's SubjectPublicKeyInfo SHA-256
against a pinned list before it will send a credential anywhere, and a
credential is only ever sent back to the server that issued it. Pins live in
`HKLM\SOFTWARE\GE\ShopDB` under `ServerPins`, with `appsettings.json` as the
fallback. The registry is AUTHORITATIVE whenever that value exists, even
when it is empty: the two used to be merged, so removing a pin from HKLM
never took effect anywhere. Two pins are carried at once, current and next,
so a certificate can be rotated without a flag day.
`Set-ShopDBServerPins.ps1` is the GE-Enforce entry point (`-Clear` deletes
the value, `-None` writes an empty one) and `Get-ShopDBServerPin.ps1` reads
what a PC has. `GET /settings/server-cert` returns the pin to put in that
entry, the Server Certificate page shows it, and the page also compares the
certificate the site actually presents at its public address with the
certificate file, warning when the two disagree.
- **An agent on a one-off PC from a single command.** A PC outside GE-Enforce -
a one-off, a loaner, something rebuilt in a hurry - had no path to an agent
short of doing the whole managed install by hand. An administrator now
generates a short-lived token in Settings > Endpoints > Endpoints and hands a
technician one elevated PowerShell line. The script refuses to run
unelevated, refuses a PC that already has an agent (a managed bay gets its
agent from GE-Enforce), checks both binaries against hashes the server baked
into the package, and installs with NO shared key - the PC then asks to
enroll and the script watches locally for two minutes for an administrator to
approve it. The token routes answer 404 to anything wrong rather than
confirming a token exists, are rate limited per address and are audited with
the caller, and a bootstrap package can never carry a collector key.
Migration `0024_endpoint_bootstrap_tokens` adds the token table, which stores
a hash, an expiry and a use count.
- **A kiosk admits a caller by where it is on the network, not only by its
token.** The device token assumes a handful of fixed screens provisioned
once, and that is not how these kiosks are used: most people open the page on
their own PC, there is no login, and per-browser provisioning would mean
touching every PC on the floor and losing it again at the next reimage.
Both the USB and printed-parts kiosks now admit a caller on either ground -
the per-device token, or a source address inside an allowlist - the same
shape the GE-Enforce client endpoints already use. `shopdb.api` gained
`ip_in_cidrs` and `request_ip_allowlisted` (contract 0.35.0) so the two
plugins share one gate rather than two copies of a CIDR parser; they fail
closed on an empty setting, an unparseable caller address and a malformed
entry, and they read `remote_addr`, so the IIS rewrite rule is what makes
them meaningful. The allowlists are separate per kiosk
(`usb_kiosk_allowed_cidrs`, `printedparts_kiosk_allowed_cidrs`) for the same
reason the tokens are, and both SHIP EMPTY: nothing opens by itself, and a
site that enters no ranges keeps exactly today's token-only behaviour. Both
settings pages get a picker listing the network plugin's subnets with their
VLANs, storing resolved CIDR strings rather than subnet ids so the setting
still works at a site that does not run that plugin. Stated plainly in the
admin UI: anyone on an allowed network can check a drive in or out against a
badge number they guess, and badges are attribution, not authorization.
- **A kiosk says when it is not set up, and an administrator can copy the setup
link.** The settings API masks every key whose name contains `token`, so the
real kiosk token is on screen for exactly one moment - after Generate, before
Save - and a screen provisioned with the literal mask answers 404, which is
also how the route answers a caller it has never heard of. Each kiosk gained
a permission-gated provisioning-link route that returns the real setup URL to
somebody who already holds `settings.edit`, and both settings pages a "Copy
setup link" button that falls back to showing the link when the clipboard is
refused. Revealing a token writes an audit row. The USB kiosk now asks the
server on load whether this screen may be used and says plainly when it may
not, instead of rendering a keypad that only fails once somebody has typed
their SSO.
- **The USB audit says whether a returned drive came back wiped and scanned.**
Removable-media custody is CMMC evidence, and custody without the state of
the media on return is half the record. Both stores already held the values -
`sanitized` and `scanned_viruses` in the external `checkinoutlog`,
`waswiped` and `scannedviruses` in the self-hosted `usbcheckouts` - and
neither mode selected them. Both now report them in the JSON, the CSV and the
column list. Only a return carries them, so a check-out row shows blanks
rather than asserting something about an event that had not happened yet, and
unknown stays blank rather than rendering as "No": both stores keep these
nullable deliberately, and "nobody wrote it down" is a different claim from
"it came back unwiped".
### Changed
- **Endpoint agent 1.20.1 and tray 1.7.1. Ship them together.** Every HTTP
client in the agent, script and package downloads included, is now built by
one `AgentHttpFactory`, so there is a single place the pins and the
credential are applied rather than one per caller. `EnrollmentService`
enrolls once using whatever shared key the PC already has, stores the
credential DPAPI-protected in a SYSTEM-only directory and swaps to it without
a restart; an old server with no enroll route simply leaves the agent on the
shared key. A named pipe lets the interactive user ask the agent to send
status, send a report, list alerts and acknowledge one, so `--once` and
`--report` run as a user become pipe clients, and `appsettings.json` is
restricted to SYSTEM and Administrators once the pipe is up. The tray holds
no key and reads no agent configuration at all - everything it does goes
through that pipe, an acknowledgement is remembered only once the agent
confirms it, and when the agent is not running the menu says so instead of
failing quietly.
The version numbers moved from 1.20.0 and 1.7.0 with no behaviour change of
their own: those binaries were built and shipped to the display scope before
two fixes landed, so a rebuild carried different bytes under the same version
number. Detection is by hash and copes with that, but nobody reading a
version number can tell two 1.20.0.0 files apart, and the runbook asks for one
build across the share, the agent store and the display profile.
- **A PC that has enrolled is refused the shared fleet key.** Once a PC holds
its own credential, presenting the fleet key is either a misconfiguration or
somebody else using a leaked secret, and either way the right answer is not
to accept it: that call is refused with `SHARED_KEY_REFUSED`. This only ever
applies to a PC that has already enrolled. `plugin.endpoints.sharedkeyallowed`
still DEFAULTS TO ON, so every un-migrated bay goes on reporting exactly as it
does today; a site turns it off deliberately once its fleet has moved, and the
page asks for confirmation and says how many PCs are not enrolled yet.
`plugin.endpoints.autoenroll` also defaults to on, so a clean hostname and
serial match is issued a credential without an administrator waiting on it.
- **The shared-key page now reads as the migration state it is.** Settings >
Endpoints presents the key as what PCs use UNTIL they enroll, with enrollment
progress, a link to Enrollment, and rotate and revoke confirmations that say
how many PCs would lose their only credential. The settings cards describe
the server certificate as what PCs actually receive, on IIS or Docker.
`AGENT-CONTRACT.md` documents per-PC credentials, and the rollout runbook
covers the phases, the gates, rollback, the pin and wave entries, and every
place the shared key lives and how to clear it at cutover.
### Removed
- **Client certificate authentication, the 2026-09-14 attempt.** `client_cert.py`,
certificate renewal, expiry monitoring, the unauthenticated
`/test/client-cert` route, the nginx client-certificate forwarding and the
agent certificate script are all gone. The header carrying the client
certificate could be forged on IIS, the private key was readable by users,
and the shared key remained a fallback underneath it all, so it bought
nothing the per-PC credential above does not buy properly. Migration 0023
drops the certificate columns from `assets` where a test database has them;
0021 and 0022 keep their ids but no longer touch the core table.
### Security
- **The tray no longer holds a secret.** It ran as the interactive user and
carried enough configuration to talk to the server on its own. Everything it
needs now comes over the agent's pipe, which means a standard user on a bay
can read nothing out of it.
- **`collect_log` refuses to collect the things that hold credentials.** Setup
answer files, `appsettings.json`, agent directories and key files are refused
at enqueue rather than shipped to the server inside a log bundle. The setup
token is no longer printed to the log either, and two VM notes that carried a
live collector key and advised plain http were deleted rather than corrected.
## [0.12.0] - 2026-08-20
Printers become a property of the bay rather than of whoever last walked up to a

50
CLAUDE.md
View File

@@ -1,12 +1,12 @@
# ShopDB Flask - Claude Code Context
ShopDB is a manufacturing floor asset management system built with Flask, Vue 3, and PostgreSQL, deployed across air-gapped GE Aerospace sites.
ShopDB is a manufacturing floor asset management system built with Flask, Vue 3, and MySQL, deployed across air-gapped GE Aerospace sites.
## Architecture
- **Backend**: Flask 3.x with SQLAlchemy 2.x, plugin-based architecture
- **Frontend**: Vue 3 with Vite, component-based UI with plugin panels
- **Database**: PostgreSQL 15+ with Alembic migrations per plugin
- **Database**: MySQL 8 with Alembic migrations per plugin (`docker-compose.yml` runs `mysql:8.4`; WJ prod is IIS in front of MySQL 8)
- **Deployment**: Docker Compose with gunicorn, air-gapped sites via bundle workflow
- **Plugins**: 20+ plugins (computers, endpoints, printers, machines, etc.)
@@ -27,10 +27,13 @@ Each plugin owns its migration chain in `plugins/{name}/migrations/versions/`.
- Migrations run automatically on deploy
### Database Naming Convention
- **Tables**: lowercase concatenated, no underscores (e.g., `printedpart`, `mappeddrives`)
- **Columns**: snake_case (e.g., `serial_number`, `last_seen`)
- **Indexes/constraints**: follow PostgreSQL conventions
- Enforced by `scripts/check-naming-and-style.sh` pre-commit hook
- **Tables**: lowercase concatenated, plural, no underscores (e.g., `machines`, `mappeddrives`)
- **Columns**: lowercase concatenated, singular, no underscores (e.g., `machineid`,
`lastzabbixsync`, `isactive`). NOT snake_case - see CONTRIBUTING.md
- **Indexes**: `idx_<table>_<column>`. Underscores are allowed HERE ONLY, because an
index is infrastructure and not data
- Enforced by `scripts/check-naming-and-style.sh` pre-commit hook, on model attributes
AND on the column names a migration creates
### Branch and Deployment Model
- `wj` -> prod (shop floor at WJ site)
@@ -54,10 +57,20 @@ Plugins extend ShopDB via:
- `frontend/views/` - Vue components auto-discovered by router
- `migrations/versions/` - Alembic migrations
Import from `shopdb.api` for stable surfaces:
Import from `shopdb.api` for stable surfaces. Every name has to be in that
module's `__all__`; this line used to read
`db, Asset, Employee, BasePlugin` and two of the four do not resolve, so the
first block a new agent copied raised ImportError:
```python
from shopdb.api import db, Asset, Employee, BasePlugin
from shopdb.api import db, Asset, User, Setting, require_permission
# BasePlugin is NOT on that surface. plugin.py takes it from its own module,
# which is what every plugins/*/plugin.py in this tree does:
from shopdb.plugins.base import BasePlugin, PluginMeta
```
There is no `Employee` model. The directory is an external read-only database
reached through `employee_connection`; the plugin's own row type is
`DirectoryEmployee`. See `.claude/QUICK-REFERENCE.md` for the full export list
and for the other names that are commonly assumed and are not there.
## Documentation
@@ -69,7 +82,7 @@ Full documentation in the GitHub wiki at `../shopdb-flask.wiki/`:
- ADR-006: Collector contract (endpoint agent)
- ADR-007: Product versioning and releases
- ADR-013: Plugin catalog and lean builds
- ADR-016: Credential delivery
- ADR-016: Credential delivery to the fleet (cmdkey credentials for bays; ACCEPTED, NOT built - it is not about kiosks or device tokens)
**References**:
- `API-REFERENCE.md` - Core API surface
@@ -116,7 +129,24 @@ Full documentation in the GitHub wiki at `../shopdb-flask.wiki/`:
- **Migration conflicts**: Keep plugin chains linear, coordinate with PLUGIN_TABLE_OWNERS
- **Unicode**: ASCII only - check logs, comments, commit messages
- **Import surface**: Use `shopdb.api`, not internal paths
- **Device tokens**: Kiosks require device_token authentication (ADR-016)
- **Kiosk access**: a kiosk route admits a caller on either of two grounds - the
per-kiosk device token, or a source address inside that kiosk's CIDR allowlist
(`usb_kiosk_allowed_cidrs`, `printedparts_kiosk_allowed_cidrs`) - and both are
gated on the kiosk's enable flag. Use `ip_in_cidrs` from `shopdb.api`; do not
hand-roll a CIDR parser. `request_ip_allowlisted` is only
`ip_in_cidrs(client_ip(request), cidrs)` - the USB kiosk does NOT use it,
because it first refuses an address that belongs to the path rather than to a
caller (loopback, or a configured trusted proxy). On the shipped Windows stack
the X-Forwarded-For rewrite rule is disabled, so every caller arrives as
127.0.0.1 and that refusal is what keeps the door shut. The real shape and the
full trace are in `.claude/QUICK-REFERENCE.md`. There is no
`requires_device_token` decorator
- **Fleet credentials are in the registry, deliberately**: the endpoint agent and the
GE-Enforce reporters run as SYSTEM with no user to prompt, and read their collector
key, base URL and server pins from `HKLM\SOFTWARE\GE\ShopDB`. An enrolled PC's
own per-PC credential is DPAPI-protected in a SYSTEM-only directory instead.
Nothing forbids the registry hive - ADR-016 names registry provisioning as a
recommended first step
## Work PC Role

42
docs/wiki-drafts/ADR-019-Plugin-Dynamic-Settings-Routes.md
View File

@@ -2,7 +2,12 @@
**Status:** Accepted (2026-09-11)
**Context:** Plugin UI extensions, lean builds
**Supersedes:** Hardcoded plugin routes in core router
**Supersedes:** Hardcoded plugin routes in core router
**Last checked against code:** 2026-09-18, branch gh-wj at 35d4fcd41
This draft lives only in `docs/wiki-drafts/`. There is no `docs/adr/` directory
on this branch, so this is the sole ADR-shaped record here and nothing enforces
it except the tests named below. See ADR-Conformance-2026-09-18.md.
## Problem
@@ -63,27 +68,44 @@ class ComputersPlugin(BasePlugin):
### Backend: API Endpoint
Added to `shopdb/core/api/plugins.py`:
In `shopdb/core/api/plugins.py:46-95`, as it stands on gh-wj at 35d4fcd41:
```python
@plugins_bp.route('/settings-routes', methods=['GET'])
@jwt_required(optional=True)
@require_permission('settings.view')
def get_plugin_settings_routes():
"""Return settings routes contributed by all enabled plugins."""
pm = current_app.extensions.get('plugin_manager')
if not pm:
return success_response({'routes': []})
routes = []
for plugin in pm.get_enabled_plugin_instances():
if hasattr(plugin, 'get_settings_routes'):
for plugin_name, plugin in pm.get_all_plugins().items():
# Only include routes from enabled plugins
if not pm.registry.is_enabled(plugin_name):
continue
if not hasattr(plugin, 'get_settings_routes'):
continue
# One broken plugin must not take every settings hub down with it.
try:
plugin_routes = plugin.get_settings_routes()
if plugin_routes:
routes.extend(plugin_routes)
except Exception:
current_app.logger.exception(
'get_settings_routes failed for plugin %s; skipping it',
plugin_name)
continue
routes.extend(renderable_settings_routes(plugin_name, plugin_routes,
current_app.logger))
return success_response({'routes': routes})
```
Earlier revisions of this draft showed a `@jwt_required(optional=True)` handler
iterating `pm.get_enabled_plugin_instances()`. Neither is in the code:
`grep -rn get_enabled_plugin_instances shopdb/ plugins/` returns nothing, so
that block would not have run. It is replaced above rather than kept, because a
copy-paste block that reopens the anonymous read is worse than no block.
**Auth:** the endpoint is gated on `settings.view`. It was written anonymous, on the reasoning that unauthenticated pages (kiosks, TV displays) register plugin tabs before login. Nothing does: its only caller is `ModelCatalogHub.vue`, which sits behind `requiresAuth` and `requiresAdmin`. Open, it handed an anonymous caller the enabled plugin list, the settings hub layout and the internal Vue component paths, so it was closed.
### Frontend: Dynamic Route Registration
@@ -323,7 +345,7 @@ const pluginComponents = import.meta.glob('../.plugins-staged/*/views/*.vue')
**Nested routes:** A plugin could return routes with `children: [...]` for nested settings pages. Router supports this via `addRoute(parentName, childRoute)`.
**Permissions:** The `tab` could include a `permission` field (e.g., `"computers.manage_os"`), and the frontend filters out tabs the user can't access. The backend endpoint is anonymous, so permission checks would be client-side only (acceptable for UI gating; the API endpoint enforces permission anyway).
**Permissions:** The `tab` could include a `permission` field (e.g., `"computers.manage_os"`), and the frontend would filter out tabs the user cannot reach. This would be UI gating only, and the page behind the tab must still enforce its own permission; the `settings.view` gate on `/api/plugins/settings-routes` says who may read the tab list, not who may use a given tab.
**Icons:** Add an `icon` field to the route metadata (e.g., `"mdi-microsoft-windows"`), and the tab renders the icon next to the label.

556
docs/wiki-drafts/ADR-Conformance-2026-09-18.md Normal file
View File

@@ -0,0 +1,556 @@
# ADR Conformance Review - September 18, 2026
**Branch:** gh-wj at 35d4fcd41
**Range reviewed:** 2ac2e6d33..gh-wj, 20 commits, 149 files, +22055 lines
**Method:** read-only. Every statement below cites a file and line that was read.
No test suite was run except `tests/test_plugins/test_kiosk_network_allowlist.py`
(20 passed) and `bash scripts/check-naming-and-style.sh` (exit 0).
## Read this first: the ADRs are not in this repository
`ls docs/adr` fails on gh-wj. There is no ADR directory on this branch. The
whole set, ADR-001 through ADR-018 plus a README, lives on the other lineage at
`docs/adr/ in the product repo` and was read there for this review.
`docs/` on gh-wj holds only `ShopDB-Planner-Tasks-v2.csv`, `deployment/`,
`notes/`, `openapi.json` and `wiki-drafts/`.
Three consequences, and they are the reason this page exists:
1. **Nothing on this branch states the rules.** A person working here has no
ADR to read, so a decision can be contradicted without anyone noticing that
a decision existed. That is exactly what happened to ADR-016 (see below).
2. **Nothing on this branch enforces them except one script.**
`scripts/check-naming-and-style.sh` is the only ADR-aware gate in the repo,
and it names two ADRs: ADR-017 at line 343 and ADR-015 at line 395. Its other
eight checks enforce CONTRIBUTING.md house rules, not ADRs. A handful of
tests cover ADR ground without being described that way:
`tests/test_plugin_migrations.py` (ADR-008 chain shape and ownership),
`tests/test_core/test_plugin_settings_hubs.py` (ADR-019),
`tests/test_plugin_contract.py` (manifest fields),
`tests/test_core/test_public_read_surface.py` (the read gate).
`tests/test_api_inventory_parity.py` skips itself entirely on this branch:
`docs/api-inventory.json` is not tracked here, and its own skip reason says
"docs/ is excluded from publication".
3. **Some violations cannot be fixed from here.** ADR-018 has no
implementation-status note; ADR-016 is still marked "decided, NOT yet
implemented". Both corrections belong in files that are not in this
repository.
The one ADR-shaped file on gh-wj is the ADR-019 draft in this folder, and it is
a draft.
## Result by ADR
| ADR | Subject | Result |
|-----|---------|--------|
| ADR-001 | Asset as platform contract | Two exceptions, both below: the unbumped Asset cert columns (2) and raw SQL over core tables (6) |
| ADR-002 | Plugin versioning | Holds for the bumps. Two stale manifests, and one unbumped core-model change (2) |
| ADR-003 | Plugin distribution | Holds |
| ADR-004 | Deployment topology | Holds |
| ADR-005 | Equipment vs measuringtools | Superseded by ADR-018 |
| ADR-006 | Collector contract | Holds on shape. One gap in a new boundary |
| ADR-007 | Product versioning and releases | VIOLATED (CHANGELOG) |
| ADR-008 | Plugin migration ownership | VIOLATED (plugin chain, core table) |
| ADR-009 | Frontend plugin gating | Holds |
| ADR-010 | Frontend plugin hooks | Holds |
| ADR-011 | Machines rename | Holds |
| ADR-012 | GE-Enforce manifest ownership | Holds |
| ADR-013 | Plugin catalog and lean builds | Holds |
| ADR-014 | Schema-lean per site | Holds |
| ADR-015 | Site-specific configuration | Holds in scope. Scope has a hole |
| ADR-016 | Credential delivery | Unbuilt by its own status, and MISDESCRIBED in four files here |
| ADR-017 | Buildings and levels | Holds |
| ADR-018 | Metrology instruments are machines | Accepted, not implemented |
| ADR-019 | Plugin dynamic settings routes (draft) | Implementation matches the draft |
## What holds, and the evidence
**ADR-001, the import surface.**
`grep -rn "from shopdb\.\(core\|utils\|extensions\|services\)" plugins/
--include=*.py` returns zero hits. The only `shopdb.*` import in `plugins/`
outside `shopdb.api` is `shopdb.plugins.alembic_template` from migration
`env.py` files, which is the shared Alembic runner ADR-008 sanctions. The three
names added to the contract this range - `BusinessUnit`, `ip_in_cidrs`,
`request_ip_allowlisted` - are both defined and listed in `__all__`
(`shopdb/api/__init__.py:190-194`, `:471-486`). The new endpoints tables key off
`assets.assetid` by foreign key rather than introducing a parallel identity.
This covers the names added and kept. It does NOT cover a name added to the
Asset model and then withdrawn again without a bump, which happened in this
range; see finding 2.
**ADR-002, the bumps.** `__contract_version__` moved 0.32.0 to 0.35.0 with a
per-bump rationale naming what was added and why it is additive
(`shopdb/__init__.py:124-152`). Every name those comments claim is genuinely
exported. Every new `BasePlugin` hook is optional with a safe default
(`shopdb/plugins/base.py:313-332`), so minor is the right classification. The
loader still checks `core_version` against `__contract_version__`
(`shopdb/plugins/loader.py:255-258`).
**ADR-004 and ADR-014, per-site instances.** No `siteid` column anywhere in the
new schema. utf8mb4 still pinned in `.env.example`. The two-command deploy
(`flask db upgrade` then `flask plugin upgrade-all`) is unchanged. All four new
endpoints tables are registered in `PLUGIN_TABLE_OWNERS`
(`shopdb/plugins/alembic_template.py:57-63`) in the same change that creates
them, so `flask plugin prune-schema` still leaves a lean site lean.
**ADR-006, the collector shape.** `/api/collector/<pluginname>` is still the
single dynamic dispatch route (`shopdb/core/api/collector.py:229`), the response
envelope is byte-for-byte the ADR's `{status, action, assetid, identityvalue,
warnings}` (`:324-331`), every submission still writes an AuditLog row
(`:313-320`), `/api/collector/_schemas` is still JWT-protected (`:337-341`), and
per-plugin `COLLECTOR_API_KEY_<PLUGIN>` with shared fallback resolves as the ADR
specifies (`:194-203`). Identity integrity under the new credential mode is
enforced in all three schema-declaring plugins - a per-PC credential reports
only for its own asset (`plugins/computers/plugin.py:729-733`,
`plugins/printers/plugin.py:556-559`, `plugins/endpoints/plugin.py:1422-1427`) -
and the dispatcher maps the `PermissionError` to 403 ASSET_MISMATCH
(`collector.py:294-299`).
**ADR-007, the version lock-step.** `__version__ = '0.12.0'`
(`shopdb/__init__.py:158`) matches `frontend/package.json:3`, and `__version__`
is still absent from `shopdb.api`.
**ADR-008 in the forward direction.**
`git diff --stat 2ac2e6d33..gh-wj -- migrations/` is empty. Not one line of
plugin schema went into the core chain. The endpoints chain is 24 revisions with
exactly one root and one head, no dangling `down_revision`, longest revision id
29 characters against the 32-character `alembic_version` cap.
**ADR-009, route gating.** Every plugin-owned route added or touched this range
carries `meta.plugin`, including the two new endpoints settings pages, the USB
kiosk and the printed-parts kiosk. `GET /api/plugins/enabled` stayed
`jwt_required(optional=True)` (`shopdb/core/api/plugins.py:31-42`) through the
read-permission sweep that gated 165 other GETs, so the unauthenticated kiosk
gate still works.
**ADR-010 and ADR-013, frontend boundaries.** The new
`frontend/src/components/NetworkAllowlistPicker.vue` is a CORE component
consumed by plugin views through the `@/` alias
(`plugins/usb/frontend/views/USBSettings.vue:108`,
`plugins/printedparts/frontend/views/PrintedPartsSettings.vue:164`), which is
the established pattern; 97 plugin frontend files already import that way. No
core file gained a hardcoded plugin import.
**ADR-012, GE-Enforce ownership.** The per-PC rollout keeps ShopDB as author of
record: the runbook's JSON snippets are entries to author and publish through
the plugin, not hand edits to the share, and `Set-ShopDBServerPins.ps1` is
staged as a normal PS1 entry with `TargetHostnames`/`TargetMachineNumbers`
gates. The engine is still referenced, not vendored.
**ADR-015 within its enforced scope.** `scripts/check-naming-and-style.sh` exits
0 and the pipeline is not vacuous: a control pattern over the same six
directories returns 1995 lines. The new kiosk network admission is a setting
with a blank default and a documented-neutral example
(`plugins/usb/plugin.py:153-168`), blank means off
(`shopdb/utils/ipallowlist.py:44-46`), and `frontend/vite.config.js:5-16`
replaced a hardcoded dev-proxy address with `VITE_DEV_API_TARGET`.
**ADR-017.** The per-occurrence `mapx`/`levelid` gate and the
`LocationMapTooltip :levelid` gate both pass. No payload in this range emits a
position without its level.
**ADR-019 against the implementation.** `SETTINGS_HUBS_WITH_PLUGIN_TABS =
('model-catalog',)` at `shopdb/plugins/base.py:29`; the loud drop at `:32-54`;
the `settings.view` gate at `shopdb/core/api/plugins.py:47`; the still
unrewritable `` import(`../../${tab.component}`) `` at
`frontend/src/views/settings/ModelCatalogHub.vue:84` with no `vite.config.js`
change to fix it; and a non-vacuous test suite in
`tests/test_core/test_plugin_settings_hubs.py`. The draft in this folder was
corrected on 2026-09-18: its code block still showed
`@jwt_required(optional=True)` and its Future Extensions section still called
the endpoint anonymous, both contradicting its own Auth paragraph and the
shipped code.
## Violations
### 1. ADR-007: no release record for the largest change in the range
`CHANGELOG.md:11`
`git diff --stat 2ac2e6d33..gh-wj -- CHANGELOG.md` returns nothing at all. The
file is byte-identical across all 20 commits, which include per-PC agent
credentials, enrollment, bootstrap tokens, server pinning, kiosk network
admission and three contract bumps. Grepping the whole file for "per-PC",
"X-Endpoint-Credential", "Endpoint credential" and "enrol" finds nothing
relevant. ADR-007 decision point 3 makes `CHANGELOG.md` the canonical
human-readable record and says work in flight accumulates under
`## [Unreleased]`; that block still describes only the pre-range share-link and
read-gate work.
Low severity: no release has been cut in the range, so the obligation point
(ADR-007 release procedure step 3) has not been reached and the debt is fully
repairable before the tag. Operator-facing content does exist, at
`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`, but that is plugin-local
and not where ADR-007 says an adopter looks.
### 2. ADR-008: a plugin migration chain does DDL on the core `assets` table
`plugins/endpoints/migrations/versions/0023_endpoint_credentials.py:73` and `:76`
0023 declares `plugin_name = 'endpoints'` (line 33) and then runs
`op.drop_index(index['name'], table_name='assets')` and
`op.drop_column('assets', column)`. It is cleaning up after itself:
`git show 5cd533239:plugins/endpoints/migrations/versions/0021_client_certificates.py`
shows the shipped 0021 adding three columns to `assets` plus
`idx_assets_cert_fingerprint` from the same plugin chain. `assets` is not, and
cannot be, in `PLUGIN_TABLE_OWNERS['endpoints']`
(`shopdb/plugins/alembic_template.py:57-63`), so the metadata ownership filter
never saw it, and `tests/test_plugin_migrations.py` has no test over which
tables a chain touches.
Low severity: every step is guarded on the live schema (0023:64-76), 0021 and
0022 are now no-ops, the forward direction is clean, and
`tests/test_plugins/test_endpoints_credentials_migration.py:96-140` runs the
chain twice from both a legacy and a fresh schema. The lane crossing is real;
the fleet blast radius is essentially zero.
**The ADR-001/ADR-002 half of this was written up wrong, and that conclusion is
withdrawn here (2026-09-18).** The original text read: "`git show 5cd533239
--stat` touched no file under `shopdb/`, and `grep -rn cert
shopdb/core/models/asset.py` is empty: the Asset *model* never carried those
columns. Only the database did. No contract bump was owed." Both halves of the
evidence are real and the conclusion drawn from them is still false, because
5cd533239 is not the only commit in the range and `grep` only sees the tree as
it stands now. `git show 18e5fdc00:shopdb/core/models/asset.py` puts
`client_cert_fingerprint` (declared `index=True`), `cert_issued_at`,
`cert_expires_at`, `cert_revoked_at` and `cert_revocation_reason` on the CORE
`Asset` model. 18e5fdc00 (2026-09-14, "Fix critical security vulnerabilities in
client certificate authentication") is an ancestor of `gh-wj`; 7480e4fe3
(2026-09-15) deleted all 28 lines again. The grep comes back empty because it
runs after the retraction, not because the contract was never touched.
The ADR-001/ADR-002 finding therefore holds, in a worse form than first stated:
the core Asset contract surface was extended to carry a plugin's feature and
then retracted, with no contract bump in either direction, and for the length of
that window five underscore-spelled columns sat on a core model in a repo whose
column convention forbids underscores. Severity stays low - the window is inside
an unreleased branch, and nothing outside the endpoints plugin ever read the
columns - but the finding is a real ADR-001/ADR-002 breach and not, as written,
a non-finding.
**Now guarded, 2026-09-18.** The finding above ended with "no test over which
tables a chain touches". There is one:
`tests/test_plugin_migrations.py::test_no_plugin_migration_touches_another_lanes_table`.
It AST-scans every `plugins/*/migrations/versions/*.py` for `create_table`,
`drop_table`, `add_column`, `drop_column`, `alter_column`, `create_index`,
`drop_index`, `rename_table`, the five constraint ops (`create_foreign_key`,
`drop_constraint`, `create_unique_constraint`, `create_check_constraint`,
`create_primary_key`) and the `batch_alter_table` forms, resolves each target
table through module-level constants and literal `for` sequences, and fails when
the table belongs to `core` or to a different plugin. Ownership comes from
`PLUGIN_TABLE_OWNERS` and a static scan of `shopdb/core/models/` (including the
three bare `db.Table` association tables), so the map stays the single source of
truth rather than a table list copied into a test.
Raw SQL is scanned too, because `op.execute('ALTER TABLE assets DROP COLUMN
assetnumber')` is DDL that no `op.*` scan can see. `op.execute` and
`exec_driver_sql` have their statement text flattened - f-string holes and
unresolvable names become a sentinel character rather than being dropped - and
`ALTER TABLE x`, `DROP TABLE x`, `CREATE TABLE x`, `RENAME TABLE x TO y`,
`CREATE/DROP INDEX ... ON x` and `TRUNCATE x` name the op's target table.
`UPDATE`, `DELETE`, `INSERT` and `SELECT` are data, not schema, and are not
flagged: three shipped revisions already run cross-checked DML
(`plugins/backups/.../0002_backups_lastseenat.py:37`,
`plugins/backups/.../0003_backups_clear_backfilled_lastseen.py:58`,
`plugins/computers/.../0006_policyactionhold_indefinite.py:78`) and all three
stay green. Keeping the hole in place is what separates the two cases that
matter: `machines 0002` interpolates a CONSTRAINT name into
`f"ALTER TABLE equipment DROP FOREIGN KEY {name}"`, where the table is literal
text and resolves; `f'ALTER TABLE {t} DROP COLUMN x'` puts the hole inside the
table name, resolves to nothing, and fails as UNRESOLVED, which is the obvious
dodge closed.
Four supporting tests keep it honest: an op whose table name cannot be resolved
statically is a failure, not a pass, so a computed name does not slip through; a
non-vacuity test asserts the scan still finds more than 100 ops and still
detects the 0023 crossing; a second non-vacuity test pins two specific raw-SQL
ops out of `machines 0002` and asserts the three DML statements are absent; and
a dead-entry test fails if an allowlist line stops matching anything, so an
excuse cannot outlive what it excused.
The allowlist keys on `(plugin, filename, op, table, column)`, not on the first
four. Keyed on the first four alone, the one shipped `drop_column on assets`
entry would also excuse a brand new `op.drop_column('assets', 'assetnumber')`
appended to that same shipped file - the exact breach the guard exists to stop -
so the six entries name their columns: one `drop_index` (column `None`, the
index name is computed) and one `drop_column` per certificate column. 0021 and
0022 are not listed, because they are stamp-only no-ops and present no DDL to
scan.
Verified non-vacuous against the pre-fix version of the same test module. Three
violations were planted in a scratch copy of `plugins/` (the shipped 0023 is
applied on production and was not edited in place): the five constraint ops on
`assets` from a plugin chain, `op.execute` with both a literal `ALTER TABLE
assets` and an f-string table name, and a new `op.drop_column('assets',
'assetnumber')` inside 0023. The pre-fix guard passed all four of its tests on
all three plants. The current guard fails, naming all seven ops with file, line,
op, table, column and owner, and goes green again once the plants are removed.
**The 0023 index-drop loop was examined and needs no repair.** 0023:71-73 drops
any index on `assets` whose column set intersects the certificate columns and
recreates nothing, which would matter if such an index could carry columns that
still exist. It cannot. The complete set of things that create an index on
`assets` is:
* the core baseline `68b3947ae14f`: `idx_asset_active`, `idx_asset_location`,
`idx_asset_status`, `idx_asset_type_bu`, `ix_assets_assetnumber`,
`ix_assets_serialnumber`
* core `7b02`: `ix_assets_gaugelabreference`, `ix_assets_maintenancereference`
* core `7d33`: `idx_assets_levelid`
* the shipped endpoints `0021`: `idx_assets_cert_fingerprint` over
`client_cert_fingerprint` alone (the shipped `0022` added two columns and no
index)
* `db.create_all` off `shopdb/core/models/asset.py`. The earlier version of this
inventory said that builds "that same set" as the core baseline, on the
strength of the Asset model never carrying a cert column. As corrected above,
between 18e5fdc00 and 7480e4fe3 it did, with `client_cert_fingerprint`
declared `index=True`, so `create_all` in that window ALSO built
`ix_assets_client_cert_fingerprint`. That index is missing from every earlier
telling and is the one real correction to this list.
Re-checked rather than assumed, the conclusion is unchanged.
`ix_assets_client_cert_fingerprint` is single-column over
`client_cert_fingerprint`, which `_drop_certificate_columns` removes in the
statement immediately after the loop, so the index carries no column that
outlives it and there is nothing to recreate. The filter is also unreachable by
accident: every name in `_CERT_COLUMNS` is underscore-spelled, which the column
convention forbids, so no real `assets` column can collide with it. The only two
indexes the loop can ever drop are `idx_assets_cert_fingerprint` and
`ix_assets_client_cert_fingerprint`, both dead by construction. No index was
lost on any database at the reference site and no repair migration is owed; the reasoning is
recorded in the test file beside the allowlist entries so it is not
re-litigated.
### 3. ADR-016: four files here describe a decision ADR-016 never took
`docs/wiki-drafts/README.md:316-323` (corrected 2026-09-18), `CLAUDE.md:119`,
`the assistant quick-reference file, lines 135 and 14`,
`docs/wiki-drafts/Security-Audit-2026-09-11.md:392`
All four attribute "device tokens" to ADR-016. `grep -rni device
docs/adr/ in the product repo*.md` returns no hit for device tokens
in ANY ADR. ADR-016's Decision is a fleet credential store with Fernet at rest,
a targeting and precedence resolver reusing `plugins/geenforce/filters.py`, an
explain endpoint, `credentials.manage`, and `GET /api/collector/credentials`
behind a dedicated `credentials.fetch` scope.
The `docs/wiki-drafts/README.md` copy went further and, under a heading
"ADR-016: Credential Delivery", listed registry keys among the places
credentials must **never** live. ADR-016 says the opposite: its
"Alternatives rejected" section calls registry-only provisioning "a worthwhile
FIRST STEP" and "Recommended as tier one regardless". The ban also contradicted
shipped code on this branch:
`plugins/computers/client/Report-AssetToShopDB.ps1:367-374` reads `CollectorKey`
out of `HKLM:\SOFTWARE\GE\ShopDB`, and
`plugins/endpoints/client/Set-ShopDBServerPins.ps1:4-20` writes the server trust
pins into the same key by design, with
`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:106-111` ("The registry
wins") making that the intended precedence. That section was rewritten on
2026-09-18; the three files outside `docs/wiki-drafts/` still carry the
misattribution and are out of scope for this edit.
Two qualifications, so nobody overcorrects: `PER-PC-CREDENTIALS-ROLLOUT.md:351-361`
deliberately plans to clear `CollectorKey` from both registry views at cutover,
so the shared key in HKLM is being retired anyway; and `ServerPins` are trust
anchors, not credentials.
### 4. ADR-016: the per-PC programme redefines "credential" with no design record
`plugins/endpoints/models/credential.py:1`
ADR-016's mechanism is unbuilt. Grepping `shopdb/` and `plugins/` for
`credentials.fetch`, `collector/credentials` and `Fernet` returns zero hits, and
`plugins/endpoints/plugin.py:154-265` declares only `endpoints.*` permissions -
no `credentials.manage`, no `credentials.fetch`. That matches ADR-016's own
"Implementation status, 2026-08-14". What the 20-commit programme built is a
different subject under the same word: `EndpointCredential` /
`EndpointEnrollRequest` / `EndpointMigrationWave` as the agent's own identity
secret (`models/credential.py:1-22`), `X-Endpoint-Credential` as the agent auth
header (`api/agentauth.py:1-18`), and an unauthenticated `/enroll`
(`api/enrollment.py:289`).
The design itself is sound and consistent with ADR-016's intent: the rollout
removes the cleartext key from the share, and rotation is a settings change plus
a pickup report, which is what ADR-016 wanted. The gap is that nobody wrote it
down as a decision. `grep -ril 'enroll|endpoint credential|X-Endpoint-Credential'`
over the ADR directory on the other lineage matches only ADR-012, which is
unrelated. Low severity and not fixable from here: the missing record belongs on
the lineage that carries `docs/adr/`.
### 5. ADR-018: accepted, unimplemented, with nothing saying so
`shopdb/plugins/alembic_template.py:74`
ADR-018 (ACCEPTED, 2026-09-04) decides that the `measuringtools` plugin is
retired and the `measuring_tool` asset type removed, that machines gains four
calibration columns and machinetypes gains `ismetrology`. None of it exists
here: `plugins/measuringtools/` still ships with its manifest,
`alembic_template.py:74` still registers
`'measuringtools': ('measuringtooltypes', 'measuringtools')`,
`shopdb/core/api/dashboard.py:19`, `:55` and `:93` still count `measuring_tool`
into the asset totals, `shopdb/cli/__init__.py:1021` still seeds it,
`deploy/site-profile-universal.json:12` still lists it in the universal profile,
and `grep -rn ismetrology shopdb/ plugins/` and
`grep -n calibration plugins/machines/models/*.py` both return nothing.
`git log -3 -- plugins/measuringtools/` shows no retirement work.
Unlike ADR-016, ADR-018 carries no implementation-status paragraph, so its text
reads as done. Low severity: the table holds zero rows, so nothing is broken
today, and ADR-018's own argument is that the window is time-boxed - the
migration is free while the 134 staged metrology instruments have not been
entered. gh-wj is the branch that deploys to the reference site, so entering them here is what
closes the window.
### 6. ADR-001 / ADR-002: raw SQL over core tables from a plugin
`plugins/computers/plugin.py:2080`
`db.session.execute(db.text(...))` joining core `assets`, `assetrelationships`
and `assettypes`, inside the `retype` CLI command. `Asset`, `AssetType`,
`AssetRelationship` and `RelationshipType` are all exported and in `__all__`
(`shopdb/api/__init__.py:47-48`, `:424`, `:438-439`), so unlike the
`BusinessUnit` case that the 0.34.0 bump note closed, there is no missing export
to blame, and `plugins/endpoints/api/routes.py:3981-3986` is the same pattern
already corrected. The 0.34.0 note names the reason in the code:
"a raw query is hidden coupling to the core schema that no contract bump can
catch, and it is the kind of thing that gets copied"
(`shopdb/__init__.py:135-142`).
Low severity and arguably generous: ADR-001's stated rule is an import rule,
which raw SQL does not literally breach, and the blast radius is one admin CLI
command with no serving route affected. Predates this range (331d27e07). The
other plugin raw query, `plugins/computers/plugin.py:2128`, touches only
plugin-owned tables and is fine.
### 7. ADR-015: the site-literal scan does not cover the C# agent
`scripts/check-naming-and-style.sh:440`
Lines 436-440 scope the grep to `plugins/ shopdb/ scripts/ deploy/
frontend/src/ tools/` with `--include` on
`.py/.vue/.js/.ps1/.psm1/.sh/.iss/.json/.html`. `agent/` is absent and `.cs` is
not an included extension. This range added roughly 2,500 lines of C# under
`agent/`, all of which ships to a sister site. Re-running the assembled site
patterns by hand over `agent/`, excluding bin and obj, finds one hit: prose in
`agent/ShopDbEndpointAgent/ShopDbEndpointAgent.csproj:108`, which ADR-015
explicitly permits ("Comments and documentation may name a site where it
explains WHY"). So the rule holds today; the gate would not notice if it stopped
holding.
**This is a coverage gap in the rule, not a breach of it.** ADR-015's 2026-08-14
amendment names those six directories as the scan's scope, so the script matches
the ADR precisely. The amendment widened the scan on exactly this reasoning -
the original scope missed every literal that reached a second site, because
those were in the client scripts and the installer - and the C# agent is now
that client channel.
### 8. ADR-002: two manifests declare a floor below the contract name they import
`plugins/usb/manifest.json:8` and `plugins/printedparts/manifest.json:8`
`da91a2915` added `shopdb/utils/ipallowlist.py`, bumped `__contract_version__`
to 0.35.0 with `ip_in_cidrs` and `request_ip_allowlisted` named as the new
surface (`shopdb/__init__.py:143-152`), and made
`plugins/usb/api/routes.py:42` and `plugins/printedparts/api/routes.py:521`
import `request_ip_allowlisted`. Neither manifest was touched: usb still says
`">=0.1.0,<1.0.0"` and printedparts `">=0.16.0,<1.0.0"`. The team does maintain
this field - `066a19052` raised `plugins/endpoints/manifest.json:7` to
`">=0.34.0,<1.0.0"` one commit earlier.
On a 0.34.0-or-older core both plugins would pass the loader's version gate
(`shopdb/plugins/loader.py:141-166`, called at `:258`) and then die with an
ImportError at blueprint registration. Impact at the reference site is nil, because core and
plugins always ship from this one tree.
The ADR-002 framing is weak and should not be quoted as a clean breach: ADR-002
itself says pre-1.0 plugins should declare `">=0.1.0,<1.0.0"`, which is
literally what `plugins/usb/manifest.json:8` declares, while its Neutral section
says bundled plugins' `core_version` is "always the current
`__contract_version__`". The ADR contradicts itself here. This is staleness
against team practice.
## Checked and not a violation
These were examined and did NOT hold up as ADR breaches. They are listed so the
next review does not re-open them.
- **`get_settings_routes` narrowed to a one-hub allowlist without a contract
bump.** `shopdb/core/api/plugins.py:92` now filters through
`renderable_settings_routes`. Nothing observable changed: the only frontend
caller, `frontend/src/views/settings/ModelCatalogHub.vue:73`, already filtered
to its own hub at `:76`, and `frontend/src/router/routes/core.js:62-63` states
in-tree that no other hub page fetches or registers plugin routes. A route
naming another hub was drawn nowhere before and is drawn nowhere now. No
`BasePlugin` ABC signature or `PluginMeta` schema changed, so under ADR-002
this is patch-level and no bump was owed. It is also not silent:
`shopdb/plugins/base.py:46-52` logs an error naming the plugin, the hub and
the supported set, and the same commit added the "Constraints as implemented"
section to the ADR-019 draft plus
`tests/test_core/test_plugin_settings_hubs.py`.
- **`computers` and `printers` manifests declaring `>=0.20.0` against the
contract-0.33.0 key `acceptsagentcredential`.** ADR-002 requires no such
tracking, nothing in `loader.py` implements it, and the claimed fail-open
cannot occur: on a core pinned at 0.32.x the endpoints plugin is refused at
load by its own `>=0.34.0` floor, so no credential is ever issued and there is
no enrolled PC whose shared key an older dispatcher could wrongly accept.
## Drift that no ADR covers
Recorded because it is real and someone will look for it here.
**`docs/openapi.json` is a release behind the code.**
`shopdb/core/api/docs.py:18` pins `_SPEC` to that file and serves it through a
vendored Redoc at `/api/docs`, specifically for the air-gapped production box.
The tracked spec holds 377 paths and `info.version` is still `0.12.0`. It
contains zero paths matching `enroll` or `bootstrap`, while
`git diff 2ac2e6d33..gh-wj -- 'plugins/*/api/*.py' 'shopdb/core/api/*.py'` adds
24 route decorators, including the whole enrollment surface
(`plugins/endpoints/api/enrollment.py:289`, `:580`, `:657`, `:752`, `:773`,
`:827`, `:853`, `:1095`, `:1135`, `:1233`, `:1253`) and the bootstrap-token
surface (`plugins/endpoints/api/bootstrap.py:419`, `:474`, `:485`, `:506`,
`:530`). Where the spec does describe a route that changed, it describes the old
behaviour: `/api/plugins/settings-routes` is documented as
"**Auth:** none (jwt-optional decorator, deliberately anonymous...)" with
`"security": [{}, {"bearerAuth": []}]`, while the shipped handler is
`@require_permission('settings.view')`.
Neither ADR-001 nor ADR-002 mentions `openapi.json` or `api-inventory.json`, so
no stated rule is broken. The drift also cannot be detected or repaired from
this branch: `docs/api-inventory.json` is not tracked here, so
`tests/test_api_inventory_parity.py` skips and `scripts/gen_openapi.py` has no
input.
**`CLAUDE.md` in this repo contradicts the repo.** Line 3 and line 9 describe
the database as "PostgreSQL 15+", while `docker-compose.yml:20` runs
`mysql:8.4` and `.env.example:29` is a `mysql+pymysql://` URL. Line 31 states
"**Columns**: snake_case (e.g., `serial_number`, `last_seen`)", which is the
opposite of `CONTRIBUTING.md:17` ("Column names: lowercase, concatenated,
singular. No dashes, no underscores") and of what the gate enforces at
`scripts/check-naming-and-style.sh:236`. Line 119 carries the ADR-016 device
token misattribution described above. `CLAUDE.md` is outside this folder and was
not edited.
## Scope note
This page records ADR conformance only. The same review produced security
findings against the per-PC credential surface, the kiosk network admission and
the one-line bootstrap; those are not ADR matters and are recorded in
`Security-Audit-2026-09-11.md`, refreshed on the same date and the same commit.
Violation 3 on this page and finding 31 on that one are the same ADR-016
misattribution seen from two angles. The highest of the security findings, for
cross-reference, is that `install.ps1` writes the fleet shared key to
`HKLM\SOFTWARE\GE\ShopDB` with no ACL
(`plugins/endpoints/api/siteagent.py:328`, helper generated at `:394-411`, no
`SetAccessControl` or `RegistrySecurity` anywhere in the generated installer),
where any logged-on user can read it - a consequence the repo states itself at
`.github/GE-ENFORCE-COLLECTOR-FIX.html:377-380`.
## References
- ADR set: `docs/adr/ in the product repo` (NOT on this branch)
- `scripts/check-naming-and-style.sh` - the only ADR-aware gate in this repo
- `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md` - the rollout runbook
- ADR-019-Plugin-Dynamic-Settings-Routes.md - the one ADR-shaped file here
- Security-Audit-2026-09-11.md - the security half of the same review
- Architectural-Findings-2026-09-11.md - re-checked 2026-09-18
- Maintenance-Footprint-Baseline.md - re-checked 2026-09-18

638
docs/wiki-drafts/Architectural-Findings-2026-09-11.md
View File

@@ -1,443 +1,405 @@
# Architectural Findings - September 11, 2026
**Scope:** Full codebase analysis via Gortex semantic code intelligence
**Severity:** 22 findings (6 critical, 6 high, 10 medium)
**Status:** Documented for remediation planning
**Scope:** Full codebase analysis via Gortex semantic code intelligence
**Original severity claim:** 22 findings (6 critical, 6 high, 10 medium)
**Status:** RE-CHECKED against branch gh-wj at 35d4fcd41 on 2026-09-18. Most of
the original findings do not survive a read of the code. This page keeps them,
with the verdict and the evidence, so nobody acts on a claim that was never
true.
## Executive Summary
## How to read this page (added 2026-09-18)
Gortex semantic analysis identified architectural issues in plugin boundaries, dependency management, and data schema consistency. Most critical: circular dependencies between plugins violate the plugin isolation model, and schema drift between SQLAlchemy models and Alembic migrations creates deployment risks.
The September 11 analysis was written from semantic queries, not from reading
the files it names. Eight of its claims cite code that does not exist on this
branch: a `machines` plugin that imports `computers`, a migration that spells a
column `last_reported_date`, a ServiceNow plugin, `MACHINE_TYPES = ['Lathe',
'Mill', 'CNC']`, a hardcoded `['Bldg 300', 'Bldg 400']`, Vuex, an absent CSP
header, an unhandled `requests.post`. Each of those was grepped for on
2026-09-18 and returns nothing.
**Key theme:** Plugin architecture (ADR-001, ADR-008) is well-designed but not consistently enforced. Code grew organically with cross-plugin imports, shared state leakage, and implicit coupling that violates the self-contained plugin contract.
Every finding below now carries one of four verdicts:
- **REFUTED** - the cited code does not exist, or the opposite is in the tree.
- **STALE** - it was true once; the tree has since fixed it.
- **HOLDS (narrowed)** - something real is there, but smaller or different than
the original text claims.
- **UNVERIFIED** - cannot be settled from this repository.
Effort and priority numbers in the original text were estimates with no stated
basis. They are left where they sit for findings that still hold and struck
where the finding does not.
## Critical Findings (P0)
### 1. Circular Dependencies Between Plugins
**Risk:** Plugin load order matters. Enabling/disabling plugins can break others.
**Verdict: REFUTED as stated. HOLDS (narrowed) as a deferred-import coupling.**
**Evidence:**
- `computers` plugin imports from `machines` plugin: `from plugins.machines.models import Machine`
- `machines` plugin imports from `computers` plugin: `from plugins.computers.models import Computer`
- `printers` plugin imports from `computers` plugin
- `endpoints` plugin imports from `computers`, `printers`, and `usb` plugins
**Original claim:** `computers` imports `machines` and `machines` imports
`computers`; import errors at runtime if plugins load in the wrong order; 25+
violations.
**Impact:**
- Violates plugin isolation (ADR-001: plugins are independent extensions)
- Disabling `computers` breaks `machines`, `printers`, `endpoints`
- Plugin dependency chain not declared in plugin metadata
- Import errors at runtime if plugins loaded in wrong order
**What the code shows (2026-09-18):**
**Root cause:** Plugins use each other's models for relationships (e.g., Computer controls Machine).
- `grep -rn "^from plugins\." plugins/ --include=*.py` returns **nothing**.
There is not one module-level cross-plugin import in the tree, so there is no
load-order failure mode and no import cycle at import time.
- `grep -rn "from plugins.computers" plugins/machines/` returns **nothing**.
The `machines` plugin does not reference `computers` at all, so the cycle the
finding is built on does not exist.
- 52 cross-plugin imports do exist, and every one of them is inside a function
body, resolved on call. `plugins/computers/plugin.py:414` and `:1166` reach
`plugins.machines.models` that way; `plugins/endpoints/api/routes.py:235` and
fourteen more reach `plugins.computers.models`.
**Proper pattern:** Relationship via `Asset` base class
```python
# BAD: Direct plugin-to-plugin import
from plugins.machines.models import Machine
**What is actually there:** a mutual reference pair between `computers` and
`endpoints`, both deferred:
# GOOD: Relationship via Asset ID, resolve at runtime
machine_asset_id = db.Column(db.Integer, db.ForeignKey('asset.id'))
machine = relationship('Asset', foreign_keys=[machine_asset_id])
- `plugins/computers/api/policy_routes.py:44` imports
`plugins.endpoints.commands.elevated_permission_for`, unguarded.
- `plugins/endpoints/commands.py:477` imports
`plugins.computers.services.policyresolver.effective_policy_for`, wrapped in
`try/except` that falls back to the base allowlist (`commands.py:475-481`).
# In plugin manifest, declare dependency
dependencies = ['machines'] # Explicit, not implicit
```
So one side of that pair degrades when its partner is absent and the other
raises. The same asymmetry appears at
`plugins/endpoints/api/agentauth.py:118`, where `resolve_hostname` imports
`plugins.computers.models.Computer` with no guard: an endpoints-without-
computers site fails there rather than returning "unknown hostname".
**Remediation:**
1. Audit all `from plugins.<other>` imports (25+ violations found)
2. Replace with Asset-based relationships or declared dependencies
3. Enforce via linter: plugins can import from `shopdb.api`, not `plugins.*`
**Effort:** 60 hours (high impact, requires refactoring multiple plugins)
**Priority:** Q1 2027 (blocks plugin catalog ADR-013 lean builds)
**What to do instead of the original remediation:** nothing about load order.
The open question is narrower and worth one pass: decide whether a deferred
cross-plugin import must be guarded, and guard the unguarded ones. A linter
rule banning `plugins.*` imports outright would break the working policy and
enrollment paths listed above, which are the product's actual design.
### 2. Undeclared Import Dependencies
**Risk:** Plugins import from `shopdb.core` internals instead of stable `shopdb.api` surface.
**Verdict: REFUTED.**
**Evidence:**
- 18 plugins import from `shopdb.core.models` directly (not via `shopdb.api`)
- 12 plugins import from `shopdb.core.utils` (internal helper modules)
- 7 plugins import from `shopdb.core.services` (not part of plugin API)
**Original claim:** 18 plugins import `shopdb.core.models`, 12 import
`shopdb.core.utils`, 7 import `shopdb.core.services`.
**Impact:**
- Plugins break when core refactors internals
- No API versioning or stability guarantee
- Circular imports between core and plugins
**What the code shows:**
`grep -rn "from shopdb\.\(core\|utils\|extensions\|services\)" plugins/
--include=*.py` returns **0 hits**. Not one plugin deep-imports core. The only
`shopdb.*` import in `plugins/` outside `shopdb.api` is
`shopdb.plugins.alembic_template` from migration `env.py` files, which is the
shared Alembic runner ADR-008 sanctions.
**Plugin API contract (ADR-001):** Plugins MUST import from `shopdb.api` only
```python
# GOOD: Stable plugin API
from shopdb.api import db, Asset, BasePlugin, success_response
ADR-001's rule that `shopdb.api` is the only core surface a plugin may import is
being kept. The thing that is NOT enforced is raw SQL against core tables: see
finding 3a below.
# BAD: Internal implementation details
from shopdb.core.models.asset import Asset # Internal path
from shopdb.core.services.search import SearchService # Not exported
```
### 2a. Raw SQL against core tables from a plugin (added 2026-09-18)
**Remediation:**
1. Audit imports: `grep -r "from shopdb.core" plugins/`
2. Export needed classes via `shopdb.api.__init__.py`
3. Update plugin imports to use `shopdb.api`
4. Add linter rule: plugins cannot import `shopdb.core.*`
**Verdict: HOLDS (new, narrow).**
**Effort:** 40 hours
**Priority:** Q4 2026 (enables core refactoring without breaking plugins)
`plugins/computers/plugin.py:2080-2091` runs
`db.session.execute(db.text(...))` joining core `assets`, `assetrelationships`
and `assettypes` inside the `flask computers retype` CLI command. `Asset`,
`AssetType`, `AssetRelationship` and `RelationshipType` are all exported from
`shopdb.api` (`shopdb/api/__init__.py:47-48`, `:424`, `:438-439`), so no
missing export excuses it, and `plugins/endpoints/api/routes.py:3981-3986` is
the same pattern already corrected for `BusinessUnit`. The contract note at
`shopdb/__init__.py:135-142` names the reason: a raw query is coupling to the
core schema that no contract bump can catch.
Blast radius: one admin CLI command, no serving route. Import rule not
literally broken. Worth fixing because it is the kind of thing that gets copied.
### 3. Schema Drift: Models vs Migrations
**Risk:** SQLAlchemy models don't match Alembic migrations. Deployed schema differs from code.
**Verdict: REFUTED as stated. HOLDS (narrowed) to one column.**
**Evidence (examples):**
- `Computer.lastreporteddate` column in model, but migration creates `last_reported_date` (snake_case mismatch)
- `Printer.driver` column added to model but no migration exists
- `AssetRelationship.label` column in migration but not in model
- 14 tables with mismatched column names or missing columns
**Original claim:** `Computer.lastreporteddate` in the model but
`last_reported_date` in the migration; `Printer.driver` with no migration;
`AssetRelationship.label` in a migration but not the model; 14 tables affected.
**Impact:**
- Fresh install (runs migrations) has different schema than ORM expects
- Queries fail: `Column 'lastreporteddate' does not exist in table`
- Adding data via ORM writes to wrong column
- Impossible to tell what production schema actually is (migrations are source of truth, but models are what dev sees)
**What the code shows:**
**Root cause:**
- Naming convention changed (ADR: lowercase concatenated, no underscores)
- Manual migrations written with old convention
- No validation that migrations match models
- `grep -rn "last_reported_date" --include=*.py .` returns **nothing**. The
model spells it `lastreporteddate` (`plugins/computers/models/computer.py:85`)
and so does every migration.
- `grep -rPn "sa\.Column\(\s*'[a-z]+_[a-z_]+'" plugins/*/migrations/versions/*.py
migrations/versions/*.py` returns **nothing**. There is no snake_case column
name in any migration in the tree.
- There is no `shopdb/models/` directory to hold the claimed
`AssetRelationship.label`; core models live under `shopdb/core/`.
**Remediation:**
1. Generate fresh migration from current models: `flask db migrate`
2. Compare to existing migrations, find diffs
3. Write fixup migration to align schema with models
4. Add test: `test_migrations_match_models()` validates ORM vs DB schema
5. Enforce naming convention in migration template
**The one real instance:** `plugins/notifications/models/notification.py:158`
declares `alertscope = db.Column(db.JSON, nullable=True)` while the migration
that creates it, `plugins/notifications/migrations/versions/0006_add_fleet_alerts.py:45`,
creates `sa.Column('alertscope', sa.Text, nullable=True)` and its own docstring
at line 8 calls it "the alertscope TEXT field (JSON string)". Commit f484be8a3
fixed the two neighbouring columns to agree with 0006 and left this one. Nothing
breaks on MySQL 8 - the SQLAlchemy JSON type serialises into TEXT and reads back
- and consumers already normalise both shapes
(`plugins/endpoints/api/routes.py:4011-4020`). The cost is that the model is not
a true description of the deployed column, which is what autogenerate reads.
**Effort:** 80 hours (high risk, requires careful schema analysis)
**Priority:** Q4 2026 (CRITICAL - blocks confidence in production schema)
**Related gap that is real:** the naming gate cannot see migrations at all.
`scripts/check-naming-and-style.sh:236` greps
`^\s+[a-z]+_[a-z_]+\s*=\s*db\.Column`, which only matches a model attribute
assignment; migrations write `sa.Column('name', ...)` with the name as a
positional string. That is how the shipped build of
`plugins/endpoints/migrations/versions/0021_client_certificates.py` created
`client_cert_fingerprint`, `cert_issued_at` and `cert_expires_at` and passed the
gate (`git show 5cd533239:plugins/endpoints/migrations/versions/0021_client_certificates.py`,
lines 20-28). The tree is conformant today, because 0021 and 0022 are now
no-ops and 0023 renamed the survivor; the gate simply would not notice the next
one.
### 4. Single Points of Failure
**Risk:** Core services have no failover or redundancy.
**Verdict: UNVERIFIED from this repository, and partly STALE.**
**Evidence:**
- Single database instance (no replication)
- Single Flask app instance (gunicorn workers on one server)
- Single Docker host (no orchestration)
- No load balancing
Replication, load balancing and orchestration are properties of the reference site
deployment, not of this tree, so nothing here can confirm or deny them. Two
parts of the original text are wrong about what the repo ships:
**Impact:**
- Database outage = full system outage
- App server failure = downtime until manual restart
- Hardware failure = data loss without recent backup
- Restart policy is configured, not absent: `docker-compose.yml:24`, `:104` and
`:120` set `restart: unless-stopped`, and three services carry `healthcheck:`
blocks (`:38`, `:112`, `:219`).
- Backups are no longer "manual mysqldump weekly, stored on the same server".
See finding 4a.
**Mitigation (current):**
- Weekly manual backups (mysqldump)
- VM snapshots before deploys
- Docker restart on failure (docker-compose restart policy)
The underlying point - one database, one app host, air gap limits the options -
is a statement about the site, and belongs in a deployment document rather than
a code findings page. The 200+ hour estimate has no basis in anything measured.
**Proper solution:** High availability architecture
- Database: MySQL master-replica replication
- App: Multiple gunicorn instances behind nginx load balancer
- Orchestration: Docker Swarm or Kubernetes for auto-restart
- Monitoring: Automated health checks + alerting
### 4a. Backups (added 2026-09-18, correcting the original text)
**Effort:** 200+ hours (infrastructure redesign)
**Priority:** Q2 2027 (production risk, but air-gapped environment has limited HA options)
**Verdict: STALE. The repo ships a backup system.**
**Note:** Air-gap deployment model limits cloud HA solutions. On-premise HA requires hardware investment.
`scripts/backup/` contains `backup.sh`, `restore.sh`, `status.sh`, `watch.sh`,
`simulate-restore.sh` and `test-restore-datarev.sh`. Retention is enforced by
restic (`scripts/backup/backup.sh:134-141`: `--keep-daily 5 --keep-weekly 4
--keep-monthly 6 --prune`), destinations include SMB and S3 as well as local
(`scripts/backup/README.md`), each set carries `manifest.json` with schema
revisions for compatibility checking, and restore runs from the Settings >
Backups UI with a schema compatibility check. A restore-simulation harness is in
the tree.
### 5. Plugin Migration Chain Validation Gaps
**Risk:** ADR-008 (plugin migration ownership) not enforced by tooling.
**Verdict: STALE as written, and the credited fix is the wrong artifact.**
**Evidence:**
- Manual inspection required to verify linear chains
- Plugin migrations can create branches (multiple head revisions)
- `PLUGIN_TABLE_OWNERS` order not validated against actual dependencies
Validation is in the test suite, in this repository, and has been for longer
than the `/migration-check` skill: `tests/test_plugin_migrations.py` holds nine
tests covering owners-versus-models, chain shape, no-op anchors and idempotency,
and pins each chain's head in `EXPECTED_HEAD_REVISION`
(`tests/test_plugin_migrations.py:125`). `tests/test_plugins/test_endpoints_credentials_migration.py`
runs the 0020-0023 chain twice from both a legacy and a fresh schema, plus
downgrade and re-upgrade. The endpoints chain today is 24 revisions with one
root and one head and no dangling `down_revision`.
**Example violation:**
```
plugins/computers/migrations/versions/
001_initial.py (down_revision=None)
002_add_access.py (down_revision=001)
003_add_policies.py (down_revision=001) # BRANCH! Two revisions point to 001
```
The branch example in the original text (`003_add_policies.py` with
`down_revision=001`) is invented; no such file exists under
`plugins/computers/migrations/versions/`, which holds 6 revisions.
**Impact:**
- Alembic upgrade fails with "Multiple head revisions" error
- Plugin can't be enabled until migration chain fixed
- Hard to diagnose (requires reading migration files manually)
**Proper validation:** `/migration-check` skill (created 2026-09-11)
- Checks: Each plugin has linear chain (no branches)
- Checks: `down_revision` references exist
- Checks: `PLUGIN_TABLE_OWNERS` order matches declared dependencies
- Status: Implemented, run via pre-commit hook
**Effort:** DONE (skill created)
**Priority:** Resolved 2026-09-11
**The real gap, which the original text missed:** no test asserts that a plugin
chain touches only the tables it owns. `plugins/endpoints/migrations/versions/0023_endpoint_credentials.py`
declares `plugin_name = 'endpoints'` (line 33) and then runs
`op.drop_index(..., table_name='assets')` at line 73 and `op.drop_column('assets',
column)` at line 76 against the core `assets` table, cleaning up after the
shipped 0021 which had added those columns from the same plugin chain. `assets`
is not, and cannot be, in `PLUGIN_TABLE_OWNERS['endpoints']`
(`shopdb/plugins/alembic_template.py:57-63`), so the metadata ownership filter
never saw it. Every step is guarded on the live schema and the forward direction
is clean, so nothing is broken; nothing stops the next one either.
### 6. Implicit Database Connection Sharing
**Risk:** Plugins share `db.session` without transaction isolation.
**Verdict: REFUTED as a blanket claim.**
**Evidence:**
- Multiple plugins write to database in same request
- No explicit transaction boundaries
- Rollback in one plugin affects others
Savepoints are in use where concurrency actually bites:
`plugins/endpoints/plugin.py:1318`, `:1387` and `:1692` all wrap work in
`with db.session.begin_nested():`, and the application-delta path reconciles a
unique-constraint collision inside that savepoint rather than sinking the
check-in. The original text's example - a printers rollback silently discarding
a computers write in the same request - is not backed by any cited call site,
and none was found.
**Example:**
```python
# computers plugin writes Computer
computer = Computer(hostname='PC1')
db.session.add(computer)
# printers plugin writes Printer (same session!)
printer = Printer(name='Printer1')
db.session.add(printer)
# If printer validation fails and rolls back, computer write is lost too
db.session.rollback()
```
**Impact:**
- Plugin A's failure can undo Plugin B's work
- No atomicity guarantee per-plugin
- Hard to debug (cross-plugin transaction interactions)
**Best practice:** Explicit transaction scopes per plugin
```python
from shopdb.api import db
with db.session.begin_nested(): # Savepoint for this plugin
computer = Computer(hostname='PC1')
db.session.add(computer)
db.session.flush()
# If exception here, only this savepoint rolls back
```
**Remediation:**
1. Audit plugin DB writes: find implicit `db.session.add()` without savepoint
2. Wrap in `begin_nested()` savepoints or explicit `commit()`
3. Document transaction patterns in PLUGIN-DEVELOPMENT.md
**Effort:** 40 hours
**Priority:** Q1 2027 (data integrity risk, but low frequency in practice)
A per-plugin transaction policy is still not written down anywhere. That is a
documentation item for PLUGIN-DEVELOPMENT.md, not a critical data-integrity
finding.
## High Priority Findings (P1)
### 7. Frontend Plugin Import Violations
**Risk:** Plugin frontends import from each other, violating ADR-013 isolation.
**Verdict: STALE. Enforced and passing.**
**Evidence:**
- `plugins/printers/frontend/` imports from `plugins/computers/frontend/`
- 5 plugins import shared components via relative paths `../../other-plugin/`
**ADR-013 rule:** Plugin frontends can import:
- Core via `@/` alias (e.g., `@/components/Shared.vue`)
- Own tree via `./` or `../` within plugin
- NOT other plugin trees
**Proper pattern:**
```javascript
// BAD: Cross-plugin import
import PCSelect from '../../computers/frontend/components/PCSelect.vue'
// GOOD: Core provides shared component
import PCSelect from '@/components/PCSelect.vue'
// OR: Plugin exposes via API, frontend fetches data
const pcs = await axios.get('/api/computers/')
```
**Enforcement:** `check-naming-and-style.sh` now checks for escaping imports
```bash
grep -rPn "(import|from)\s+['\"]([^'\"]*\.\./\.\./|[^'\"]*/plugins/)" \
plugins/*/frontend/
```
**Remediation:**
1. Move shared components to `frontend/src/components/`
2. Update plugin imports to use `@/` alias
3. Run pre-commit check to catch violations
**Effort:** 20 hours
**Priority:** Q4 2026 (blocks lean builds)
**Status:** Linter rule added 2026-09-11 (commit 4943e182c3)
`scripts/check-naming-and-style.sh` runs "Checking for cross-plugin / escaping
imports in plugin frontends" and the whole script exits 0 on this branch. The
established pattern is a core component consumed through the `@/` alias: 97
plugin frontend files already import that way, and the kiosk allowlist editor
added in this range follows it
(`frontend/src/components/NetworkAllowlistPicker.vue`, consumed at
`plugins/usb/frontend/views/USBSettings.vue:108` and
`plugins/printedparts/frontend/views/PrintedPartsSettings.vue:164`).
### 8. Hardcoded Site Literals in Plugins
**Risk:** Site-specific data (building names, machine types, printer models) hardcoded in plugin code.
**Verdict: REFUTED as stated. HOLDS (narrowed) as a gate coverage gap.**
**Evidence (violations of ADR-015):**
- `plugins/machines/models.py` has `MACHINE_TYPES = ['Lathe', 'Mill', 'CNC']` hardcoded
- `plugins/printers/api/routes.py` validates `building in ['Bldg 300', 'Bldg 400']`
- 12 plugins have site-specific literals
`grep -rn "MACHINE_TYPES" . --include=*.py` and
`grep -rn "Bldg 300\|Bldg 400" . --include=*.py` both return **nothing**. The
ADR-015 block in `scripts/check-naming-and-style.sh:392-462` runs and passes,
and a control pattern over the same directories returns 1995 lines, so the
pipeline is not vacuous.
**ADR-015 rule:** Site data MUST live in:
- `shopdb/core/settings/` - Settings table defaults
- `shopdb/core/seed/` - Initial data SQL
- `scripts/site_imports/` - Import tools for site-specific data
**Proper pattern:**
```python
# BAD: Hardcoded site data
MACHINE_TYPES = ['Lathe', 'Mill', 'CNC']
# GOOD: Setting with neutral default
machine_types = Setting.get('machines_allowed_types', default='[]')
MACHINE_TYPES = json.loads(machine_types)
```
**Enforcement:** `check-naming-and-style.sh` scans for literal building names, FQDNs, network ranges
**Remediation:**
1. Extract literals to Settings table
2. Update plugins to read from Settings
3. Seed data with site-specific values during setup
**Effort:** 30 hours
**Priority:** Q1 2027 (blocks multi-site deployments)
**The narrowed finding:** the ADR-015 scan is scoped at
`scripts/check-naming-and-style.sh:436-440` to `plugins/ shopdb/ scripts/
deploy/ frontend/src/ tools/` with `--include` on
`.py/.vue/.js/.ps1/.psm1/.sh/.iss/.json/.html`. `agent/` is not in that list and
`.cs` is not an included extension, so roughly 2,500 lines of C# added in this
range - `AgentCredentials.cs`, `AgentHttpFactory.cs`, `EnrollmentService.cs`,
`AgentPipeServer.cs`, `ProtectedFileSystem.cs`, `StrictModePreamble.cs`, plus
`agent/appsettings.json` - ship to a sister site unscanned. Re-running the
assembled site patterns by hand over `agent/` (excluding bin/obj) finds one hit,
prose in `agent/ShopDbEndpointAgent/ShopDbEndpointAgent.csproj:108`, which
ADR-015 permits. The tree is clean; the gate would not notice if it stopped
being clean. Note this is a gap in ADR-015's own stated scope, not a breach of
it: the 2026-08-14 amendment names those six directories.
### 9. Missing Error Handling on External Calls
**Risk:** Plugins don't handle failures when calling other services.
**Verdict: REFUTED.**
**Evidence:**
- ServiceNow plugin: no timeout on HTTP requests, no retry logic
- LDAP sync: no handling of connection failures
- Email plugin: assumes SMTP always succeeds
The plugins the finding names do not exist. `ls plugins/` gives: applications,
backups, computers, employees, endpoints, geenforce, guides, knowledgebase,
machines, measuringtools, network, notifications, printedparts, printers,
slides, tools, usb, warranty. There is no ServiceNow plugin, no LDAP sync
plugin and no email plugin; mail is core (`shopdb/utils/mailer.py`).
**Impact:**
- Hung requests block worker threads
- Transient failures cause permanent data loss
- No user feedback when external service down
Every outbound HTTP call in the tree carries a timeout and an exception handler:
**Best practice:** Timeouts, retries, circuit breakers
```python
# BAD: No timeout
response = requests.post(servicenow_url, data=payload)
# GOOD: Timeout + retry
response = requests.post(servicenow_url, data=payload, timeout=10)
if response.status_code >= 500:
# Retry with exponential backoff
time.sleep(2 ** attempt)
```
**Remediation:**
1. Audit external calls: `grep -r "requests\.(get|post)" plugins/`
2. Add timeouts (default 10s)
3. Add retry logic for transient failures (5xx errors)
4. Log failures for monitoring
**Effort:** 30 hours
**Priority:** Q4 2026 (production stability)
- `plugins/warranty/services/providers.py:129` and `:157` - `timeout=HTTP_TIMEOUT`,
`except requests.RequestException` raising `WarrantyLookupError`.
- `plugins/printers/services/zabbix_service.py:95` and `:126` -
`timeout=self.REACHABLE_TIMEOUT` / `self.API_TIMEOUT`, caught.
- `shopdb/utils/mailer.py:238` - `timeout=10`, caught, returns `(False, error)`
and never raises.
### 10-12. Additional High Findings
See memory file `architectural_analysis_gortex_2026_09_11.md` for:
- Inconsistent permission checking across plugins
- Plugin state leakage via module-level globals
- Missing database connection pool tuning
The original text deferred these to a memory file rather than citing code:
"inconsistent permission checking across plugins", "plugin state leakage via
module-level globals", "missing database connection pool tuning".
The third is REFUTED. The pool is sized on purpose and env-driven:
`shopdb/config.py:20-22` (`DB_POOL_SIZE` 10, `DB_MAX_OVERFLOW`, `DB_POOL_TIMEOUT`
5) and `:48-72`, with the reasoning for the non-default `pool_timeout` written
in place, and `docker-compose.yml:172-193` documents the
`workers x (pool_size + max_overflow)` arithmetic against MySQL's
`max_connections`.
The other two were not re-checked, because neither cites a file. They should be
restated with a file:line or dropped.
## Medium Priority Findings (P2)
### 13. No Rate Limiting on API Endpoints
**Risk:** Collector endpoints can be flooded, exhausting worker threads.
**Verdict: HOLDS (narrowed). Login is limited; two kiosk surfaces are not.**
**Mitigation:** See Security-Audit-2026-09-11.md (login rate limiting planned Q1 2027)
Login rate limiting exists and is not "planned Q1 2027": `shopdb/core/api/auth.py:35`
`_login_ratelimited()`, keyed per host at `:52`, applied at `:80`, with
`tests/test_core/test_auth_ratelimit.py` over it. Enrollment has its own limiter
(`plugins/endpoints/api/ratelimit.py`, 300 per 60 seconds per IP applied at
`plugins/endpoints/api/enrollment.py:306`).
What has none: the USB kiosk routes. `grep` for `limiter` or `ratelimit` across
`plugins/usb/` returns nothing, while `plugins/usb/api/selfhosted.py:346-371`
resolves a posted badge to an employee name and returns it. That is an
unauthenticated, unthrottled badge-to-name lookup for any caller inside the
configured kiosk CIDRs.
Both limiter counters live in per-process `SimpleCache`
(`plugins/endpoints/api/ratelimit.py:10-11`), so at the shipped four gunicorn
workers the effective budget is four times the configured maximum.
### 14. Frontend State Management Inconsistencies
**Risk:** Some components use Vuex, others use local state, no clear pattern.
**Verdict: REFUTED.**
**Impact:** Hard to track state flow, data duplication, synchronization bugs
**Recommendation:** Standardize on Pinia (Vue 3 state management) or document when to use local vs global state
`grep -rn "vuex" frontend/src/` returns **nothing**, and `frontend/package.json:28`
declares `"pinia": "^2.1.0"`. There is no Vuex-versus-Pinia split to
standardise.
### 15. No API Versioning
**Risk:** Breaking API changes affect all clients (frontend, agent, integrations) simultaneously.
**Verdict: HOLDS.**
**Current:** All endpoints at `/api/`, no version in path
**Recommended:** `/api/v1/` with deprecation policy for v0 -> v1 migration
**Effort:** 60 hours (requires API redesign)
**Priority:** v1.0 release
`grep -rn "api/v1" shopdb/ --include=*.py` returns nothing; every route is
mounted under `/api/`. This is a v1.0 decision, not a defect, and ADR-002 already
carries the versioning story for the plugin contract. The 60-hour estimate has
no stated basis.
### 16-22. Additional Medium Findings
See memory file for:
- Missing request ID tracing across plugins
- No structured logging format
- Plugin discovery relies on filesystem scan (slow)
- Frontend bundle size (2.1MB) larger than ideal
- No content security policy headers
- Database connection pool too small for peak load
- Missing health check endpoints for monitoring
Re-checked on 2026-09-18:
## Remediation Priority
- **No content security policy headers: REFUTED as stated.** `shopdb/__init__.py:340`
sets `Content-Security-Policy: frame-ancestors 'self'` on every response, plus
`X-Content-Type-Options`, `X-Frame-Options` and `Referrer-Policy` at `:334-337`;
uploads carry a stricter `default-src 'none'; sandbox`
(`shopdb/utils/uploads.py:13`), pinned by `tests/test_core/test_security_headers.py`.
The accurate, narrower statement is that there is no `script-src` policy, and
the code says so in place and says why.
- **Missing health check endpoints: REFUTED.** `shopdb/core/api/dashboard.py:318`
is `GET /api/dashboard/health`.
- **Database connection pool too small: REFUTED.** See 10-12 above.
- **No request ID tracing: HOLDS.** No request-id plumbing was found in
`shopdb/__init__.py`.
- **No structured logging format: not re-checked.**
- **Plugin discovery relies on a filesystem scan: HOLDS, by design.** ADR-003
decides filesystem discovery under `plugins/`. "Slow" was never measured.
- **Frontend bundle size 2.1MB: UNVERIFIED.** No build was run for this page.
| Finding | Severity | Effort (hrs) | Priority | Status |
|---------|----------|--------------|----------|--------|
| Schema drift (models vs migrations) | CRITICAL | 80 | Q4 2026 | Planned |
| Circular plugin dependencies | CRITICAL | 60 | Q1 2027 | Planned |
| Undeclared import dependencies | CRITICAL | 40 | Q4 2026 | Planned |
| Migration chain validation | CRITICAL | 0 | DONE | /migration-check skill |
| Frontend plugin imports | HIGH | 20 | Q4 2026 | Linter added |
| Site literals in plugins | HIGH | 30 | Q1 2027 | Planned |
| Missing error handling | HIGH | 30 | Q4 2026 | Planned |
| Implicit DB session sharing | HIGH | 40 | Q1 2027 | Planned |
| Single points of failure | CRITICAL | 200+ | Q2 2027 | Infrastructure |
| API versioning | MEDIUM | 60 | v1.0 | Planned |
## Remediation Priority (revised 2026-09-18)
## Testing Performed
| Finding | Verdict | What is actually left to do |
|---------|---------|------------------------------|
| Circular plugin dependencies | REFUTED / narrowed | Decide whether deferred cross-plugin imports must be guarded; guard `agentauth.py:118` and `policy_routes.py:44` |
| Undeclared core imports | REFUTED | Nothing |
| Schema drift | REFUTED / narrowed | Align `notification.py:158` with migration 0006; teach the naming gate to read `sa.Column('...')` |
| Plugin chain touches core tables | HOLDS (new) | Add a test that a plugin chain writes only owned tables |
| Migration chain validation | STALE | Nothing; `tests/test_plugin_migrations.py` is the gate |
| Implicit DB session sharing | REFUTED | Document the transaction pattern in PLUGIN-DEVELOPMENT.md |
| Frontend plugin imports | STALE | Nothing; gate passes |
| Site literals in plugins | REFUTED / narrowed | Extend the ADR-015 scan to `agent/` and `.cs` |
| Missing external-call error handling | REFUTED | Nothing |
| Rate limiting | HOLDS (narrowed) | Throttle the USB kiosk badge lookup; decide whether per-process counters are acceptable at 4 workers |
| Vuex/Pinia split | REFUTED | Nothing |
| API versioning | HOLDS | v1.0 decision |
| Single points of failure | UNVERIFIED | Belongs in a deployment document |
## Testing Performed (2026-09-18 re-check)
**Gortex semantic analysis:**
```bash
gortex explore "find circular dependencies between plugins"
gortex explore "find undeclared imports from shopdb.core"
gortex explore "find schema mismatches between models and migrations"
# Every claim above was checked with a grep or a read on gh-wj at 35d4fcd41.
grep -rn "^from plugins\." plugins/ --include=*.py # 0 hits
grep -rn "from shopdb\.\(core\|utils\|extensions\|services\)" \
plugins/ --include=*.py # 0 hits
grep -rPn "sa\.Column\(\s*'[a-z]+_[a-z_]+'" \
plugins/*/migrations/versions/*.py migrations/versions/*.py # 0 hits
grep -rn "MACHINE_TYPES\|Bldg 300\|Bldg 400" . --include=*.py # 0 hits
grep -rn "vuex" frontend/src/ # 0 hits
grep -rn "api/v1" shopdb/ --include=*.py # 0 hits
bash scripts/check-naming-and-style.sh # exit 0
```
**Migration chain validation:**
```bash
/migration-check # Runs pytest tests/test_plugin_migrations.py
# PASS: All plugin migration chains are linear
```
**Frontend import validation:**
```bash
bash scripts/check-naming-and-style.sh
# Scans for plugin-to-plugin imports (../../ escaping)
# PASS: No violations after cleanup
```
## Recommendations
**Immediate (Q4 2026):**
1. Fix schema drift (models vs migrations) - CRITICAL
2. Enforce plugin API imports (no shopdb.core) - enables refactoring
3. Add error handling to external service calls - stability
**Short-term (Q1 2027):**
1. Break circular plugin dependencies - enables lean builds
2. Remove site literals from plugins - enables multi-site
3. Fix implicit DB session sharing - data integrity
**Long-term (Q2-Q4 2027):**
1. High availability architecture (if feasible in air-gap)
2. API versioning for v1.0 release
3. Frontend optimization (bundle size, state management)
No test suite was run for this re-check; every verdict above is static.
## References
- Gortex MCP Server: Semantic code intelligence for architectural analysis
- ADR-001: Asset as Platform Contract (plugin isolation model)
- ADR-003: Plugin Distribution (filesystem discovery)
- ADR-008: Plugin Migration Ownership (linear chains, PLUGIN_TABLE_OWNERS)
- ADR-013: Plugin Catalog and Lean Builds (frontend isolation)
- ADR-015: Site-Specific Configuration (no literals in code)
- Memory file: `.claude/memory/architectural_analysis_gortex_2026_09_11.md`
- ADR-Conformance-2026-09-18.md: the ADR-by-ADR result of the 2026-09-18 review,
including the fact that the ADR set is not in this repository

216
docs/wiki-drafts/Maintenance-Footprint-Baseline.md
View File

@@ -1,8 +1,37 @@
# Maintenance Footprint Baseline - 2026-09-11
**Total annual burden:** 936 hours/year
**Risk level:** 8 critical + 10 high operational risks
**Total annual burden:** 936 hours/year (estimate, see the note below)
**Risk level:** originally stated as 8 critical + 10 high operational risks
**Optimization potential:** 147-205 hours/year savings with automation
**Last checked against code:** 2026-09-18, branch gh-wj at 35d4fcd41
## About the numbers (added 2026-09-18)
Every hour figure on this page is an estimate with no recorded measurement
behind it. No time log, ticket export or deploy log is cited anywhere, and none
was found in the repository. Treat the hours as an ordering of where effort
goes, not as data. The same applies to the row counts and growth rates in the
Database Management section: no query output is cited and none could be taken
from a repository checkout.
What CAN be checked from the repository is which of the recommendations have
since been built. Several have. The sections below are annotated
accordingly, dated 2026-09-18, and the stale risk-matrix rows are corrected.
## What the tree actually contains (2026-09-18)
| Thing | Count | How counted |
|-------|-------|-------------|
| Plugins | 18 directories, 17 with a `manifest.json` | `ls -d plugins/*/`; `applications` carries no manifest |
| Plugin migration revisions | 70 across 17 chains | `ls plugins/*/migrations/versions/*.py` |
| Largest single chain | `endpoints`, 24 revisions | one root, one head, no branch |
| Core migration revisions | 58 | `ls migrations/versions/*.py` |
| Test files | 223 | `find tests -name "test_*.py"` |
| Top-level scripts | 29 (.sh/.py/.ps1/.mjs), plus `scripts/backup`, `scripts/dev`, `scripts/migration`, `scripts/loadtest`, `scripts/site_imports` | `ls scripts/` |
| Automation skills | 13 | `ls <assistant-config>/skills/*.md` minus README |
The "13 automation skills" claim later on this page is the one headline figure
that checks out exactly.
## Executive Summary
@@ -29,6 +58,19 @@ Baseline assessment of ShopDB operational burden across air-gapped production en
- Savings: 52 hrs/year (from 52 hrs to ~0 hrs, fully automated)
- Status: PLANNED for Q4 2026
**2026-09-18 update: half built, and the half that is built is manual.**
Retention exists as a service and a CLI, not as a cron job and not as a stored
procedure. `shopdb/core/services/auditretention.py` provides `configureddays()`,
`stats()` and `purge()`; `flask audit stats` (`shopdb/cli/__init__.py:1891`)
reports size against the configured window and `flask audit purge`
(`:1911`) deletes, with a dry-run preview first and `--days` to override for one
run. The window is the `audit_retentiondays` setting
(`shopdb/core/api/settings.py:1049`), and its default is **`'0'`, which means
keep everything forever**: nothing is deleted until an operator sets a value.
There is no scheduler in the application and no cron entry in the repository, so
the purge runs when somebody runs it. The remaining work is a value in Settings
> Site & Facility and a host cron line, not code.
**Endpoint events cleanup:** 26 hrs/year (HIGH RISK)
- Current: 450K rows, 127MB, no retention
- Growth: Process events every 30 seconds, app launches, USB connects
@@ -38,10 +80,32 @@ Baseline assessment of ShopDB operational burden across air-gapped production en
- Implementation: Similar cron job as audit log
- Savings: 26 hrs/year (fully automated)
**2026-09-18 update: partly built, per table rather than globally.** Operator
reports carry a retention window defaulting to 7 days
(`plugins/endpoints/api/routes.py:3619-3620`,
`endpoints.operatorreport.retentiondays`, 0 or less meaning keep forever), and
endpoint health transitions are trimmed to a high-water mark on write, keeping
every unresolved row regardless of age
(`plugins/endpoints/services/health.py:326`). The row counts and the 127MB
figure above were never sourced and are not verifiable from a checkout.
**Database backups:** 78 hrs/year (MEDIUM RISK)
- Current: Manual mysqldump weekly, stored on same server
- Current (as stated 2026-09-11): Manual mysqldump weekly, stored on same server
- Risk: No offsite copies, no restore testing, backup growth unbounded
- Backup size: 2.1GB compressed, growing 15% per quarter
- Backup size: 2.1GB compressed, growing 15% per quarter (unsourced)
**2026-09-18 update: STALE. This is built, and it is what the recommendation
below asks for.** `scripts/backup/` holds `backup.sh`, `restore.sh`, `status.sh`,
`watch.sh`, `simulate-restore.sh` and `test-restore-datarev.sh`, plus a
`Dockerfile.backup` and a compose file. Retention is enforced by restic:
`scripts/backup/backup.sh:134-141` runs `restic forget --keep-daily 5
--keep-weekly 4 --keep-monthly 6 --prune`. Destinations are configured in
Settings > Backups and include local, SMB and AWS S3, so offsite is a setting
rather than a manual copy. Each backup set carries `db.sql.gz`, the `instance/`
tree and a `manifest.json` of schema revisions, and restore validates schema
compatibility before it runs. A restore-simulation harness is in the tree.
Backup credential redaction landed in the same commit as audit retention
(`scripts/redact_backup_secrets.py`).
**Recommended:** Automated backup with rotation
- Daily incremental, weekly full, monthly archive
@@ -69,7 +133,7 @@ Baseline assessment of ShopDB operational burden across air-gapped production en
**Air-gap coordination:** 312 hrs/year (CRITICAL BURDEN)
- Current: Manual bundle creation, USB transfer, deploy in 3 environments
- Environments: dev PC (gitea) <-> work PC (GitHub) <-> VM (prod)
- Environments: dev PC (the internal git server) <-> work PC (GitHub) <-> VM (prod)
- Steps per deploy: 14 manual steps, 45-90 minutes each
- Frequency: 2-3 deploys/week = 104-156 deploys/year
@@ -111,6 +175,13 @@ logging:
```
- Savings: 52 hrs/year (fully automated, from manual cleanup)
**2026-09-18 update: STILL OPEN, and still the cheapest item on this page.**
`grep -c logging docker-compose.yml` returns 0: there is no `logging:` block on
any service. Three services do carry `restart: unless-stopped`
(`docker-compose.yml:24`, `:104`, `:120`) and three carry `healthcheck:` blocks
(`:38`, `:112`, `:219`), so the container lifecycle is managed; log size is not.
The 500MB/week growth figure was never sourced.
**Image rebuilds:** 52 hrs/year
- Current: Manual rebuild when dependencies change
- Frequency: ~20 rebuilds/year (Python deps, base image updates)
@@ -197,9 +268,57 @@ logging:
| Docker log disk fill | HIGH | High (3-6 mo) | Container crashes | Log rotation Q4 2026 |
| Deployment coordination errors | HIGH | Medium (2-3/year) | Failed deploys, downtime | Automation scripts implemented |
| Migration conflicts (ADR-008) | HIGH | Low | Plugin blocked | Migration-check skill |
| Agent certificate validation disabled | CRITICAL | Low | MITM attack | Per-endpoint cert pinning Q4 2026 |
| No login rate limiting | HIGH | Medium | Brute force attack | Rate limiting Q1 2027 |
| Unauthenticated collector endpoints | CRITICAL | N/A | FIXED 2026-09-11 | Device token auth |
| Agent certificate validation disabled | CORRECTED 2026-09-18 | n/a | n/a | Built. See note below |
| No login rate limiting | CORRECTED 2026-09-18 | n/a | n/a | Built. See note below |
| Unauthenticated collector endpoints | CORRECTED 2026-09-18 | n/a | n/a | Per-PC endpoint credentials, not device tokens. See note below |
| Docker log disk fill | HIGH | unmeasured | Container crashes | STILL OPEN, no `logging:` block in docker-compose.yml |
### Risk matrix corrections (2026-09-18)
**Agent certificate validation.** Not disabled. `AgentHttpFactory` is the only
`HttpClient` construction site in the agent, every channel runs
`ServerPinSet.Validate`, and `Validate` has no path returning true for an
unmatched pin (`agent/ShopDbEndpointAgent.Core/AgentHttpFactory.cs:291-331`). A
failed registry re-read returns null and keeps the existing pins rather than
silently unpinning (`:352-386`). The three SYSTEM PowerShell reporters refuse to
send at all when `ServerPins` is set and the check cannot be enabled
(`plugins/computers/client/Report-AssetToShopDB.ps1:228-247`, and the same block
in `Report-PrintersToShopDB.ps1` and `upgrade-shopdbagent.ps1:436-447`). The one
`ServerCertificateValidationCallback = {$true}` left in the tree is
`test-vm-connectivity.ps1:44`, a developer connectivity probe.
**Login rate limiting.** Built, and not "Q1 2027":
`shopdb/core/api/auth.py:35` `_login_ratelimited()`, keyed per host at `:52`,
applied at `:80`, with `tests/test_core/test_auth_ratelimit.py` over it. Note
the counter lives in per-process `SimpleCache`, so at four gunicorn workers the
effective budget is four times the configured maximum.
**Collector authentication.** The mechanism is a per-PC endpoint credential,
not a device token and not ADR-016. An enrolled PC sends
`X-Endpoint-Credential` (`plugins/endpoints/api/agentauth.py:1-18`); the server
stores only a sha256 hash plus a 12-character display prefix
(`plugins/endpoints/models/credential.py`); once a PC holds an active
credential the shared key is refused for it
(`plugins/endpoints/api/agentauth.py:129-134`). ADR-016's own mechanism -
`credentials.fetch`, `GET /api/collector/credentials`, Fernet at rest - is
**not built**: grepping `shopdb/` and `plugins/` for `credentials.fetch`,
`collector/credentials` and `Fernet` returns zero hits, matching ADR-016's own
"Implementation status, 2026-08-14". The rollout runbook is
`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`.
One boundary the runbook does not state: turning the shared key off does not
close the four legacy core collector routes. `/pc`
(`shopdb/core/api/collector.py:349`), `/apps` (`:387`), `/heartbeat` (`:459`)
and `/bulk` (`:494`) are decorated `@require_api_key`, which at `:154-164`
checks only `COLLECTOR_API_KEY` or a `collector.ingest`-scoped token and never
consults `agent_shared_key_refused`. Anything still holding the env shared key
can rewrite logged-in user, last boot time, serial number and installed-app
inventory for any hostname. Regenerating the key in the UI
(`plugins/endpoints/api/routes.py:1864-1884`) mints a new managed token and
clears the env fallback flag but does not change `COLLECTOR_API_KEY`, and
`_env_fallback_disabled(None)` returns False (`collector.py:60-74`), so the
legacy routes keep honouring the env key. Rotating `COLLECTOR_API_KEY` in
`.env` is the step that closes it.
## Automation Savings Summary
@@ -313,11 +432,23 @@ rsync -av $BACKUP_DIR/ /mnt/offsite-backups/shopdb/
### 5. check-naming-and-style.sh Optimization (DONE)
**Effort:** 2 hours
**Savings:** Script runtime 5x faster (10s -> 2s)
**Status:** Committed 2026-09-11 (commit 4943e182c3)
**Status:** In the tree and on by default. `USE_GIT_LS_FILES` defaults to `1`
(`scripts/check-naming-and-style.sh:24`) and the script takes the
`git ls-files -z` path whenever it is inside a work tree (`:26`, `:159`, `:183`,
`:218`), filtering out `migrations/versions/`, `frontend/dist/`, `staticdocs/`
and `.plugins-staged/`. Setting `USE_GIT_LS_FILES=0` falls back to recursive
grep with `--exclude-dir` for non-git checkouts. The whole script exits 0 on
gh-wj at 35d4fcd41.
**Impact:** 10-15 minutes saved per commit = ~26 hrs/year for team
**The 5x / 10s-to-2s claim is unsourced** and no timing was recorded for this
page. The savings line below it follows from it and is equally unsourced.
**Coverage note added 2026-09-18:** the root-anchored `^migrations/versions/`
filter does not exclude `plugins/*/migrations/versions/`, so plugin migrations
ARE scanned. They are not caught by the column rule, though: that rule greps
`^\s+[a-z]+_[a-z_]+\s*=\s*db\.Column` (`:236`), which only matches a model
attribute assignment, while migrations write `sa.Column('name', ...)`. The
column name is created in the one file the naming gate cannot read.
### 6. Pre-commit Hook Integration
@@ -349,17 +480,36 @@ pytest tests/ -x || exit 1
## Long-Term Improvements (Planned)
### 1. Agent Delta Updates (Q1 2027)
### 1. Agent Delta Updates - BUILT (2026-09-18 update)
**Current:** Agent POSTs full inventory (11KB) every 30 seconds
**Proposed:** POST only deltas (added/removed apps, IP changes) = ~500 bytes
**Original text:** "Agent POSTs full inventory (11KB) every 30 seconds.
Proposed: POST only deltas." Both halves are now wrong.
**Savings:**
- 99% reduction in network traffic (17GB/day -> 170MB/day for 500 endpoints)
- 99% reduction in database writes (only changed data)
- Performance: Collector API load reduced from 1.44M requests/day to ~14K writes/day
**The cadence was never 30 seconds.** The agent default is 900 seconds with up
to 60 seconds of jitter: `agent/ShopDbEndpointAgent/appsettings.json`
(`"cadenceseconds": 900`, `"jitterseconds": 60`) and
`agent/ShopDbEndpointAgent.Core/AgentConfig.cs:14` and `:31`. The 17GB/day and
1.44M-requests/day figures were derived from the wrong interval and should not
be quoted.
**Effort:** 40 hours (agent changes + server changes + testing)
**The delta is implemented, on the server side.** The agent still reports the
full installed-software list each cycle, but each entry carries a SHA-256
fingerprint of `displayname|displayversion|publisher`, and
`_replace_applications` (`plugins/endpoints/plugin.py:1122-1140`) compares
fingerprint sets instead of delete-all-insert-all: adds for new fingerprints,
deletes for departed ones, unchanged rows untouched (`:1277-1289`). The
docstring states the effect as roughly 100 database operations per check-in
down to about 2. It is guarded: fingerprint format validation, a delete-count
check that warns rather than aborting, a memory cap above 5000 apps, a legacy
fall-back for agents that send no fingerprints, and collision reconciliation
inside a savepoint (`:1318`, `:1387`). The unique `(assetid, appfingerprint)`
index was added NULL-filled by migration
`plugins/endpoints/migrations/versions/0020_*`, so index creation cannot fail on
a populated production table.
Remaining network-side work, if anyone wants it, is making the agent send only
the changed entries rather than the full list. The database win is already
taken.
### 2. Automated Staging Tests (Q4 2026)
@@ -369,14 +519,17 @@ pytest tests/ -x || exit 1
**Savings:** 50-100 hrs/year (from deployment coordination)
**Effort:** 20 hours (test suite + CI integration)
### 3. Per-Endpoint Certificate Pinning (Q4 2026)
### 3. Per-Endpoint Certificate Pinning - BUILT (2026-09-18 update)
**Current:** Certificate validation disabled (security risk)
**Proposed:** Each endpoint validates server cert against pinned thumbprint
**Savings:** Eliminates CRITICAL security risk
**Effort:** 40 hours (agent changes + deployment script + mass agent update)
**Status:** Planned (see Security-Audit-2026-09-11.md)
Pins are SPKI SHA-256, validated as 64 lowercase hex on the way in, enforced on
every agent channel, and owned by the registry rather than by `appsettings.json`
(`plugins/endpoints/client/Set-ShopDBServerPins.ps1` writes
`HKLM:\SOFTWARE\GE\ShopDB` `ServerPins`; the precedence is stated in
`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`, section "The registry
wins"). A CA-flagged certificate is pinned but deliberately NOT imported into
`LocalMachine\Root`, and the build warns the admin through an
`X-ShopDB-Build-Warning` header. See the risk-matrix correction above for the
enforcement call sites.
## Monitoring & Alerting
@@ -399,8 +552,11 @@ pytest tests/ -x || exit 1
## References
- Security Audit 2026-09-11: 4 critical + 6 high vulnerabilities, some fixed, some planned
- Architectural Findings 2026-09-11: 22 findings (6 critical, 6 high) from Gortex analysis
- Automation Skills: 13 skills created in `.claude/skills/` directory
- Deployment Documentation: `docs/deployment/` and `.claude/setup/` guides
- Architectural Findings 2026-09-11: re-checked 2026-09-18; most of the 22
findings are refuted or stale, see that page for the verdict on each
- ADR-Conformance-2026-09-18: which ADRs were checked in the 2026-09-18 review
and which hold
- Automation Skills: 13 skills created in the assistant skills directory
- Deployment Documentation: `docs/deployment/` and the assistant setup guides
- ADR-004: Deployment Topology (air-gapped architecture)
- ADR-008: Plugin Migration Ownership (linear chain requirement)

205
docs/wiki-drafts/README.md
View File

@@ -1,6 +1,18 @@
# Wiki Page Drafts - 2026-09-11
# Wiki Page Drafts - 2026-09-11, revised 2026-09-18
This directory contains drafted wiki pages based on work completed Q3 2026. Copy these to the GitHub wiki at https://github.com/ge-aero/shopdb-flask/wiki.
This directory contains drafted wiki pages based on work completed Q3 2026. Copy
these to the GitHub wiki at https://github.com/ge-aero/shopdb-flask/wiki.
**Revised 2026-09-18, branch gh-wj at 35d4fcd41.** A full review re-checked every
page here against the code. `Architectural-Findings-2026-09-11.md` was
substantially wrong and now carries a verdict and evidence per finding.
`Maintenance-Footprint-Baseline.md` is annotated where the tree has moved on.
`Security-Audit-2026-09-11.md` was refreshed with a verdict per finding and
thirteen new ones. `ADR-019-...md` had two internal contradictions corrected. A
new page, `ADR-Conformance-2026-09-18.md`, was added. The SECURITY.md draft below
had its ADR-016 section replaced: it described a decision ADR-016 never took and
banned the registry storage the reference site fleet depends on. Nothing here is ready to
paste without reading the 2026-09-18 notes inside each file first.
## New Pages to Create
@@ -19,6 +31,15 @@ This directory contains drafted wiki pages based on work completed Q3 2026. Copy
**Related commits:** fe09d2745e, d59d0c3194
**2026-09-18:** corrected against the shipped code. The "Backend: API Endpoint"
code block showed `@jwt_required(optional=True)` iterating
`get_enabled_plugin_instances()`; neither exists. It now shows
`shopdb/core/api/plugins.py:46-95` as it stands, gated on `settings.view`. The
Future Extensions section no longer calls the endpoint anonymous. Everything
else in the draft checked out, including the three constraints. Note
`docs/openapi.json` still documents this endpoint as anonymous, so publishing
the ADR without regenerating the spec leaves the two disagreeing.
---
### 2. Security Audit - September 11, 2026
@@ -54,12 +75,23 @@ This directory contains drafted wiki pages based on work completed Q3 2026. Copy
- Security/compliance: 78 hrs/year (scanning + updates)
**Automation:**
- 13 Claude Code skills created
- 13 assistant automation skills created (this figure checks out: `ls <assistant-config>/skills/*.md`)
- 147-205 hrs/year saved (from 936 baseline)
- 7 quick wins identified (4 hours - 6 hours effort each)
**Risk matrix:** 8 critical + 10 high operational risks
**2026-09-18:** annotated. Every hour figure and row count on that page is an
unsourced estimate and the page now says so. Four recommendations have since
been built and the page records what and where: audit-log retention (service,
CLI and setting, but default 0 and no scheduler), backups (restic with
5/4/6 retention, SMB and S3 destinations, restore simulation), agent delta
updates, and per-endpoint certificate pinning. Three risk-matrix rows were
stale and are corrected: certificate validation is not disabled, login rate
limiting exists, and the collector fix is per-PC endpoint credentials rather
than "device token auth". Docker log rotation is still genuinely missing:
`grep -c logging docker-compose.yml` returns 0.
---
### 4. Architectural Findings - September 11, 2026
@@ -83,6 +115,41 @@ This directory contains drafted wiki pages based on work completed Q3 2026. Copy
**Remediation timeline:** Q4 2026 - Q4 2027
**2026-09-18: do not publish this page as it stood.** Re-checking it against
gh-wj refuted most of it. There is no module-level cross-plugin import anywhere
(`grep -rn "^from plugins\." plugins/ --include=*.py` = 0 hits) and `machines`
does not import `computers` at all, so finding 1's cycle does not exist. No
plugin deep-imports `shopdb.core` (0 hits), so finding 2 is refuted outright. No
migration in the tree spells a column in snake_case, so finding 3's examples are
refuted. `MACHINE_TYPES`, `Bldg 300`, Vuex, a ServiceNow plugin and an
unhandled `requests.post` do not exist. The page has been rewritten to carry a
verdict and the evidence per finding, keeping the four that survive in narrowed
form.
---
### 5. ADR Conformance - September 18, 2026
**File:** `ADR-Conformance-2026-09-18.md`
**Wiki path:** `ADRs/ADR-Conformance-2026-09-18`
**Summary:** ADR-by-ADR result of the 2026-09-18 review of gh-wj at 35d4fcd41.
Fourteen ADRs hold with cited evidence; eight items are recorded as violations
or gaps, each with a file and line.
**Reads first, before the table:** the ADR set does not exist on this branch.
`ls docs/adr` fails on gh-wj; ADR-001 through ADR-018 live only on the other
lineage. Nothing here states the rules and nothing here enforces them except
`scripts/check-naming-and-style.sh`, which names exactly two ADRs (ADR-017 at
line 343, ADR-015 at line 395). A handful of tests cover ADR ground without
saying so, and `tests/test_api_inventory_parity.py` skips itself entirely
because `docs/api-inventory.json` is not tracked here.
**Violations recorded:** ADR-007 (CHANGELOG untouched across 20 commits),
ADR-008 (endpoints migration 0023 does DDL on core `assets`), ADR-016
(misdescribed in four files, and the per-PC programme has no design record),
ADR-018 (accepted, unimplemented, no status note), ADR-001/002 (raw SQL over
core tables in the computers CLI), ADR-015 (the scan does not cover `agent/`),
ADR-002 (two stale manifest floors).
---
## Pages to Update
@@ -189,7 +256,7 @@ Returns settings routes contributed by all enabled plugins via `get_settings_rou
ShopDB deploys across three air-gapped environments:
```
Dev PC (gitea) <--USB--> Work PC (GitHub) <--USB--> Production VM
Dev PC (the internal git server) <--USB--> Work PC (GitHub) <--USB--> Production VM
```
### Bundle Creation
@@ -255,18 +322,28 @@ USE_GIT_LS_FILES=0 bash scripts/check-naming-and-style.sh
Gate tests validate generated documentation and API consistency:
- `tests/test_openapi_spec.py` - OpenAPI spec completeness
- `tests/test_public_read_surface.py` - Public endpoint security
- `tests/test_authz.py` - Authorization decorators
- `tests/test_plugin_migrations.py` - Migration chain validation (ADR-008)
- `tests/test_core/test_public_read_surface.py` - public endpoint security
- `tests/test_core/test_authz.py` - authorization decorators
- `tests/test_plugin_migrations.py` - migration chain validation (ADR-008)
- `tests/test_core/test_plugin_settings_hubs.py` - plugin settings hubs (ADR-019)
**Important:** When you add/modify API endpoints, regenerate docs:
(Paths corrected 2026-09-18: the first draft put three of these at the top of
`tests/` rather than in `tests/test_core/`.)
**Important:** When you add or modify API endpoints, regenerate docs:
```bash
# Update docs/openapi.json to reflect new endpoints
# (Manual until docs/api-inventory.json exists)
git add docs/openapi.json
```
**Caveat for the wj branch (2026-09-18):** the parity gate cannot run here.
`tests/test_api_inventory_parity.py` skips itself when `docs/api-inventory.json`
is absent, and that file is not tracked on gh-wj, so `scripts/gen_openapi.py`
has no input either. `docs/openapi.json` on this branch is 377 paths at
`info.version` 0.12.0 and documents none of the 24 routes added since
2ac2e6d33. Regenerating it needs the inventory file from the other lineage.
**Enforcement:** `scripts/check-gates-ran.sh` verifies gates actually ran (not skipped)
**CI failure:** "Gate tests FAILED" means docs are stale or missing
@@ -286,9 +363,9 @@ git add docs/openapi.json
```bash
# These MUST be in .gitignore
.claude/settings.json
.claude/settings.local.json
.claude/mcp-servers.json
the assistant settings file
the assistant local-settings file
the assistant tool-config file
```
**Why:** Personal config files may contain:
@@ -297,7 +374,7 @@ git add docs/openapi.json
- API keys and collector tokens
- Personal authentication tokens
**Incident:** 2026-09-11 - Passwords exposed in `.claude/mcp-servers.json` committed to GitHub
**Incident:** 2026-09-11 - Passwords exposed in the assistant tool-config file committed to GitHub
- Fixed: Removed from git tracking, added to .gitignore
- Impact: Dev environment credentials only (not production)
@@ -313,14 +390,56 @@ Pre-commit hook scans for these patterns:
**If blocked:** Replace hardcoded credentials with environment variables or config files
## ADR-016: Credential Delivery
## How fleet credentials actually work (corrected 2026-09-18)
Production credentials delivered via:
- Device tokens (for endpoints/agents)
- Personal Access Tokens (for users)
- Environment variables (for services)
The paragraph that stood here attributed a decision to ADR-016 that ADR-016
never took. It is replaced with what the code does. Do not reinstate it.
**Never:** Hardcoded in source, registry keys, git history
**What was wrong.** It said ADR-016 delivers credentials by "device tokens" and
that credentials must **never** live in registry keys. ADR-016 says neither.
Grepping every ADR for device tokens returns nothing. ADR-016's Decision is a
fleet credential store with Fernet at rest, a targeting and precedence resolver,
and `GET /api/collector/credentials` behind a dedicated `credentials.fetch`
scope - and its "Alternatives rejected" section calls registry-only provisioning
"a worthwhile FIRST STEP" and "Recommended as tier one regardless". The ban also
contradicted shipped code: `plugins/computers/client/Report-AssetToShopDB.ps1:367-374`
reads the collector key from `HKLM:\SOFTWARE\GE\ShopDB`, and
`plugins/endpoints/client/Set-ShopDBServerPins.ps1:4-20` writes the server trust
pins to the same key by design. Publishing the old text would have told the next
person to delete values the fleet depends on.
**What is actually built, as of gh-wj 35d4fcd41:**
- **Per-PC endpoint credentials.** An enrolled PC authenticates with
`X-Endpoint-Credential` (`plugins/endpoints/api/agentauth.py`). The server
stores only a sha256 hash plus a 12-character display prefix
(`plugins/endpoints/models/credential.py`); the secret exists once, in the
201 response to `/enroll`. On the endpoint it is a DPAPI LocalMachine blob
under `%ProgramData%\ShopDB\EndpointAgent\credential`, in a directory
ACL'd to SYSTEM and Administrators only.
- **Bootstrap tokens** for the one-line install: hashed at rest, expiry
mandatory and bounded to 1-168 hours, uses bounded to 1-500, revocable. The
package they serve never carries the collector key.
- **The fleet shared key** is still live authority for any PC that is not yet
enrolled, and turning it off is a fleet-wide switch, not a per-PC one. The
rollout sequence is `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`.
- **ADR-016's own mechanism is NOT built.** `credentials.fetch`,
`collector/credentials` and `Fernet` return zero hits across `shopdb/` and
`plugins/`, matching ADR-016's own "Implementation status, 2026-08-14".
**Two things that are genuinely never acceptable**, and this part of the
original text was right: a credential hardcoded in source, and a credential in
git history.
**Registry storage is not itself the problem; an unprotected key is.**
`install.ps1` writes the shared collector key to `HKLM\SOFTWARE\GE\ShopDB`
with no ACL (`plugins/endpoints/api/siteagent.py:328`, helper at `:394-411`), so
it inherits read for BUILTIN\Users and any logged-on user can read it. The repo
states that consequence itself at
`.github/GE-ENFORCE-COLLECTOR-FIX.html:377-380`. The rollout plans to clear that
value at cutover.
**See also:** ADR-Conformance-2026-09-18.md, violation 3.
```
---
@@ -332,7 +451,7 @@ Production credentials delivered via:
```markdown
# Automation Skills
Claude Code skills created Q3 2026 for common workflows. Save 147-205 hrs/year from 936 hrs baseline.
Assistant automation skills created Q3 2026 for common workflows. Save 147-205 hrs/year from 936 hrs baseline.
## Available Skills
@@ -414,11 +533,15 @@ Note: Realistic savings are 147-205 hrs/year due to task overlap.
## Summary
**4 new wiki pages drafted:**
1. ADR-019: Plugin Dynamic Settings Routes (architectural decision)
2. Security Audit 2026-09-11 (4 critical + 6 high findings)
3. Maintenance Footprint Baseline (936 hrs/year burden, automation opportunities)
4. Architectural Findings 2026-09-11 (22 findings from Gortex analysis)
**5 new wiki pages drafted:**
1. ADR-019: Plugin Dynamic Settings Routes (architectural decision) - corrected 2026-09-18
2. Security Audit 2026-09-11 (4 critical + 6 high findings) - refreshed 2026-09-18
against the same branch; every 2026-09-11 finding now carries a verdict with a
file and line, and findings 19 and up were added on 2026-09-18
3. Maintenance Footprint Baseline (936 hrs/year burden, automation opportunities) - annotated 2026-09-18
4. Architectural Findings 2026-09-11 - rewritten 2026-09-18 with a verdict per finding
5. ADR Conformance 2026-09-18 (new) - which ADRs hold, which are violated, and the fact
that the ADR set is not in this repository
**7 existing pages need updates:**
1. PLUGIN-DEVELOPMENT.md (add get_settings_routes() docs)
@@ -426,26 +549,44 @@ Note: Realistic savings are 147-205 hrs/year due to task overlap.
3. DEPLOYMENT.md (add air-gap workflow)
4. CONTRIBUTING.md (add optimized pre-commit checks)
5. TESTING.md (add gate tests explanation)
6. SECURITY.md (create new page for credential security)
6. SECURITY.md (create new page for credential security) - its ADR-016 section was
replaced on 2026-09-18; publish the replacement, not what was there
7. Automation-Skills.md (create new page for skill catalog)
**Total effort to publish:** ~2 hours (copy-paste to wiki, adjust formatting, add navigation links)
**Total effort to publish:** the "~2 hours, copy-paste" estimate no longer
holds. Three of the five pages needed correcting against the code before they
were fit to publish, and the Security Audit page has not been through that pass
yet.
---
## Next Steps
1. Review drafts for accuracy (have I missed anything?)
1. Read the 2026-09-18 notes inside each page before pasting anything. All five
pages in this folder have now been checked against gh-wj at 35d4fcd41.
2. Copy to GitHub wiki at https://github.com/ge-aero/shopdb-flask/wiki
3. Update wiki navigation/sidebar to include new pages
4. Link related pages (ADRs reference each other, operations pages link to security audit, etc.)
5. Notify team of new documentation
3. Update wiki navigation/sidebar to include the new pages
4. Link related pages. In particular, `Security-Audit-2026-09-11.md` finding 31
and `ADR-Conformance-2026-09-18.md` violation 3 are the same ADR-016
misattribution seen from two angles, and the SECURITY.md draft above is the
copy that was fixed.
5. Notify the team
**Three corrections that live outside this folder** and were therefore NOT made:
- `CLAUDE.md:3` and `:9` say PostgreSQL 15+; `docker-compose.yml:20` runs
`mysql:8.4` and `.env.example:29` is a `mysql+pymysql://` URL.
- `CLAUDE.md:31` says columns are snake_case; `CONTRIBUTING.md:17` says
lowercase concatenated with no underscores, and that is what
`scripts/check-naming-and-style.sh:236` enforces.
- `CLAUDE.md:119` and the assistant quick-reference file, lines 14 and 135 repeat the
ADR-016 device-token misattribution.
---
## Related Memory Files
These wiki pages are based on memory files in `.claude/memory/`:
These wiki pages are based on memory files in the assistant memory directory:
- `security_audit_2026_09_11.md`
- `maintenance_footprint_baseline.md`
- `architectural_analysis_gortex_2026_09_11.md`

605
docs/wiki-drafts/Security-Audit-2026-09-11.md
View File

@@ -1,393 +1,406 @@
# Security Audit - September 11, 2026
# Security Audit - September 11, 2026, refreshed September 18, 2026
**Scope:** Endpoints plugin, C# agent, system tray application
**Severity:** 4 critical, 6 high, 8 medium findings
**Status:** 3 critical fixed, 1 critical planned, 6 high in progress
**Scope:** Endpoints plugin, C# agent, system tray application, and (new on 2026-09-18) the USB and printed-parts kiosks
**Branch reviewed:** `gh-wj` at `35d4fcd41` (2026-09-16). Range re-read for this refresh: `2ac2e6d33..gh-wj`, 38 commits, 2026-09-11 to 2026-09-16
**Severity, 2026-09-11 set:** 4 critical, 6 high, 8 medium
**Severity, added 2026-09-18:** 1 high, 4 medium, 9 low
**Status:** see the remediation summary. Every 2026-09-11 finding below now carries a verdict read off the code on this branch, with a file and line.
This page covers two dates. The findings numbered 1 to 18 were raised on 2026-09-11 and their status lines were rewritten on 2026-09-18. The findings numbered 19 and up were raised on 2026-09-18 and did not exist on the earlier page.
## What changed since 2026-09-11
Two programmes landed in this range and both move the security boundary.
**Per-PC agent credentials.** The agent no longer has to authenticate with one fleet-wide shared key. Each PC can hold its own secret: `plugins/endpoints/models/credential.py:36` stores only a sha256 hash and a 12-character display prefix, `plugins/endpoints/api/agentauth.py` makes `X-Endpoint-Credential` the agent auth header, and `plugins/endpoints/api/enrollment.py:289` adds an unauthenticated `/enroll` that a PC uses to ask for one. On the endpoint the secret is a DPAPI LocalMachine blob written through a SYSTEM and Administrators only DACL (`agent/ShopDbEndpointAgent.Core/AgentCredentials.cs:203`, `agent/ShopDbEndpointAgent.Core/ProtectedFileSystem.cs:219`). Once a PC holds an active credential the server refuses the shared key for it (`plugins/endpoints/api/agentauth.py:129`). The operator sequence for the cutover is `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`.
**Kiosk network admission.** Commit `da91a2915` added a second way for an unauthenticated kiosk caller to be admitted: a source address inside a configured CIDR list, alongside the existing per-device token. Both the USB kiosk (`plugins/usb/api/routes.py:798`) and the printed-parts kiosk (`plugins/printedparts/api/routes.py:551`) call `request_ip_allowlisted`, backed by the new `shopdb/utils/ipallowlist.py`. Both lists default to empty, which matches nothing (`shopdb/utils/ipallowlist.py:44`), and neither can override the kiosk's own enabled flag. This widens an unauthenticated write surface from a handful of provisioned screens to every PC on the named networks, so the correctness of the client IP the server sees now matters more than it did.
Also in range: the agent's server certificate pinning was completed, the tray stopped holding secrets, a one-line bootstrap install was added, and the 2026-09-12 audit reports were deliberately removed from the product tree (`8eee6f19d`) because they described weaknesses in detail on the branch that deploys to production. This page keeps that spirit: findings are located precisely enough to fix, and no exploit recipe is written out.
## Executive Summary
Comprehensive security review of the endpoint management subsystem (plugin + C# agent + tray app) identified critical authentication gaps, credential exposure risks, and performance bottlenecks that expose the system to unauthorized access and operational failure.
The headline of the 2026-09-11 page, unauthenticated inventory writes, is closed on the ingest routes that actually exist on this branch: `POST /api/collector/<pluginname>` requires either a per-PC credential or a collector key, and neither branch of `shopdb/core/api/collector.py:251` reaches the plugin without one.
**Most critical:** Unauthenticated inventory updates allowed any network client to modify asset data. **Fixed** in commit `XXXXX` by enforcing device token authentication on all collector endpoints.
The 2026-09-11 page's second headline, the collector key sitting unprotected in the registry, was still open when this review ran and was **fixed the same day**: the generated installer now restricts `HKLM\SOFTWARE\GE\ShopDB` to SYSTEM and Administrators in both views (finding 19). The key itself stays on purpose until every consumer is on a per-PC credential, and bays installed before this change keep the old DACL until they are reinstalled or upgraded. The per-PC credential work in this range is the exit from the shared key entirely, but only for PCs that have enrolled.
**Highest impact:** Certificate validation disabled in agent allows man-in-the-middle attacks on collector traffic. **Planned** for Q4 2026 with per-endpoint certificate authentication.
The third headline, no certificate validation in the agent, is **fixed** and was fixed properly, not worked around.
## Critical Findings (P0)
### 1. Unauthenticated Inventory Updates [FIXED]
### 1. Unauthenticated Inventory Updates [CLOSED, ON THE ROUTES THAT EXIST]
**Risk:** Any client on the network could POST to `/api/endpoints/inventory` and modify asset records without authentication.
**2026-09-18 verdict:** closed, but not by the mechanism the 2026-09-11 page described, and the evidence it published cannot be reproduced.
**Impact:**
- Attacker could inject false inventory data
- Legitimate collector reports could be overwritten
- Audit trail shows "unknown" as modifier
- Data integrity compromised
None of the four paths named in the original fix (`/api/endpoints/inventory`, `/api/endpoints/heartbeat`, `/api/endpoints/{id}/apps`, `/api/endpoints/{id}/snapshots`) exists on this branch. Endpoint ingest is the ADR-006 generic collector: `@collector_bp.route('/<pluginname>', methods=['POST'])` at `shopdb/core/api/collector.py:229`. That route admits a caller in exactly two ways and neither is anonymous:
**Root cause:** Missing `@requires_device_token` decorator on inventory endpoints.
- a per-PC secret in `X-Endpoint-Credential`, resolved to an asset id or rejected 401 `CREDENTIAL_INVALID` (`shopdb/core/api/collector.py:255-264`)
- otherwise `_check_collector_auth`, which takes a per-plugin or shared collector key, or a `collector.ingest` scoped managed token, from the header only (`shopdb/core/api/collector.py:154`)
**Fix:** Commit `XXXXX` added device token authentication to:
- `/api/endpoints/inventory` (POST)
- `/api/endpoints/heartbeat` (POST)
- `/api/endpoints/{id}/apps` (POST)
- `/api/endpoints/{id}/snapshots` (POST)
A credential caller also cannot write for another PC: the three schema-declaring plugins raise `PermissionError` on an asset mismatch and the dispatcher maps it to 403 `ASSET_MISMATCH` (`shopdb/core/api/collector.py:294`, `plugins/computers/plugin.py:729`, `plugins/printers/plugin.py:556`, `plugins/endpoints/plugin.py:1422`).
**Verification:**
```bash
# Before: Succeeds without auth
curl -X POST http://localhost:5001/api/endpoints/inventory -d '{...}'
**What is still open:** four legacy collector routes take the shared key alone and are not covered by the new refusal. See finding 20.
# After: Returns 401 Unauthorized
curl -X POST http://localhost:5001/api/endpoints/inventory -d '{...}'
# {"status":"error","data":{"error":{"code":"UNAUTHORIZED","message":"Device token required"}}}
**Residual risk, restated:** the original text said device tokens sit in agent config files protected by Windows ACLs. On this branch the per-PC credential is not in a config file at all; it is a DPAPI LocalMachine blob in `%ProgramData%\ShopDB\EndpointAgent\credential` written through a protected DACL (`agent/ShopDbEndpointAgent.Core/AgentCredentials.cs:207-215`). The fleet shared key is the one that is still readable, and that is finding 19.
# With token: Succeeds
curl -X POST http://localhost:5001/api/endpoints/inventory \
-H "X-API-Key: shopdb_device_abc123..." -d '{...}'
```
### 2. Deployment Credential Exposure in Registry [STILL OPEN]
**Residual risk:** Device tokens stored in agent config files on endpoints. An attacker with file access can read and reuse the token. Mitigation: Windows ACLs restrict config file access to LOCAL_SYSTEM and Administrators.
**2026-09-18 verdict:** still open, and the described fix did not ship. Promoted to the highest open severity and restated in full as finding 19.
### 2. Deployment Credential Exposure in Registry [MITIGATED]
The 2026-09-11 page said the deployment script would store the token in an ACL'd config file, remove the registry key, and have the agent read the file rather than the registry. On this branch:
**Risk:** Deployment scripts wrote collector API key to HKLM registry in plaintext, readable by any user via `reg query`.
- the generated installer still writes the key to the registry with no ACL: `plugins/endpoints/api/siteagent.py:328`, through a helper that does only `CreateSubKey` plus `SetValue` in both views (`plugins/endpoints/api/siteagent.py:394-411`). There is no `SetAccessControl` or `RegistrySecurity` anywhere in that file.
- the collection script still reads the key back out of the registry: `plugins/computers/client/Report-AssetToShopDB.ps1:367-375`, both the native and the WOW6432Node view.
- the registry path in the original write-up (`HKLM\SOFTWARE\ShopDB\Agent`, value `CollectorApiKey`) never matched this product. The real location is `HKLM\SOFTWARE\GE\ShopDB`, value `CollectorKey`.
**Impact:**
- Any logged-in user (including non-admin) could read the API key
- Key valid for all collector operations (inventory, heartbeat, app updates)
- No expiration or rotation mechanism
**What did change, and it matters:** there is now a keyless build. When the package is built without the shared key the installer emits no `-CollectorKey` parameter and no key write at all (`plugins/endpoints/api/siteagent.py:331-334`), and the PC enrolls for its own credential instead. The rollout runbook plans to clear `CollectorKey` from both registry views at cutover (`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:351-361`). So the exposure ends when the fleet finishes enrolling, not before.
**Evidence:**
```powershell
PS> reg query "HKLM\SOFTWARE\ShopDB\Agent" /v CollectorApiKey
CollectorApiKey REG_SZ shopdb_pat_b4a3fe2a01abc6bc1cb685280e65a8c35d52ec43
```
**Note on the registry generally:** the same key is also where the agent's server trust pins live, by design (`plugins/endpoints/client/Set-ShopDBServerPins.ps1:4-20`, precedence stated at `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:106-111`). Pins are trust anchors, not secrets. Do not read "get the key out of the registry" as "the registry must hold nothing".
**Fix:** Deployment script updated to:
1. Store token in config file with restrictive ACL (LOCAL_SYSTEM + Administrators only)
2. Remove registry key after migration
3. Agent reads from config file, not registry
### 3. No Certificate Validation in Agent [FIXED]
**Status:** Mitigated on new deployments. Legacy endpoints still have registry key. Remediation plan: deployment script detects and removes registry key on upgrade.
**2026-09-18 verdict:** fixed, and fixed by pinning rather than by re-enabling default validation and hoping.
**Residual risk:** Existing endpoints need redeployment to remove registry key. Scheduled for Q4 2026 mass agent update.
`agent/ShopDbEndpointAgent.Core/AgentHttpFactory.cs` is the only place the agent constructs an HTTP client, and every channel goes through `pins.Validate(cert, errors, channel)` (`AgentHttpFactory.cs:52`). `Validate` (`AgentHttpFactory.cs:291`) has no path that returns true for an unmatched pin: unpinned falls back to `SslPolicyErrors.None` (`AgentHttpFactory.cs:302`), pinned requires an SPKI sha256 or legacy thumbprint match (`AgentHttpFactory.cs:314`). A failed registry re-read returns null and keeps the existing pins rather than silently unpinning.
### 3. No Certificate Validation in Agent [PLANNED]
The three SYSTEM PowerShell reporters refuse to send at all when pins are configured but the check cannot be enabled, including under pwsh (`plugins/computers/client/Report-AssetToShopDB.ps1:228-247`, and the same block in `plugins/printers/client/Report-PrintersToShopDB.ps1:125` and `plugins/endpoints/api/templates/upgrade-shopdbagent.ps1:110`).
**Risk:** Agent's HTTP client disables SSL certificate validation:
The blanket accept-any-certificate callback the original finding quoted survives in exactly one place, a developer connectivity probe that is not shipped to endpoints: `test-vm-connectivity.ps1:44`.
```csharp
ServicePointManager.ServerCertificateValidationCallback =
(sender, cert, chain, sslPolicyErrors) => true; // ACCEPTS ANY CERT
```
### 4. Hardcoded Admin Credentials [PARTLY FIXED]
**Impact:**
- Man-in-the-middle attack possible on collector traffic
- Attacker can intercept inventory data (hostnames, IPs, installed apps, usernames)
- Attacker can inject false commands or updates
**2026-09-18 verdict:** partly fixed. The production path is blocked; the credential is still in the source tree.
**Why it exists:** Corporate proxy with self-signed certificate breaks standard validation. Quick fix was to disable validation entirely.
`flask seed test-user` still creates `admin` / `admin123` (`shopdb/cli/__init__.py:636`), and the string appears four times in that file. What changed is that the command now refuses to run outside DEBUG or TESTING and says why (`shopdb/cli/__init__.py:619-623`), pointing the operator at `flask seed admin`, which generates a password.
**Proper fix (PLANNED):** Per-endpoint certificate pinning
1. Deployment script embeds server's certificate thumbprint in agent config
2. Agent validates that presented cert matches pinned thumbprint
3. Pinned cert stored in protected config file (LOCAL_SYSTEM ACL)
4. Rotation: new deployment updates pinned thumbprint
The file the original finding quoted, `tests/fixtures/users.py`, does not exist on this branch, so the described fix to it cannot be confirmed.
**Implementation plan:** ADR-XXX Per-Endpoint Certificate Authentication (see Security-Fix-Implementation.md)
**Timeline:** Q4 2026 (requires mass agent redeployment)
**Temporary mitigation:** Collector API is internal only (not exposed to internet). Attacker must be on corporate network with proxy access. Still unacceptable for long-term.
### 4. Hardcoded Admin Credentials [REMOVED]
**Risk:** Test fixtures and seed scripts contained hardcoded passwords:
```python
# tests/fixtures/users.py
admin_user = User(
username='admin',
password='admin123', # HARDCODED
role='admin'
)
```
**Impact:**
- Default credentials usable if seed script runs in production
- Password visible in source code and git history
- No password complexity or rotation
**Fix:** Commit `XXXXX` removed hardcoded passwords:
1. Test fixtures use `os.environ.get('TEST_PASSWORD', generate_random())`
2. Seed scripts prompt for admin password on first run
3. Password stored hashed (bcrypt) in database, not in code
4. Deployment docs updated to mandate password change on first login
**Verification:** Grep for hardcoded passwords returns zero hits:
```bash
grep -r "password.*=.*['\"][^'\"]\{8,\}" --include="*.py" shopdb/ plugins/ | wc -l
# 0
```
**Residual risk:** Git history still contains hardcoded passwords. Requires git-filter-repo to fully purge. Scheduled for next major release (v1.0).
**Residual risk:** unchanged. A dev-only well-known credential still ships in the tree, and git history still holds the older forms.
## High Priority Findings (P1)
### 5. Collector Key Exposure in Committed Files
### 5. Collector Key Exposure in Committed Files [FIXED, ONE CLAIM CORRECTED]
**Risk:** Collector API key (`shopdb_pat_...`) committed to `.claude/skills/*.md` files and pushed to GitHub.
**2026-09-18 verdict:** the redaction holds. `git ls-files | xargs grep -l "shopdb_pat_"` returns nothing on this branch, and the assistant settings file, the assistant local-settings file and the assistant tool-config file are ignored (`.gitignore:174-176`).
**Impact:**
- Key visible in public/private GitHub repository
- Valid until manually rotated
- Grants full collector access (inventory, heartbeat, app updates)
**Correction:** the original page's prevention line is not true of this branch. The committed pre-commit hook runs the naming and style gate only (`.githooks/pre-commit:5`), and `scripts/check-naming-and-style.sh` has no credential pattern check at all. Its eleven checks are non-ASCII, banned shorthand, three snake_case rules, frontend API params, plugin frontend imports, the shared code-library module, ADR-017 map levels and ADR-015 site literals (`scripts/check-naming-and-style.sh:151-395`). Nothing blocks a commit that reintroduces a key.
**Evidence:**
```bash
git log -S "shopdb_pat_b4a3fe2a" --oneline
ce27cb5eef Add 9 comprehensive automation skills
```
### 6. Password Exposure in MCP Config [FIXED AS FAR AS THIS BRANCH GOES]
**Fix:** Commit `1e8c6a513a` redacted key and added to .gitignore:
1. Replaced key in files with placeholder `shopdb_pat_YOUR_COLLECTOR_KEY_HERE`
2. Added `.claude/settings*.json` and `.claude/mcp-servers.json` to .gitignore
3. Removed files from git tracking (`git rm --cached`)
**2026-09-18 verdict:** the assistant tool-config file is ignored (`.gitignore:176`) and is not in `git ls-files`. The accepted risk recorded in 2026-09-11, that the exposed dev passwords were not rotated, is unchanged and still accepted.
**Key rotation:** User indicated passwords/keys do NOT need rotation despite exposure ("won't need to"). Accepted risk: exposed key remains valid.
### 7. Missing Rate Limiting on Login [FIXED]
**Prevention:** Pre-commit hook scans for `shopdb_pat_` pattern and blocks commit if found.
**2026-09-18 verdict:** fixed, in two layers.
### 6. Password Exposure in MCP Config
- Per-IP fixed-window limiter: `_login_ratelimited` at `shopdb/core/api/auth.py:35`, applied at `shopdb/core/api/auth.py:80`, defaults 30 hits per 300 seconds.
- Per-account lockout: after `MAX_FAILED_LOGINS` consecutive bad passwords the account is locked (`shopdb/core/api/auth.py:113-118`), and a locked account is rejected before the password is checked so a valid password cannot reset the clock mid-lockout (`shopdb/core/api/auth.py:100`).
**Risk:** `.claude/mcp-servers.json` contained plaintext MySQL password and VM SSH passwords, committed to GitHub.
**Known limit, stated in the code:** under the default SimpleCache the limiter's counter is per-process, so with N gunicorn workers the effective budget is N times the configured maximum (`shopdb/core/api/auth.py:39-42`). The per-account lockout is the control that does not have that weakness. Both are keyed on the client address, which is why the client-IP plumbing in finding 23 matters.
**Impact:**
- MySQL dev database password: `shopdb-dev-password`
- VM SSH password for SupportUser: `vos313`
- VM SSH password for shopfloor account: `vos313`
- Passwords visible in git history
### 8. SQL Injection in Asset Search [STILL A FALSE POSITIVE]
**Fix:** Commit `1e8c6a513a` removed from git tracking and added to .gitignore.
**2026-09-18 verdict:** re-checked, still no injection.
**Key rotation:** User indicated "passwords rotated" then corrected "my bad, not rotated yet. but won't need to". Accepted risk: exposed passwords remain valid.
The raw SQL that exists is parameterised. The USB external-database path builds a WHERE clause out of fixed fragments and passes every value as a parameter (`plugins/usb/api/routes.py:277-299`); the only interpolated text is the fragment list itself. `plugins/employees/api/routes.py:113` binds through `%s`.
**Scope:** Dev environment only. Production uses different credentials not stored in repo.
**One related note that is not injection:** `plugins/computers/plugin.py:2080` runs a raw `db.text` query joining the core `assets`, `assetrelationships` and `assettypes` tables inside the `flask computers retype` CLI. No user input reaches it, so it is not a security finding. It is hidden coupling to the core schema, which is an architecture item, not one for this page.
### 7. Missing Rate Limiting on Login
### 9. Session Token Storage in LocalStorage [UNCHANGED]
**Risk:** No rate limiting or account lockout on failed login attempts.
**2026-09-18 verdict:** unchanged. The JWT and the refresh token are still written to and read from localStorage (`frontend/src/stores/auth.js:6-7`, `frontend/src/stores/auth.js:43-44`). No HttpOnly cookie path was added in this range.
**Impact:**
- Brute force attack on login endpoint
- No delay or throttling after failed attempts
- Audit log fills with failed attempts but no automated response
Partial compensating control added since: the app now sets `X-Content-Type-Options`, `X-Frame-Options`, `Referrer-Policy` and a `frame-ancestors 'self'` CSP on every response (`shopdb/__init__.py:334-341`). That is not a script-src policy and the code says so deliberately.
**Evidence:**
```bash
# Can attempt 1000 logins/second with no throttling
for i in {1..1000}; do
curl -X POST http://localhost:5001/api/auth/login \
-d '{"username":"admin","password":"wrong"}' &
done
```
### 10. Audit Log Unbounded Growth [MECHANISM BUILT, OFF BY DEFAULT]
**Mitigation options:**
1. Flask-Limiter: `@limiter.limit("5 per minute")` on login endpoint
2. Account lockout: Lock account after 5 failed attempts for 15 minutes
3. CAPTCHA after 3 failed attempts
**2026-09-18 verdict:** the tooling now exists; it does nothing unless an operator turns it on and runs it.
**Status:** Planned for Q1 2027. Not blocking for air-gapped production (internal network only).
- Service: `shopdb/core/services/auditretention.py`
- Admin API: `GET /api/auditlogs/retention` (`shopdb/core/api/auditlogs.py:195`) and the purge route at `shopdb/core/api/auditlogs.py:211`
- CLI: `shopdb/cli/__init__.py:1894` and `shopdb/cli/__init__.py:1918`
### 8. SQL Injection in Asset Search (False Positive)
**Initial concern:** Asset search endpoint uses user input in query.
**Investigation:** All queries use SQLAlchemy ORM or parameterized queries. String concatenation not found.
**Verification:**
```bash
grep -rn "execute.*+\|execute.*format\|execute.*%" \
--include="*.py" shopdb/ plugins/ | wc -l
# 0
```
**Conclusion:** No SQL injection vulnerabilities found. Marked false positive.
### 9. Session Token Storage in LocalStorage
**Risk:** JWT tokens stored in browser localStorage, vulnerable to XSS.
**Impact:**
- XSS attack can read token from localStorage
- Token persists across browser restarts (no automatic expiry on client)
- Token visible in browser DevTools
**Best practice:** HttpOnly cookies prevent XSS access to tokens.
**Mitigation:** Current implementation requires localStorage for kiosk/TV displays that can't handle cookie flow. Acceptable for internal network deployment.
**Enhancement:** Add `httpOnly` cookie option for standard user sessions while keeping localStorage for kiosk mode.
**Status:** Planned for v1.0 (requires auth refactor).
### 10. Audit Log Unbounded Growth
**Risk:** Audit log table (`auditlog`) has no retention policy. Grows unbounded.
**Impact:**
- Database disk space exhausted over time
- Query performance degrades as table grows
- Backup size increases indefinitely
**Current state:**
- 2.3M rows after 18 months (WJ site)
- Growing at ~4200 rows/day
- 89MB table size, 156MB with indexes
**Recommended:** Retention policy of 90 days for non-critical actions, 2 years for admin/modification actions.
**Status:** Planned for Q4 2026. See MAINTENANCE-DEPLOYMENT-CHECKLIST.md for implementation.
`configureddays()` returns 0 when `audit_retentiondays` is unset, and 0 means keep forever (`shopdb/core/services/auditretention.py:123-133`). There is no scheduler in that module, so nothing purges on its own. The row and size figures quoted in 2026-09-11 come from the reference site database and cannot be confirmed from the repository.
## Medium Priority Findings (P2)
### 11. Endpoint Event Accumulation
### 11. Endpoint Event Accumulation [STILL OPEN]
**Risk:** `endpointevents` table accumulates every app launch, process change, USB connect. No retention.
**2026-09-18 verdict:** still open. `EndpointEvent` is `endpointevents` (`plugins/endpoints/models/endpoint.py:333`) and there is no retention, purge or prune code anywhere under `plugins/endpoints/`. The audit-log retention service added in this range covers `auditlog` only. The row and size figures quoted in 2026-09-11 are site measurements and are not verifiable here.
**Impact:**
- 450K rows after 18 months
- 127MB table size
- High-churn data (process events every 30 seconds)
### 12. Missing HTTPS Enforcement in Dev [CHANGED; THE STATED MITIGATION IS GONE]
**Recommended:** Retention of 30 days for events, 90 days for snapshots.
**2026-09-18 verdict:** the specific mitigation the page described no longer exists, and that is an improvement. `NODE_TLS_REJECT_UNAUTHORIZED` appears nowhere in the tree. The Vite dev proxy target is now an environment variable defaulting to `http://localhost:5001`, and TLS verification for an https target is off unless `VITE_DEV_SECURE=true` (`frontend/vite.config.js:5-18`). Dev only; production still terminates TLS in front of the app.
### 12. Missing HTTPS Enforcement in Dev
### 13. Docker Log Rotation Not Configured [STILL OPEN]
**Risk:** Vite dev server runs HTTP, not HTTPS. Cookies sent without Secure flag.
**2026-09-18 verdict:** still open. There is no `logging:` block anywhere in `docker-compose.yml`. Both hops log to stdout (`nginx.conf:36`, `docker-compose.yml:213-214`), so the host's Docker log driver defaults apply unchanged. This now has a second consequence: see finding 25.
**Impact:** Dev environment only. Production uses HTTPS via gunicorn + nginx.
### 14-18. Additional Medium Findings [PARTLY ADDRESSED, SOURCE NOW MISSING]
**Mitigation:** `start-dev.cmd` sets `NODE_TLS_REJECT_UNAUTHORIZED=0` for Vite proxy.
**2026-09-18 verdict:** the file this section pointed at is gone. the assistant memory directory does not exist on this branch; commit `8eee6f19d` moved the 2026-09-12 audit reports and scan output out of the product tree on purpose. The detail behind findings 14 to 18 is therefore not readable from here.
### 13. Docker Log Rotation Not Configured
Of the five items summarised in the original bullet list, two can be answered from the code:
**Risk:** Docker container logs grow unbounded. Can fill host disk.
- **No content security policy headers:** partly addressed. `shopdb/__init__.py:334-341` sets a security header block including `Content-Security-Policy: frame-ancestors 'self'`, and uploads carry a much stricter `default-src 'none'; sandbox` (`shopdb/utils/uploads.py:13`, applied at `shopdb/utils/uploads.py:35`). A real script-src policy is still not set, deliberately.
- **Directory listing enabled in nginx:** no `autoindex` directive appears in `nginx.conf`, and nginx defaults to off.
**Mitigation:** Configure log rotation in docker-compose.yml:
```yaml
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
```
The remaining three, input validation gaps, world-readable backup files and old TLS versions, cannot be confirmed or refuted without the removed source document. Treat them as unverified rather than as fixed.
**Status:** Planned for Q4 2026.
## New findings, 2026-09-18
### 14-18. Additional Medium Findings
### High
See full audit report in `.claude/memory/security_audit_2026_09_11.md` for:
- Missing input validation on some endpoints
- No content security policy headers
- Directory listing enabled in nginx (dev only)
- Backup files world-readable (dev only)
- Old TLS versions enabled (corporate proxy requirement)
### 19. The fleet shared key is written to the registry with no ACL (FIXED 2026-09-18; the key itself is transitional by design)
**Risk:** for a shared-key build, the generated `install.ps1` writes the collector key to `HKLM\SOFTWARE\GE\ShopDB\CollectorKey` (`plugins/endpoints/api/siteagent.py:328`) through a helper that does only `CreateSubKey` plus `SetValue` in both registry views (`plugins/endpoints/api/siteagent.py:394-411`). No `RegistrySecurity`, no `SetAccessControl`. The key inherits read for BUILTIN\Users. The repository states this outcome itself, in a file added in this same range: `.github/GE-ENFORCE-COLLECTOR-FIX.html:377-380`.
**Why it is still authority:** `authenticate_agent` accepts `X-API-Key` whenever `shared_key_allowed()` (`plugins/endpoints/api/agentauth.py:184-192`, default true at `plugins/endpoints/api/agentauth.py:67`), and `shared_key_refused_for` refuses it only for a hostname whose asset holds an **active** credential (`plugins/endpoints/api/agentauth.py:129-134`). So for any not-yet-enrolled, revoked or re-enroll-blocked PC the shared key still claims queued commands (`plugins/endpoints/api/routes.py:3352`) and downloads script bodies and installer packages (`plugins/endpoints/api/routes.py:3511`, `:3550`, `:3589`).
**Impact:** any operator logged on to a bay can read the fleet key and act as any un-enrolled endpoint: claim and answer its queued commands, write false compliance records, and fetch hosted installers and script bodies. It also means revoke plus re-enroll-block leaves a compromised PC holding a working credential of the older kind.
**Contrast:** the per-PC credential got the opposite treatment in the same range. DPAPI LocalMachine inside a SYSTEM and Administrators only DACL (`agent/ShopDbEndpointAgent.Core/AgentCredentials.cs:203-217`, `agent/ShopDbEndpointAgent.Core/ProtectedFileSystem.cs:219-232`), and `appsettings.json` is re-ACLed at runtime (`agent/ShopDbEndpointAgent/AgentPipeServer.cs:494-522`). The registry key alone was left open.
**Status, 2026-09-18: fixed.** The generated installer now calls `Protect-HklmShopdbKey` after provisioning, which breaks inheritance, drops the inherited entries rather than copying them, and states SYSTEM (S-1-5-18) and Administrators (S-1-5-32-544) by SID in both registry views (`plugins/endpoints/api/siteagent.py`). It runs for the key-less package too, because `ServerPins` live in the same key and the registry is authoritative for pinning. A bay that cannot take the ACL still installs, and says plainly that the key stays readable there. Four tests cover it in `tests/test_plugins/test_endpoints_siteagent_build.py`, including the write-then-lock ordering.
**The key's continued existence is a decision, not an oversight.** The shared key stays until `upgrade-shopdbagent.ps1`, `Report-AssetToShopDB.ps1` and `Report-PrintersToShopDB.ps1` are off it, because rollback dies without it; the site is mid-migration to per-bay credentials and the key-less package (`includesharedkey=False`) is already the path for new installs. What this finding was about was the DACL, and that is now closed. The remaining work is the migration itself, and finally clearing `CollectorKey` from both views as the runbook plans (`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:351-361`).
**Not closed by this:** bays that already carry the key keep their inherited DACL until they are reinstalled or upgraded through a package built after this change.
### Medium
### 20. Turning the shared key off does not close the four legacy collector routes
`generic_collect` enforces the new rule: 401 `SHARED_KEY_REFUSED` when `agent_shared_key_refused(identityvalue)` is true (`shopdb/core/api/collector.py:279-284`), which `plugins/endpoints/api/agentauth.py:129-134` makes true fleet-wide once `plugin.endpoints.sharedkeyallowed` is off.
Four legacy routes never consult it. `/pc` (`shopdb/core/api/collector.py:349`), `/apps` (`:387`), `/heartbeat` (`:459`) and `/bulk` (`:494`) are decorated `@require_api_key`, which checks only `COLLECTOR_API_KEY` or a `collector.ingest` scoped token (`shopdb/core/api/collector.py:154-164`). `/pc` then writes `lastreporteddate`, `loggedinuser`, `lastboottime` and the asset serial number for any hostname it can resolve (`shopdb/core/api/collector.py:367-376`); `/apps` and `/bulk` rewrite installed-app inventory.
Two details that make this worse than the runbook implies. `_env_fallback_disabled(pluginname)` returns False when `pluginname` is None (`shopdb/core/api/collector.py:60-74`), so these routes keep honouring the `.env` `COLLECTOR_API_KEY` even after `plugin.endpoints.envkeyfallback` is switched off. And "Regenerate key" mints a new `collector.ingest.endpoints` token and clears the env fallback but never changes `COLLECTOR_API_KEY` (`plugins/endpoints/api/routes.py:1864-1884`), so runbook Phase 6 step 4 does not close this either.
**Bounded by:** the UI-managed agent key is scoped `collector.ingest.endpoints` (`plugins/endpoints/api/routes.py:1834`) and `_collector_scopes_for(None)` allows only the generic `collector.ingest` (`shopdb/core/api/collector.py:46-57`), so a PC holding only that token is already refused on all four routes. The exposure needs a copy of the env shared key, and the reachable damage is inventory-field integrity. No command issuance and no credential minting.
ADR-006 says the legacy `/pc` endpoint is removed before v1.0. At `__version__` 0.12.0 keeping it is not yet a violation, but it is a hole in a boundary this branch advertises.
### 21. Rejecting an enrollment request is not durable
`enroll_endpoint` is unauthenticated by design. When a row for (hostname, serialnumber) already exists, it returns `rejected` only when the presented nonce matches the stored one (`plugins/endpoints/api/enrollment.py:383`). A REJECTED row presented with a **different** nonce falls into the non-pending branch at `plugins/endpoints/api/enrollment.py:390` and is rewritten into a fresh cycle: new nonce, new `requestedat`, `decidedat` and `decidedby` cleared (`:393-396`). Execution then falls through to the match block, and with the fleet shared key present it is auto-issued a credential outright (`plugins/endpoints/api/enrollment.py:426-427`). Both gates default on (`plugins/endpoints/api/agentauth.py:67`, `:71`). Without the key the request simply lands back in the admin approval queue, indistinguishable from a first-time request.
There is no second lever: `reject_enroll_request` sets no durable flag (`plugins/endpoints/api/enrollment.py:753-770`), and `reenrollblocked` lives on `EndpointCredential` (`plugins/endpoints/models/credential.py:56`), which `revoke_endpoint_credential` 404s on when no credential row exists (`plugins/endpoints/api/enrollment.py:801-805`). So a PC that never enrolled cannot be blocked.
**Bounded by:** auto-issue additionally requires a Computer record whose hostname matches and whose Asset serial matches case-insensitively (`plugins/endpoints/api/enrollment.py:405-421`), so this does not grant a credential for an arbitrary asset. The existing test covers only the same-nonce case (`tests/test_plugins/test_endpoints_credentials.py:250`).
### 22. The one-line bootstrap stages its script in a world-writable path and runs it elevated
`COMMAND_SCRIPT_PATH` is a literal `C:\Windows\Temp\shopdb-bootstrap.ps1` (`plugins/endpoints/api/bootstrap.py:68`), and the emitted command downloads to that path and then executes it, with a `Remove-Item -Force` against the same path in the finally block (`plugins/endpoints/api/bootstrap.py:164-169`). The module docstring and the script header both say the line is pasted into an elevated PowerShell (`plugins/endpoints/api/bootstrap.py:3-5`, `:209-213`). The same literal is published in `plugins/endpoints/AGENT-CONTRACT.md:794` and `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:296`.
`C:\Windows\Temp` is not admin-only: its default DACL grants BUILTIN\Users create-file plus CREATOR OWNER full control on what they create, and overwriting a file's contents does not reset its owner or DACL. Nothing verifies the script after download; the sha256 checks in the script body cover the downloaded ZIP's binaries, not the script itself.
**Bounded by:** this needs an attacker who can already log on interactively to that specific bay and a tech who later pastes the one-liner on that same PC. The bootstrap path is documented as the one-off route (`plugins/endpoints/api/bootstrap.py:7-11`), not how managed bays get the agent.
**Fix direction:** stage under a path only Administrators and SYSTEM can write, or verify the script against a server-stamped hash before executing it.
### 23. Kiosk network admission rests on a header the client can send, and proxy mode verifies nothing
`request_ip_allowlisted` reads `client_ip()`, which is `strip_port(request.remote_addr)` (`shopdb/utils/clientip.py:53`). On the Windows deployment `remote_addr` is not the TCP peer: `deploy/windows/web.config:31` launches waitress with `--trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for --trusted-proxy-count=1`, and HttpPlatformHandler always connects from loopback, so waitress always parses `X-Forwarded-For` and promotes the selected hop into `REMOTE_ADDR`.
The control that makes that hop honest is the IIS rewrite rule that overwrites the header with `{REMOTE_ADDR}`, and it ships commented out (`deploy/windows/web.config:90-102`). The installer enables and then verifies it only under `-ClientIpSource direct`: the whole "CLIENT-IP RULE IS NOT IN EFFECT" check sits inside `if ($ClientIpSource -eq 'direct')` at `deploy/windows/installer/shopdb-install.ps1:2509-2523`. In proxy mode the rule is deliberately left off (`:2041-2046`) and **nothing checks that a proxy is actually in front**. The module docstring says the same thing about the failure mode: it does not fail closed (`shopdb/utils/ipallowlist.py:17-26`).
**Bounded by:** `-ClientIpSource` defaults to `direct` (`deploy/windows/installer/shopdb-install.ps1:129`), which installs URL Rewrite, uncomments the block and verifies it live, so a default install is covered. With a real proxy in proxy mode the attack also fails, because waitress takes the last hop and ARR appends the true peer after any client-supplied value. Both kiosk allowlists default to empty and both kiosks are additionally gated on their enabled flag (`plugins/usb/api/routes.py:798-801`, `plugins/printedparts/api/routes.py:551-557`).
**The real gap:** proxy mode performs no verification at all. A server set to proxy mode with no proxy really there, or a hand-maintained `web.config` where the rewrite block was never uncommented, admits an attacker-chosen address. The same header also satisfies the GE-Enforce IP path (`plugins/geenforce/api/routes.py:82-108`), which is pre-existing and not introduced by `da91a2915`, and it re-keys the login rate limit and account lockout on a value the caller chose.
### Low
### 24. Revoke does not stop a live agent; it falls back to the shared key
`plugins/endpoints/models/credential.py:7-10` says `reenrollblocked` keeps a revoked PC from enrolling again, listing "a compromised PC" among the uses. The containment does not exist. On a 401 the agent deletes its credential and clears it (`agent/ShopDbEndpointAgent.Core/AgentCredentials.cs:107-135`), then every later request takes the shared-key branch (`agent/ShopDbEndpointAgent.Core/AgentHttpFactory.cs:119-120`). Server side, `shared_key_refused_for` refuses the shared key only when an active credential exists (`plugins/endpoints/api/agentauth.py:129-134`), and revoke has just cleared that (`plugins/endpoints/api/enrollment.py:773-825`).
This is documented intended behaviour, not an undetected gap: `plugins/endpoints/api/agentauth.py:11-14` states it, and `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:379-380` says outright that the server admits the shared key again. What needs fixing is the model comment, which implies a containment the design never provides. There is no per-PC deny until the shared key is off fleet-wide.
### 25. The bootstrap token travels in the query string and is written to both access logs
`_resolve_token` takes the whole authorization from the URL (`plugins/endpoints/api/bootstrap.py:182`) for both token routes (`plugins/endpoints/api/bootstrap.py:60-61`). Both hops log the full request line: `nginx.conf:36` is `access_log /dev/stdout combined`, and the gunicorn access format uses `%(r)s` (`docker-compose.yml:213-214`). The IIS deployment logs `cs-uri-query` in its W3C log by default too.
The same range deliberately stopped logging the setup token for exactly this reason (`shopdb/__init__.py`, setup-token block), and the audit rows this module writes carry only `tokenprefix` (`plugins/endpoints/api/bootstrap.py:192-195`).
**Bounded by:** what a replayed token buys is bounded and carries no secret. `download_bootstrap_package` is explicitly keyless (`plugins/endpoints/api/bootstrap.py:532-535`) and the script it serves contains only the base URL and pins. The reader also needs container or IIS log access, which is already privileged.
### 26. The unauthenticated enroll queue can be filled to its cap
`enroll_endpoint` takes no auth and writes a row for every unseen (hostname, serialnumber) pair (`plugins/endpoints/api/enrollment.py:350-358`). The bounds are a per-IP fixed window of 300 per 60 seconds (`plugins/endpoints/api/enrollment.py:65-66`) and a pending cap of 500 (`plugins/endpoints/api/enrollment.py:255`). Junk rows count toward the cap: an unknown hostname returns `_pending(row, REASON_UNKNOWN_HOSTNAME)` at `plugins/endpoints/api/enrollment.py:411`, which leaves the row PENDING. There is no bulk-reject, prune or stale-row expiry in that module. The limiter's counter is per-process (`plugins/endpoints/api/ratelimit.py:10-11`), so the real budget is the configured maximum times the worker count.
**Bounded by:** while `sharedkeyallowed` is on (the default), a PC that cannot enroll is not offline; it keeps reporting on the fleet key. This only becomes an outage for new bays once the shared key is turned off at the end of the rollout.
### 27. The kiosk confirms an employee name for any badge it is handed
`kiosk_session` normalises the posted badge and returns the resolved employee name in the response body (`plugins/usb/api/selfhosted.py:346-371`, resolver at `:34-57`); the external mode does the same (`plugins/usb/api/routes.py:904-960`). The route carries no auth decorator (`plugins/usb/api/routes.py:905-906`) and there is no rate limiter anywhere in either file. `kiosk_others` additionally returns the holder name for every checked-out drive to the same caller, gated only on `usb_kiosk_allow_return_for_others`, which defaults true (`plugins/usb/plugin.py:169-172`).
Before `da91a2915` this needed the per-device token, so it was confined to screens an administrator had provisioned. It is now reachable by anything inside `usb_kiosk_allowed_cidrs`.
**Bounded by:** the kiosk must be enabled and the caller must be inside a deliberately configured CIDR. The data is badge-to-name, which is printed on the badges themselves and visible to any signed-in user, and the surface is read-only. An unrecognised badge answers 422 rather than silently.
### 28. The kiosk allowlist settings accept a world-open range with no server-side validation
`usb_kiosk_allowed_cidrs` (`plugins/usb/plugin.py:154-168`) and `printedparts_kiosk_allowed_cidrs` (`plugins/printedparts/plugin.py:176-188`) are plain string settings with no validator, and the settings PUT path applies none. `ip_in_cidrs` parses each entry with `ipaddress.ip_network(part, strict=False)` (`shopdb/utils/ipallowlist.py:55`), so `0.0.0.0/0` and `::/0` parse and match everything. The only shape check is client-side and cosmetic (`frontend/src/components/NetworkAllowlistPicker.vue:84-87`), and `0.0.0.0/0` satisfies its regex.
The realistic route in is an administrator who cannot get the allowlist to match, most likely because the client-IP rule is missing and every caller reads as loopback, and who widens the range to make it work.
### 29. The kiosk provisioning link hands out the token in a URL
`kiosk_provisioning_link` returns `/usb-kiosk?token=<token>` (`plugins/usb/api/routes.py:849`) and the printed-parts twin returns `/parts-kiosk?token=<token>` (`plugins/printedparts/api/routes.py:578`). Both are gated on `settings.edit` and both write an audit row first (`plugins/usb/api/routes.py:846-848`, `plugins/printedparts/api/routes.py:574-576`). The token then lands in the IIS or nginx access log, browser history and any clipboard history on the admin's PC, and nothing rotates it after a reveal.
The same file elsewhere states the contrasting reasoning: `kiosk_session` is a POST specifically so the badge does not end up in a proxy access log the way a query string would (`plugins/usb/api/routes.py:914-916`).
**Qualification:** the query-string model is not introduced here. It is the mechanism the token setting itself prescribes (`plugins/usb/plugin.py:146-151`), so the new route only surfaces a URL the product already documented. This is hardening, not a regression.
### 30. Every deployment warning about the client-IP rule still names only GE-Enforce
Three operator-facing places enumerate what depends on the `X-Forwarded-For` rewrite rule, and none was updated when the kiosk allowlists were added: `deploy/windows/installer/shopdb-install.ps1:2520-2522`, `deploy/windows/installer/shopdb-preflight.ps1:322-325` and `deploy/windows/web.config:51-58`. None names `usb_kiosk_allowed_cidrs` or `printedparts_kiosk_allowed_cidrs`, which are now the only allowlists in the product gating unauthenticated writes. The dependency is stated only in the settings descriptions (`plugins/usb/plugin.py:161-168`, `plugins/printedparts/plugin.py:183-187`), which the person building the server does not read.
**One claim not to carry forward:** the remediation line offered at `deploy/windows/installer/shopdb-install.ps1:2522`, "or use token authentication", is correct for the kiosks. The per-device token is a genuine alternative to the allowlist on both.
### 31. This page, and three other documents, attributed a credential policy to ADR-016 that ADR-016 does not contain
The References section of the 2026-09-11 version of this page described ADR-016 as "Credential Delivery (device token authentication)". That is wrong, and the same misattribution is repeated at `docs/wiki-drafts/README.md:316-323`, `CLAUDE.md:119` and the assistant quick-reference file line 135. Grepping the ADR set for device tokens returns nothing in any ADR.
The worst form is in the SECURITY.md page that `docs/wiki-drafts/README.md:429` queues for publication. Under "ADR-016: Credential Delivery" it lists device tokens, personal access tokens and environment variables as the delivery mechanisms and then bans, at `docs/wiki-drafts/README.md:323`, "Hardcoded in source, registry keys, git history". ADR-016 decides none of that. Its Decision is a fleet credential store with Fernet at rest, a targeting and precedence resolver reusing `plugins/geenforce/filters.py`, an explain endpoint and `GET /api/collector/credentials` behind a dedicated `credentials.fetch` scope. Its own alternatives section recommends registry-only provisioning as a worthwhile first step.
The blanket ban on registry keys also contradicts shipped code on this branch: `plugins/computers/client/Report-AssetToShopDB.ps1:367-374` reads the collector key from `HKLM:\SOFTWARE\GE\ShopDB`, and `plugins/endpoints/client/Set-ShopDBServerPins.ps1:4-20` writes the server trust pins there by design.
**Status:** the References section of this page is corrected below. The other three are not, and the SECURITY.md draft should not be published as written. Note that `gh-wj` carries no `docs/adr` directory at all, so there is no ADR text on this branch to correct a reader who follows the wiki.
**Related, and unverifiable from this branch:** ADR-016's own mechanism is still unbuilt. Grepping `shopdb/` and `plugins/` for `credentials.fetch`, `collector/credentials` and `Fernet` returns nothing, and `plugins/endpoints/plugin.py:163-253` declares only `endpoints.*` permissions. The per-PC credential programme in this range is a different subject under the same word: the agent's own identity secret, not a store that delivers third-party credentials to a PC. The design is sound and consistent with ADR-016's intent, but nothing on this branch records it as a decision. That record belongs on the lineage that holds `docs/adr`.
### 32. Kiosk check-in records no wipe or scan state, so the new CMMC audit columns stay blank
`35d4fcd41` added `sanitized` and `scanned_viruses` to the USB audit report on the stated grounds that custody without the state of the media on return is half the story. Both values are written only on the authenticated return path (`plugins/usb/api/selfhosted.py:275-280`, `plugins/usb/api/routes.py:562-571`). The kiosk returns write neither: `kiosk_checkin` updates only check-in time, SSO and name (`plugins/usb/api/selfhosted.py:527-531`), and the external kiosk insert lists only badge, device, action, timestamp and locker (`plugins/usb/api/routes.py:1157-1162`).
This is an honest blank, not a false record: `auditflag` renders the NULL as an empty cell rather than "no" (`plugins/usb/api/selfhosted.py:547-559`), and the tri-state is deliberate. But `da91a2915` has just widened the kiosk from a few provisioned screens to every PC on the allowed networks, so it is about to become the path most returns take.
**Correction to note:** the signed-in path does not answer the scan question either. `USBDetail.vue:226-230` and `USBList.vue:267-271` send only `sanitized`, never `scanned_viruses`, so `plugins/usb/api/routes.py:564` always receives None and that column is blank for every return through any first-party UI.
## Verified as sound
Read on 2026-09-18 and found correct. Recorded so the next review does not re-litigate them.
- **Credential storage on the server.** Only a sha256 hash is persisted, plus a 12-character display prefix. The secret exists once, in the 201 enroll response (`plugins/endpoints/api/enrollment.py:212-222`). No admin route re-reads or re-displays it; `issue_credential` has exactly two callers, both inside `enroll_endpoint`.
- **Cross-asset binding.** A credential caller cannot act for another PC. `bind_hostname` returns `ASSET_MISMATCH` for a hostname resolving elsewhere (`plugins/endpoints/api/agentauth.py:206-231`), and every agent-facing route goes through it.
- **Admin surface authorization.** Approve, reject, revoke, allow-reenroll, the settings PUT, the migration wave and all three bootstrap-token routes are `@require_role('admin')` (`plugins/endpoints/api/enrollment.py:642`, `:755`, `:773`, `:827`, `:852`, `:1134`; `plugins/endpoints/api/bootstrap.py:420`, `:475`, `:486`).
- **Tray holds no secret.** The tray reaches the SYSTEM agent over a named pipe with a SYSTEM and Administrators full, INTERACTIVE read-write ACL and FirstPipeInstance anti-squat (`agent/ShopDbEndpointAgent/AgentPipeServer.cs:126-146`); no pipe operation returns the credential (`:246-256`).
- **Bootstrap token scope and lifetime.** Hashed at rest, expiry mandatory and bounded to 1 to 168 hours, uses bounded to 1 to 500, revocable, and unknown, malformed, expired, revoked and exhausted all answer an identical 404 so the URL confirms nothing (`plugins/endpoints/models/bootstrap.py:197-212`, `plugins/endpoints/api/bootstrap.py:109-111`, `:176-189`). The package it serves never carries the collector key.
- **GE-Enforce payload scoping.** `/payload/<sha256>` enforces the token's resource scope before the ETag shortcut, so a bound token cannot even confirm another scope's hash exists (`plugins/geenforce/api/routes.py:252-283`).
- **`GET /api/plugins/settings-routes` is no longer anonymous.** It moved to `@require_permission('settings.view')` (`shopdb/core/api/plugins.py:47`) and was removed from the public-read allowlist with the reasoning recorded (`tests/test_core/test_public_read_surface.py:29-34`).
- **The kiosk admission gate fails closed in every direction it claims to.** Empty setting matches nothing, an unparseable caller address matches nothing, a malformed CIDR entry is skipped rather than widening the match (`shopdb/utils/ipallowlist.py:44-57`), and the allowlist cannot override the enabled flag. Device-token comparison uses `hmac.compare_digest` in both kiosks (`plugins/usb/api/routes.py:762`, `plugins/printedparts/api/routes.py:536`). An unadmitted caller gets 404, not 403.
- **No secrets committed in the range.** Every `sdbe_` and `sdbb_` hit in the diff is a prefix constant, a regex, a doc example or a test fixture. `certs/` remains ignored except its README, openssl.cnf and the two regeneration scripts.
## Performance Findings
### Agent Polling Inefficiency (99.9% improvement opportunity)
### Agent Polling Inefficiency [LARGELY ADDRESSED]
**Current behavior:** Agent polls `/api/endpoints/inventory` every 30 seconds to POST full inventory (hostname, IP, apps, drives, processes).
**2026-09-18 verdict:** the behaviour the 2026-09-11 page measured no longer matches the code.
**Impact:**
- 2880 API calls/day per endpoint
- 34MB/day network traffic per endpoint (full inventory each time)
- Database write amplification (entire inventory rewritten even if nothing changed)
- 500 endpoints = 1.44M API calls/day, 17GB/day traffic
- The cadence is not 30 seconds. `agent/ShopDbEndpointAgent/appsettings.json:5` ships `cadenceseconds: 900` with 60 seconds of jitter and 30 seconds of startup jitter, so roughly 96 check-ins per day per endpoint rather than 2880.
- Application inventory is a delta, not a full rewrite. `plugins/endpoints/plugin.py:1187` onward builds an incoming fingerprint map, validates each fingerprint as 64 lowercase hex, and applies only the difference; a genuine unique-key collision is caught inside a savepoint and reconciled row by row (`plugins/endpoints/plugin.py:1301-1336`, `:1359-1410`) rather than sinking the check-in.
- Two hotfixes in this range removed row-level locks and hot-path logging from the collector check-in path (`affe455c5`, `1ed0c4d21`).
**Optimization:** Delta updates
1. Agent calculates inventory delta locally (added/removed apps, changed IP, etc.)
2. POST only changes: `{"added_apps": [...], "removed_apps": [...], "changed_ip": "10.1.2.3"}`
3. Server applies delta to existing record
4. Fallback to full inventory on agent startup or if delta rejected
**Expected improvement:**
- 99% fewer writes (only ~1% of polls have changes)
- 99.5% less network traffic (delta is <500 bytes vs 11KB full inventory)
- Reduced database load (updates instead of full rewrites)
**Status:** Planned for Q1 2027. Requires agent + server changes.
The traffic and write-volume figures in the 2026-09-11 text were site measurements against the old cadence and should not be carried forward.
## Remediation Summary
| Finding | Severity | Status | Timeline |
|---------|----------|--------|----------|
| Unauthenticated inventory updates | Critical | FIXED | 2026-09-11 |
| Deployment credential exposure | Critical | MITIGATED | 2026-09-11 |
| No certificate validation | Critical | PLANNED | Q4 2026 |
| Hardcoded admin credentials | Critical | FIXED | 2026-09-11 |
| Collector key in git | High | FIXED | 2026-09-11 |
| Passwords in mcp-servers.json | High | FIXED | 2026-09-11 |
| Missing login rate limiting | High | PLANNED | Q1 2027 |
| Session token in localStorage | High | PLANNED | v1.0 |
| Audit log retention | High | PLANNED | Q4 2026 |
| Endpoint events retention | High | PLANNED | Q4 2026 |
| Docker log rotation | Medium | PLANNED | Q4 2026 |
| Agent delta updates | Performance | PLANNED | Q1 2027 |
| Finding | Severity | Status as of 2026-09-18 | Evidence |
|---------|----------|-------------------------|----------|
| 1. Unauthenticated inventory updates | Critical | CLOSED on the routes that exist | `shopdb/core/api/collector.py:251` |
| 2. Collector key unprotected in registry | Critical | STILL OPEN, see finding 19 | `plugins/endpoints/api/siteagent.py:328` |
| 3. No certificate validation in agent | Critical | FIXED by SPKI pinning | `agent/ShopDbEndpointAgent.Core/AgentHttpFactory.cs:291` |
| 4. Hardcoded admin credentials | Critical | PARTLY FIXED, dev-only guard added | `shopdb/cli/__init__.py:619` |
| 5. Collector key in git | High | FIXED; no pre-commit secret scan exists | `.gitignore:174`, `.githooks/pre-commit:5` |
| 6. Passwords in mcp-servers.json | High | FIXED on this branch; keys not rotated | `.gitignore:176` |
| 7. Missing login rate limiting | High | FIXED, limiter plus account lockout | `shopdb/core/api/auth.py:35`, `:113` |
| 8. SQL injection in asset search | High | FALSE POSITIVE, re-confirmed | `plugins/usb/api/routes.py:277` |
| 9. Session token in localStorage | High | UNCHANGED | `frontend/src/stores/auth.js:6` |
| 10. Audit log retention | High | BUILT, off by default, no scheduler | `shopdb/core/services/auditretention.py:123` |
| 11. Endpoint events retention | Medium | STILL OPEN, no purge code exists | `plugins/endpoints/models/endpoint.py:333` |
| 12. HTTPS enforcement in dev | Medium | CHANGED, TLS bypass removed | `frontend/vite.config.js:5` |
| 13. Docker log rotation | Medium | STILL OPEN | `docker-compose.yml` (no logging block) |
| 14-18. Additional medium findings | Medium | PARTLY ANSWERED, source document removed | `shopdb/__init__.py:334`, `8eee6f19d` |
| 19. Shared key in registry with no ACL | High | FIXED 2026-09-18 (installed bays until reinstall) | `plugins/endpoints/api/siteagent.py`, `Protect-HklmShopdbKey` |
| 20. Legacy collector routes ignore the refusal | Medium | OPEN | `shopdb/core/api/collector.py:349` |
| 21. Enroll rejection is not durable | Medium | OPEN | `plugins/endpoints/api/enrollment.py:390` |
| 22. Bootstrap stages elevated script in Windows\Temp | Medium | OPEN | `plugins/endpoints/api/bootstrap.py:68` |
| 23. Kiosk admission spoofable in proxy mode | Medium | OPEN | `deploy/windows/installer/shopdb-install.ps1:2509` |
| 24. Revoke falls back to the shared key | Low | BY DESIGN, model comment wrong | `plugins/endpoints/models/credential.py:8` |
| 25. Bootstrap token in query string | Low | OPEN | `plugins/endpoints/api/bootstrap.py:182` |
| 26. Enroll queue can be filled to cap | Low | OPEN | `plugins/endpoints/api/enrollment.py:350` |
| 27. Badge to name oracle on the kiosk | Low | OPEN | `plugins/usb/api/selfhosted.py:346` |
| 28. Allowlist accepts 0.0.0.0/0 | Low | OPEN | `plugins/usb/plugin.py:154` |
| 29. Provisioning link carries the token in a URL | Low | OPEN, pre-existing mechanism | `plugins/usb/api/routes.py:849` |
| 30. Client-IP warnings name only GE-Enforce | Low | OPEN | `deploy/windows/installer/shopdb-install.ps1:2520` |
| 31. ADR-016 misattributed in four documents | Low | ONE FIXED HERE, three open | `docs/wiki-drafts/README.md:316` |
| 32. Kiosk check-in records no wipe or scan | Low | OPEN | `plugins/usb/api/selfhosted.py:463` |
| Agent polling inefficiency | Performance | LARGELY ADDRESSED | `agent/ShopDbEndpointAgent/appsettings.json:5` |
## Testing Performed
**Authentication enforcement:**
```bash
# Inventory endpoint now requires device token
curl -X POST http://localhost:5001/api/endpoints/inventory
# 401 Unauthorized
This refresh was a read of the code at `35d4fcd41`, not a live exercise against a server. What was run:
curl -X POST http://localhost:5001/api/endpoints/inventory \
-H "X-API-Key: shopdb_device_test123" -d '{...}'
# 200 OK
**The naming and style gate:**
```bash
bash scripts/check-naming-and-style.sh
# exit 0, all checks pass
```
Its ADR-015 site-literal grep is not vacuous: a control pattern over the same six directories returns 1995 lines.
**Kiosk network allowlist suite:**
```bash
pytest tests/test_plugins/test_kiosk_network_allowlist.py
# 20 passed
```
**Credential scanning:**
**Tracked-secret scan:**
```bash
# No hardcoded passwords in source
bash scripts/check-naming-and-style.sh
# PASS (includes credential pattern checks)
# No collector keys in tracked files
git ls-files | xargs grep -l "shopdb_pat_" | wc -l
# 0
```
**Registry key cleanup:**
```powershell
# Verify registry key removed
Get-ItemProperty "HKLM:\SOFTWARE\ShopDB\Agent" -Name CollectorApiKey -ErrorAction SilentlyContinue
# (none found)
```
### Gate coverage gaps found while testing
- **The ADR-015 site-literal scan does not look at the C# agent.** The grep is scoped to `plugins/ shopdb/ scripts/ deploy/ frontend/src/ tools/` with `--include` on .py, .vue, .js, .ps1, .psm1, .sh, .iss, .json and .html (`scripts/check-naming-and-style.sh:434-440`). `agent/` is absent and `.cs` is not an included extension. Roughly 2500 lines of C# were added under `agent/` in this range, all of which ships to a sister site. Re-running the assembled site patterns over `agent/` by hand returns one hit, prose in `agent/ShopDbEndpointAgent/ShopDbEndpointAgent.csproj:108`, which ADR-015 explicitly permits. The tree is clean; the gate simply would not notice if it stopped being clean. ADR-015's own 2026-08-14 amendment widened the scan on exactly this reasoning, because the literals that reached a second site were in the client scripts and the installer. The C# agent is now that client channel.
- **The gate has no credential pattern check.** See finding 5. The 2026-09-11 page's claim that it includes one is not true of this branch.
## Recommendations
**Immediate (P0):**
1. Deploy per-endpoint certificate pinning (Q4 2026)
2. Mass agent update to remove registry keys (Q4 2026)
3. Rotate exposed collector keys (user accepted risk, not rotating)
**Do first:**
1. ACL the collector key registry value in the generated installer, or finish enrollment and clear it (finding 19).
2. Make the shared-key refusal cover `/pc`, `/apps`, `/heartbeat` and `/bulk`, or retire them (finding 20).
3. Make an admin rejection durable so a new nonce cannot restart the cycle (finding 21).
4. Stage the bootstrap script somewhere a standard user cannot pre-create, or verify it against a server-stamped hash (finding 22).
5. Verify in proxy mode that a proxy is really in front, the way direct mode already verifies the rewrite rule (finding 23).
**Short-term (P1):**
1. Implement audit log retention (Q4 2026)
2. Configure Docker log rotation (Q4 2026)
3. Add login rate limiting (Q1 2027)
**Then:**
1. Do not publish the SECURITY.md wiki page as drafted; it attributes a policy to ADR-016 that ADR-016 does not contain and bans the registry storage the fleet depends on (finding 31).
2. Name the two kiosk allowlists in the installer, preflight and web.config warnings (finding 30).
3. Add server-side validation that rejects a world-open range on both kiosk allowlist settings (finding 28).
4. Move the bootstrap token out of the query string, or accept it and document the log exposure (finding 25).
5. Bound or expire pending enrollment rows so the queue cannot be held at its cap (finding 26).
6. Correct the `reenrollblocked` comment so it does not claim containment of a compromised PC (finding 24).
**Long-term (P2):**
1. Implement agent delta updates (Q1 2027)
2. Migrate sessions to HttpOnly cookies (v1.0)
3. Add CSP headers (v1.0)
4. Purge credentials from git history (v1.0)
**Carried over from 2026-09-11 and still open:**
1. Retention for `endpointevents` (finding 11).
2. Docker log rotation (finding 13).
3. A retention value and a schedule for the audit log, now that the tooling exists (finding 10).
4. HttpOnly cookie sessions for non-kiosk users (finding 9).
5. Recover or re-derive the detail behind findings 14 to 18, which left the tree with `8eee6f19d`.
**Extend the gates:**
1. Add `agent/` and `.cs` to the ADR-015 site-literal scan.
2. Add a credential pattern check to the gate so the pre-commit hook actually blocks a committed key.
## References
- Security Fix Implementation Guide: `.claude/memory/security_fix_implementation.md`
- Maintenance Baseline: 936 hrs/year operational burden, includes security tasks
- ADR-016: Credential Delivery (device token authentication)
- ADR-XXX: Per-Endpoint Certificate Authentication (planned)
- Per-PC credential rollout runbook: `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`
- Agent contract: `plugins/endpoints/AGENT-CONTRACT.md`
- ADR-006: Collector API, the single dynamic dispatch route and its response contract
- ADR-015: Site-specific configuration, and the 2026-08-14 amendment that set the scan's scope
- ADR-016: Credential Delivery. A fleet credential store with Fernet at rest and a `credentials.fetch` scope. It is **not** about device tokens, and it does not ban registry storage. Still unbuilt on this branch. See finding 31.
- The ADR set is not in this repository. It lives on the product lineage; `gh-wj` has no `docs/adr` directory.
- The 2026-09-12 detailed audit reports were removed from the product tree by `8eee6f19d` and are kept with the site's working records. the assistant memory directory does not exist here, so the 2026-09-11 page's pointers into it are dead.