ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/wj' into wilm

Browse Source
This commit is contained in:
benjamin.baker
2026-09-18 17:25:24 -04:00
parent 2ff7b7e906 f16b463736
commit e00d86827b
64 changed files with 35788 additions and 20614 deletions
Download Patch File Download Diff File Expand all files Collapse all files

155
.claude/QUICK-REFERENCE.md
View File

@@ -11,29 +11,58 @@ Fast lookup for architectural decisions and contracts.
- **ADR-008**: Plugin migration ownership - PLUGIN_TABLE_OWNERS, migration chains
- **ADR-013**: Plugin catalog - Lean builds, plugin manifest
- **ADR-015**: Site-specific configuration - Where site literals belong
- **ADR-016**: Credential delivery - Device tokens, kiosk auth
- **ADR-016**: Credential delivery to the fleet - cmdkey credentials for bays,
targeting and rotation. ACCEPTED but NOT built. It says nothing about device
tokens or kiosk auth, and nothing forbids the registry: it names registry
provisioning as a recommended first step
## Import Surface (stable API)
Every name below is in `shopdb/api/__init__.py` `__all__`. That list is the
surface; a name that is not in it is not importable from `shopdb.api`, whatever
else in the tree defines it.
```python
from shopdb.api import (
# Database
db,
# Core Models
Asset, Employee, User, Building, MapLevel,
# Plugin Base
BasePlugin,
# Auth/Authz
requires_role, requires_device_token,
# Core models
Asset, AssetType, AssetStatus, AssetRelationship,
Location, BusinessUnit, Model, OperatingSystem, Vendor,
User, Role, Setting,
# Auth / authz
require_permission, require_role, require_any_permission,
caller_holds, caller_is_role,
client_ip, ip_in_cidrs, request_ip_allowlisted,
# Responses and paging
success_response, error_response, ErrorCodes,
get_pagination_params, paginate_query, paginated_response,
# Utilities
badges, datevalues, queryargs
audit_log, cache,
normalise_badge, badge_kind, BadgeShapeError,
parse_date, parse_date_fields, DateValueError,
intarg, intargs, QueryArgError,
resolve_asset_position,
)
```
NOT on this surface. This block used to name all six, so the first import a new
agent copied raised ImportError:
- `BasePlugin`, `PluginMeta` - `from shopdb.plugins.base import ...`, which is
what every `plugins/*/plugin.py` in the tree does.
- `Building`, `MapLevel` - `shopdb.core.models.maplevel`. An asset's position
normally comes back through `resolve_asset_position`, which IS exported.
- `Employee` - no such model. The directory lives in a read-only external
database reached through `employee_connection`, and the plugin's own row type
is `DirectoryEmployee` in `plugins/employees/models/`.
- `badges`, `datevalues`, `queryargs` - module names under `shopdb/utils/`, not
exported. Import the functions listed above instead.
## Plugin Structure
```
@@ -55,9 +84,12 @@ plugins/{name}/
## Database Naming
- **Tables**: `printedpart`, `mappeddrives` (lowercase concatenated)
- **Columns**: `serial_number`, `last_seen` (snake_case)
- **Enforced**: `scripts/check-naming-and-style.sh`
- **Tables**: `machines`, `mappeddrives` (lowercase concatenated, plural)
- **Columns**: `machineid`, `lastzabbixsync`, `isactive` (lowercase concatenated,
singular). NOT snake_case
- **Indexes**: `idx_machines_locationid`. The only place an underscore is allowed
- **Enforced**: `scripts/check-naming-and-style.sh`, on model attributes AND on the
column names a migration creates
## Branch Model
@@ -132,19 +164,98 @@ PLUGIN_TABLE_OWNERS = {
}
```
## Device Tokens (ADR-016)
## Kiosk Access
Kiosks require device authentication:
Not ADR-016 - that ADR is about cmdkey credentials for bays and is not built.
A kiosk route admits a caller on EITHER ground, both gated on the kiosk's own
enable flag:
This is `plugins/usb/api/routes.py` as it stands, not a paraphrase:
```python
from shopdb.api import requires_device_token
from shopdb.api import Setting, client_ip, ip_in_cidrs
@bp.route('/kiosk/endpoint')
@requires_device_token(['usb_kiosk', 'parts_kiosk'])
def kiosk_view():
return render_template('kiosk.html')
KIOSK_NEVER_A_CLIENT = '127.0.0.0/8, ::1/128'
KIOSK_TRUSTED_PROXIES = 'usb_kiosk_trusted_proxies'
def _kiosk_client_ip():
return client_ip(request)
def _kiosk_address_is_infrastructure(address):
if ip_in_cidrs(address, KIOSK_NEVER_A_CLIENT):
return True
return ip_in_cidrs(address, Setting.get(KIOSK_TRUSTED_PROXIES))
def _kiosk_ip_allowlisted():
address = _kiosk_client_ip()
if _kiosk_address_is_infrastructure(address):
return False
return ip_in_cidrs(address, Setting.get('usb_kiosk_allowed_cidrs'))
def _kiosk_open():
if not _kiosk_enabled(): # the site has to turn it on first
return False
return _kiosk_token_presented() or _kiosk_ip_allowlisted()
```
THIS IS NOT `request_ip_allowlisted(request, cidrs)`. That helper is exactly
`ip_in_cidrs(client_ip(request_obj), rawcidrs)` and nothing else
(`shopdb/utils/ipallowlist.py`), so it has no infrastructure refusal. The USB
kiosk refuses an address that belongs to the path rather than to a caller BEFORE
it consults the allowlist, and that refusal is the whole point: see the topology
below. Writing the helper here instead of the real shape is how the two got
confused.
- The allowlists are per kiosk (`usb_kiosk_allowed_cidrs`,
`printedparts_kiosk_allowed_cidrs`) and ship EMPTY, so a site that enters no
ranges stays token-only.
- `ip_in_cidrs` fails closed on an empty setting, an unparseable caller address
and a malformed entry. Never copy a CIDR parser into a plugin.
- A refused caller gets 404, not 403: these routes do not advertise themselves.
- The two kiosks HAVE DIVERGED. `plugins/printedparts/api/routes.py:556` still
calls `request_ip_allowlisted(request, Setting.get('printedparts_kiosk_allowed_cidrs'))`
with no infrastructure check, so a site that allowlists loopback to work
around the topology below opens that kiosk to every caller. Read the plugin
you are editing; do not assume the other one matches.
### Why the infrastructure refusal exists (the shipped Windows path)
Trace it, do not guess. IIS hands the request to HttpPlatformHandler, which
connects to waitress on 127.0.0.1 with
`--trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for
--trusted-proxy-count=1` (`deploy/windows/web.config`). waitress, seeing the
trusted peer, overwrites `environ['REMOTE_ADDR']` from the LAST X-Forwarded-For
hop. `client_ip()` then returns `strip_port(request.remote_addr)`
(`shopdb/utils/clientip.py`) - it never reads the header itself.
The catch: IIS does not set X-Forwarded-For on its own, and the rewrite rule
that would is commented out in `web.config` - it SHIPS DISABLED because it needs
the URL Rewrite module. So on a default install there is no header, waitress has
nothing to promote, and every caller arrives as 127.0.0.1. `TRUSTED_PROXY_COUNT`
defaults to 0 (`shopdb/config.py`), so ProxyFix is not installed either, and it
would not help: waitress already did that job.
Consequences a kiosk change has to respect:
- Until someone enables the rule (`shopdb-install.ps1 -ClientIpSource direct`
installs URL Rewrite, allows `HTTP_X_FORWARDED_FOR` at server level and
uncomments it), a CIDR allowlist admits nobody and the kiosk is token-only.
That is the fail-closed direction and it is correct.
- Allowlisting 127.0.0.0/8 to "fix" that would admit everyone, which is why
`KIOSK_NEVER_A_CLIENT` refuses it whatever the setting says.
- `-ClientIpSource proxy` leaves the rule off ON PURPOSE: behind ARR or a load
balancer, REMOTE_ADDR is the proxy and the rule would destroy the real client
address. There the upstream proxy sets the header.
## Fleet Credentials
- Agent and GE-Enforce reporters run as SYSTEM, so there is nobody to prompt.
The collector key, base URL and server pins come from
`HKLM\SOFTWARE\GE\ShopDB`. That hive is the design, not a violation.
- An enrolled PC's own per-PC credential is DPAPI-protected in a SYSTEM-only
directory and sent as `X-Endpoint-Credential`; the shared fleet key still
works for a PC that has not enrolled and is refused for one that has.
## Wiki Location
Full docs: `~/Documents/github/shopdb-flask.wiki/`

44
.github/workflows/ci.yml vendored
View File

@@ -89,8 +89,19 @@ jobs:
lean-build:
# ADR-013 Phase 5: prove a per-site build carries only its chosen plugins.
# Builds a lean site (machines + printers) and asserts an omitted plugin's
# code is absent from the bundle - the delete-a-plugin guarantee in CI.
# The profile is the WJ plugin set, so this job proves the build this branch
# actually ships, not a two-plugin toy. The omission guarantee is what is
# under test, so the assertions below name a component from EVERY omitted
# plugin: a set that grows without its check growing is how a leak ships.
#
# Chosen (12): backups computers endpoints guides knowledgebase machines
# network notifications printers slides tools warranty
# Omitted (5): employees geenforce measuringtools printedparts usb
# 'applications' is manifest-less core and ships in every build (ADR-013).
# endpoints hard-depends on computers, which is already chosen, so the
# closure adds nothing - if that changes, resolve_plugin_closure.py pulls
# the dependency in and the absence assertions below start failing, which is
# the signal to update this list rather than to loosen the check.
runs-on: arc-runner-set
steps:
- uses: actions/checkout@v5
@@ -100,21 +111,28 @@ jobs:
cache: npm
cache-dependency-path: frontend/package-lock.json
- run: cd frontend && npm ci
- name: Build lean site (machines + printers)
- name: Build lean site (WJ plugin set)
run: |
printf '{ "site": "ci-lean", "plugins": ["machines", "printers"] }' \
printf '%s' '{ "site": "ci-lean", "plugins": ["backups", "computers", "endpoints", "guides", "knowledgebase", "machines", "network", "notifications", "printers", "slides", "tools", "warranty"] }' \
> /tmp/lean-profile.json
bash scripts/build-site.sh /tmp/lean-profile.json /tmp/leansite
- name: Assert omitted plugin code is absent, chosen present
run: |
assets=/tmp/leansite/frontend-dist/assets
for code in PartsKiosk ManifestEditor USBLabelBatch KnowledgeBaseDetail; do
# One component per OMITTED plugin: employees, geenforce,
# measuringtools, printedparts, usb.
for code in EmployeeDirectory ManifestEditor MeasuringToolsList PartsKiosk USBLabelBatch; do
if grep -rqoh "$code" "$assets"/*.js; then
echo "FAIL: omitted-plugin code '$code' leaked into the lean bundle"
exit 1
fi
done
for code in MachineDetail PrinterDetail; do
# One component per CHOSEN plugin, so a plugin silently dropped from
# the bundle fails here instead of on the floor.
for code in BackupHistory PCsList EndpointsList GuidesList \
KnowledgeBaseDetail MachineDetail NetworkDevicesList \
NotificationsList PrinterDetail SlideManager ToolsIndex \
WarrantiesList; do
grep -rqoh "$code" "$assets"/*.js || {
echo "FAIL: chosen-plugin code '$code' missing from the lean bundle"
exit 1; }
@@ -124,8 +142,18 @@ jobs:
grep -rqoh "ApplicationsList" "$assets"/*.js || {
echo "FAIL: core page 'ApplicationsList' missing from the lean bundle"
exit 1; }
test -d /tmp/leansite/plugins/machines
test ! -d /tmp/leansite/plugins/printedparts
for plugin in backups computers endpoints guides knowledgebase \
machines network notifications printers slides tools \
warranty; do
test -d "/tmp/leansite/plugins/$plugin" || {
echo "FAIL: chosen plugin '$plugin' missing from the backend tree"
exit 1; }
done
for plugin in employees geenforce measuringtools printedparts usb; do
test ! -d "/tmp/leansite/plugins/$plugin" || {
echo "FAIL: omitted plugin '$plugin' staged into the backend tree"
exit 1; }
done
echo "lean build verified: only chosen plugins present"
# NOTE: the MySQL-8 migration/seed job (fresh `flask db upgrade` + every

181
CHANGELOG.md
View File

@@ -702,6 +702,187 @@ ADR-007 and ADR-002.
`plugins/tools/frontend/labelPages.js`, where the column contract and the
two-sided page order are covered by tests.
### Added
- **Every PC gets its own endpoint credential, and enrolls to get it.** The
fleet authenticated with one shared collector key: every bay held the same
secret, one of them leaking meant reissuing all of them, and a PC could post
a check-in naming any asset it liked. A PC now enrolls once and is
authenticated as itself thereafter. `POST /api/endpoints/enroll` issues a
credential to a PC that matches exactly one computer on hostname and BIOS
serial and holds none yet; anything else - unknown hostname, a duplicate, a
serial that disagrees, a PC already enrolled - is recorded as a request an
administrator approves or rejects, and the agent's persisted nonce is what
binds an approved request back to the PC that asked. Enroll never refuses, so
a PC installed with no key at all still turns up in the queue instead of
vanishing. One auth helper serves every agent route: with
`X-Endpoint-Credential` the asset comes from the credential, so a request
naming a different PC is refused with `ASSET_MISMATCH` rather than believed,
and a revoked credential answers `CREDENTIAL_INVALID` so the agent
re-enrolls rather than going quiet. The core collector takes the credential
for the computers, printers and endpoints collectors through two hooks on
`shopdb.api` (contract 0.33.0).
Settings > Endpoints > Enrollment is where this is run. The Migration tab
shows readiness counts, each PC's state with the reasons it is blocked,
"Upgrade selected" and "Upgrade next 25", and live wave progress; a wave
packages the agent once and queues the existing per-PC upgrade. The
Credentials tab lists enrolled PCs with revoke, pending requests with
approve and reject, and the two switches. The fleet list gained a Credential
column and filter, and a PC's detail page shows its credential state, with
Revoke and Allow re-enrollment for an administrator. Migration 0023 creates
`endpointcredentials`, `endpointenrollrequests` and
`endpointmigrationwaves`.
- **The agent pins the server it talks to, and the registry owns the pins.**
An air-gapped site runs its own certificate, so "trusted chain" says very
little; the agent now checks the server's SubjectPublicKeyInfo SHA-256
against a pinned list before it will send a credential anywhere, and a
credential is only ever sent back to the server that issued it. Pins live in
`HKLM\SOFTWARE\GE\ShopDB` under `ServerPins`, with `appsettings.json` as the
fallback. The registry is AUTHORITATIVE whenever that value exists, even
when it is empty: the two used to be merged, so removing a pin from HKLM
never took effect anywhere. Two pins are carried at once, current and next,
so a certificate can be rotated without a flag day.
`Set-ShopDBServerPins.ps1` is the GE-Enforce entry point (`-Clear` deletes
the value, `-None` writes an empty one) and `Get-ShopDBServerPin.ps1` reads
what a PC has. `GET /settings/server-cert` returns the pin to put in that
entry, the Server Certificate page shows it, and the page also compares the
certificate the site actually presents at its public address with the
certificate file, warning when the two disagree.
- **An agent on a one-off PC from a single command.** A PC outside GE-Enforce -
a one-off, a loaner, something rebuilt in a hurry - had no path to an agent
short of doing the whole managed install by hand. An administrator now
generates a short-lived token in Settings > Endpoints > Endpoints and hands a
technician one elevated PowerShell line. The script refuses to run
unelevated, refuses a PC that already has an agent (a managed bay gets its
agent from GE-Enforce), checks both binaries against hashes the server baked
into the package, and installs with NO shared key - the PC then asks to
enroll and the script watches locally for two minutes for an administrator to
approve it. The token routes answer 404 to anything wrong rather than
confirming a token exists, are rate limited per address and are audited with
the caller, and a bootstrap package can never carry a collector key.
Migration `0024_endpoint_bootstrap_tokens` adds the token table, which stores
a hash, an expiry and a use count.
- **A kiosk admits a caller by where it is on the network, not only by its
token.** The device token assumes a handful of fixed screens provisioned
once, and that is not how these kiosks are used: most people open the page on
their own PC, there is no login, and per-browser provisioning would mean
touching every PC on the floor and losing it again at the next reimage.
Both the USB and printed-parts kiosks now admit a caller on either ground -
the per-device token, or a source address inside an allowlist - the same
shape the GE-Enforce client endpoints already use. `shopdb.api` gained
`ip_in_cidrs` and `request_ip_allowlisted` (contract 0.35.0) so the two
plugins share one gate rather than two copies of a CIDR parser; they fail
closed on an empty setting, an unparseable caller address and a malformed
entry, and they read `remote_addr`, so the IIS rewrite rule is what makes
them meaningful. The allowlists are separate per kiosk
(`usb_kiosk_allowed_cidrs`, `printedparts_kiosk_allowed_cidrs`) for the same
reason the tokens are, and both SHIP EMPTY: nothing opens by itself, and a
site that enters no ranges keeps exactly today's token-only behaviour. Both
settings pages get a picker listing the network plugin's subnets with their
VLANs, storing resolved CIDR strings rather than subnet ids so the setting
still works at a site that does not run that plugin. Stated plainly in the
admin UI: anyone on an allowed network can check a drive in or out against a
badge number they guess, and badges are attribution, not authorization.
- **A kiosk says when it is not set up, and an administrator can copy the setup
link.** The settings API masks every key whose name contains `token`, so the
real kiosk token is on screen for exactly one moment - after Generate, before
Save - and a screen provisioned with the literal mask answers 404, which is
also how the route answers a caller it has never heard of. Each kiosk gained
a permission-gated provisioning-link route that returns the real setup URL to
somebody who already holds `settings.edit`, and both settings pages a "Copy
setup link" button that falls back to showing the link when the clipboard is
refused. Revealing a token writes an audit row. The USB kiosk now asks the
server on load whether this screen may be used and says plainly when it may
not, instead of rendering a keypad that only fails once somebody has typed
their SSO.
- **The USB audit says whether a returned drive came back wiped and scanned.**
Removable-media custody is CMMC evidence, and custody without the state of
the media on return is half the record. Both stores already held the values -
`sanitized` and `scanned_viruses` in the external `checkinoutlog`,
`waswiped` and `scannedviruses` in the self-hosted `usbcheckouts` - and
neither mode selected them. Both now report them in the JSON, the CSV and the
column list. Only a return carries them, so a check-out row shows blanks
rather than asserting something about an event that had not happened yet, and
unknown stays blank rather than rendering as "No": both stores keep these
nullable deliberately, and "nobody wrote it down" is a different claim from
"it came back unwiped".
### Changed
- **Endpoint agent 1.20.1 and tray 1.7.1. Ship them together.** Every HTTP
client in the agent, script and package downloads included, is now built by
one `AgentHttpFactory`, so there is a single place the pins and the
credential are applied rather than one per caller. `EnrollmentService`
enrolls once using whatever shared key the PC already has, stores the
credential DPAPI-protected in a SYSTEM-only directory and swaps to it without
a restart; an old server with no enroll route simply leaves the agent on the
shared key. A named pipe lets the interactive user ask the agent to send
status, send a report, list alerts and acknowledge one, so `--once` and
`--report` run as a user become pipe clients, and `appsettings.json` is
restricted to SYSTEM and Administrators once the pipe is up. The tray holds
no key and reads no agent configuration at all - everything it does goes
through that pipe, an acknowledgement is remembered only once the agent
confirms it, and when the agent is not running the menu says so instead of
failing quietly.
The version numbers moved from 1.20.0 and 1.7.0 with no behaviour change of
their own: those binaries were built and shipped to the display scope before
two fixes landed, so a rebuild carried different bytes under the same version
number. Detection is by hash and copes with that, but nobody reading a
version number can tell two 1.20.0.0 files apart, and the runbook asks for one
build across the share, the agent store and the display profile.
- **A PC that has enrolled is refused the shared fleet key.** Once a PC holds
its own credential, presenting the fleet key is either a misconfiguration or
somebody else using a leaked secret, and either way the right answer is not
to accept it: that call is refused with `SHARED_KEY_REFUSED`. This only ever
applies to a PC that has already enrolled. `plugin.endpoints.sharedkeyallowed`
still DEFAULTS TO ON, so every un-migrated bay goes on reporting exactly as it
does today; a site turns it off deliberately once its fleet has moved, and the
page asks for confirmation and says how many PCs are not enrolled yet.
`plugin.endpoints.autoenroll` also defaults to on, so a clean hostname and
serial match is issued a credential without an administrator waiting on it.
- **The shared-key page now reads as the migration state it is.** Settings >
Endpoints presents the key as what PCs use UNTIL they enroll, with enrollment
progress, a link to Enrollment, and rotate and revoke confirmations that say
how many PCs would lose their only credential. The settings cards describe
the server certificate as what PCs actually receive, on IIS or Docker.
`AGENT-CONTRACT.md` documents per-PC credentials, and the rollout runbook
covers the phases, the gates, rollback, the pin and wave entries, and every
place the shared key lives and how to clear it at cutover.
### Removed
- **Client certificate authentication, the 2026-09-14 attempt.** `client_cert.py`,
certificate renewal, expiry monitoring, the unauthenticated
`/test/client-cert` route, the nginx client-certificate forwarding and the
agent certificate script are all gone. The header carrying the client
certificate could be forged on IIS, the private key was readable by users,
and the shared key remained a fallback underneath it all, so it bought
nothing the per-PC credential above does not buy properly. Migration 0023
drops the certificate columns from `assets` where a test database has them;
0021 and 0022 keep their ids but no longer touch the core table.
### Security
- **The tray no longer holds a secret.** It ran as the interactive user and
carried enough configuration to talk to the server on its own. Everything it
needs now comes over the agent's pipe, which means a standard user on a bay
can read nothing out of it.
- **`collect_log` refuses to collect the things that hold credentials.** Setup
answer files, `appsettings.json`, agent directories and key files are refused
at enqueue rather than shipped to the server inside a log bundle. The setup
token is no longer printed to the log either, and two VM notes that carried a
live collector key and advised plain http were deleted rather than corrected.
## [0.12.0] - 2026-08-20
Printers become a property of the bay rather than of whoever last walked up to a

50
CLAUDE.md
View File

@@ -1,12 +1,12 @@
# ShopDB Flask - Claude Code Context
ShopDB is a manufacturing floor asset management system built with Flask, Vue 3, and PostgreSQL, deployed across air-gapped GE Aerospace sites.
ShopDB is a manufacturing floor asset management system built with Flask, Vue 3, and MySQL, deployed across air-gapped GE Aerospace sites.
## Architecture
- **Backend**: Flask 3.x with SQLAlchemy 2.x, plugin-based architecture
- **Frontend**: Vue 3 with Vite, component-based UI with plugin panels
- **Database**: PostgreSQL 15+ with Alembic migrations per plugin
- **Database**: MySQL 8 with Alembic migrations per plugin (`docker-compose.yml` runs `mysql:8.4`; WJ prod is IIS in front of MySQL 8)
- **Deployment**: Docker Compose with gunicorn, air-gapped sites via bundle workflow
- **Plugins**: 20+ plugins (computers, endpoints, printers, machines, etc.)
@@ -27,10 +27,13 @@ Each plugin owns its migration chain in `plugins/{name}/migrations/versions/`.
- Migrations run automatically on deploy
### Database Naming Convention
- **Tables**: lowercase concatenated, no underscores (e.g., `printedpart`, `mappeddrives`)
- **Columns**: snake_case (e.g., `serial_number`, `last_seen`)
- **Indexes/constraints**: follow PostgreSQL conventions
- Enforced by `scripts/check-naming-and-style.sh` pre-commit hook
- **Tables**: lowercase concatenated, plural, no underscores (e.g., `machines`, `mappeddrives`)
- **Columns**: lowercase concatenated, singular, no underscores (e.g., `machineid`,
`lastzabbixsync`, `isactive`). NOT snake_case - see CONTRIBUTING.md
- **Indexes**: `idx_<table>_<column>`. Underscores are allowed HERE ONLY, because an
index is infrastructure and not data
- Enforced by `scripts/check-naming-and-style.sh` pre-commit hook, on model attributes
AND on the column names a migration creates
### Branch and Deployment Model
- `wj` -> prod (shop floor at WJ site)
@@ -54,10 +57,20 @@ Plugins extend ShopDB via:
- `frontend/views/` - Vue components auto-discovered by router
- `migrations/versions/` - Alembic migrations
Import from `shopdb.api` for stable surfaces:
Import from `shopdb.api` for stable surfaces. Every name has to be in that
module's `__all__`; this line used to read
`db, Asset, Employee, BasePlugin` and two of the four do not resolve, so the
first block a new agent copied raised ImportError:
```python
from shopdb.api import db, Asset, Employee, BasePlugin
from shopdb.api import db, Asset, User, Setting, require_permission
# BasePlugin is NOT on that surface. plugin.py takes it from its own module,
# which is what every plugins/*/plugin.py in this tree does:
from shopdb.plugins.base import BasePlugin, PluginMeta
```
There is no `Employee` model. The directory is an external read-only database
reached through `employee_connection`; the plugin's own row type is
`DirectoryEmployee`. See `.claude/QUICK-REFERENCE.md` for the full export list
and for the other names that are commonly assumed and are not there.
## Documentation
@@ -69,7 +82,7 @@ Full documentation in the GitHub wiki at `../shopdb-flask.wiki/`:
- ADR-006: Collector contract (endpoint agent)
- ADR-007: Product versioning and releases
- ADR-013: Plugin catalog and lean builds
- ADR-016: Credential delivery
- ADR-016: Credential delivery to the fleet (cmdkey credentials for bays; ACCEPTED, NOT built - it is not about kiosks or device tokens)
**References**:
- `API-REFERENCE.md` - Core API surface
@@ -116,7 +129,24 @@ Full documentation in the GitHub wiki at `../shopdb-flask.wiki/`:
- **Migration conflicts**: Keep plugin chains linear, coordinate with PLUGIN_TABLE_OWNERS
- **Unicode**: ASCII only - check logs, comments, commit messages
- **Import surface**: Use `shopdb.api`, not internal paths
- **Device tokens**: Kiosks require device_token authentication (ADR-016)
- **Kiosk access**: a kiosk route admits a caller on either of two grounds - the
per-kiosk device token, or a source address inside that kiosk's CIDR allowlist
(`usb_kiosk_allowed_cidrs`, `printedparts_kiosk_allowed_cidrs`) - and both are
gated on the kiosk's enable flag. Use `ip_in_cidrs` from `shopdb.api`; do not
hand-roll a CIDR parser. `request_ip_allowlisted` is only
`ip_in_cidrs(client_ip(request), cidrs)` - the USB kiosk does NOT use it,
because it first refuses an address that belongs to the path rather than to a
caller (loopback, or a configured trusted proxy). On the shipped Windows stack
the X-Forwarded-For rewrite rule is disabled, so every caller arrives as
127.0.0.1 and that refusal is what keeps the door shut. The real shape and the
full trace are in `.claude/QUICK-REFERENCE.md`. There is no
`requires_device_token` decorator
- **Fleet credentials are in the registry, deliberately**: the endpoint agent and the
GE-Enforce reporters run as SYSTEM with no user to prompt, and read their collector
key, base URL and server pins from `HKLM\SOFTWARE\GE\ShopDB`. An enrolled PC's
own per-PC credential is DPAPI-protected in a SYSTEM-only directory instead.
Nothing forbids the registry hive - ADR-016 names registry provisioning as a
recommended first step
## Work PC Role

77
deploy/windows/installer/shopdb-install.ps1
View File

@@ -124,8 +124,11 @@ param(
# the proxy, so applying it would DESTROY the real client IP.
#
# Without one or the other, IIS sends no X-Forwarded-For at all and every
# client reads as 127.0.0.1: the GE-Enforce IP allowlist, the dashboard
# visitor-location lookup and per-host login rate limiting all break quietly.
# client reads as 127.0.0.1: the GE-Enforce IP allowlist, BOTH logged-out
# kiosk allowlists (usb_kiosk_allowed_cidrs, printedparts_kiosk_allowed_cidrs
# - these admit writes with no login), the dashboard visitor-location lookup
# and per-host login rate limiting all break quietly. Stage 4 verifies this
# in both modes; see the client-IP check there.
[ValidateSet('direct','proxy')] [string] $ClientIpSource = 'direct',
# Load the shared vendor/model catalog during stage 3. Offered by the
# wizard rather than assumed: a site that machines nothing does not want
@@ -2500,27 +2503,71 @@ public class ShopdbSmokeTestCertPolicy : ICertificatePolicy {
}
}
# Is the client-IP rule actually in effect? It is the control that makes
# remote_addr trustworthy: without it IIS forwards whatever X-Forwarded-For
# the CALLER sent, waitress trusts it because it arrives from loopback, and
# the GE-Enforce IP allowlist becomes bypassable from anywhere on the network.
# It does not fail closed. Verified here rather than assumed, because the
# failure is silent and looks exactly like a working server.
# Is the client-IP control actually in effect? Something has to make the
# address the app sees the CALLER's: waitress is launched with
# --trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for, and
# HttpPlatformHandler always connects from loopback, so waitress ALWAYS
# believes the X-Forwarded-For it is handed. Whether that header is worth
# believing is decided in front of it, and nothing fails closed if it is not.
#
# THREE allowlists now ride on this, not one: the GE-Enforce client
# allowlist and the two kiosk allowlists (usb_kiosk_allowed_cidrs,
# printedparts_kiosk_allowed_cidrs), which admit UNAUTHENTICATED writes by
# network position. Naming only GE-Enforce here sent people to the wrong
# place - and left the kiosks out of the decision entirely.
#
# Checked in BOTH modes. It used to run only under -ClientIpSource direct,
# so the proxy deployment - the one where the header really does arrive from
# outside - had nothing verified at all.
$liveCfg = Join-Path $AppRoot 'web.config'
$cfgText = if (Test-Path $liveCfg) { Get-Content $liveCfg -Raw } else { '' }
$ruleLive = ($cfgText -match '<rewrite>') -and ($cfgText -notmatch 'SHOPDB-CLIENTIP-BEGIN')
$moduleThere = Test-Path (Join-Path $env:windir 'system32\inetsrv\rewrite.dll')
if ($ClientIpSource -eq 'direct') {
$liveCfg = Join-Path $AppRoot 'web.config'
$cfgText = if (Test-Path $liveCfg) { Get-Content $liveCfg -Raw } else { '' }
$ruleLive = ($cfgText -match '<rewrite>') -and ($cfgText -notmatch 'SHOPDB-CLIENTIP-BEGIN')
$moduleThere = Test-Path (Join-Path $env:windir 'system32\inetsrv\rewrite.dll')
if ($ruleLive -and $moduleThere) {
Write-Log 'client-IP rule is live; remote_addr is trustworthy' 'OK'
} else {
Write-Log 'CLIENT-IP RULE IS NOT IN EFFECT' 'WARN'
if (-not $moduleThere) { Write-Log ' URL Rewrite module is not installed' 'WARN' }
if (-not $ruleLive) { Write-Log ' web.config does not have an active <rewrite> block' 'WARN' }
Write-Log ' every client will be recorded as 127.0.0.1, and the GE-Enforce IP' 'WARN'
Write-Log ' allowlist is BYPASSABLE - a caller can send its own X-Forwarded-For' 'WARN'
Write-Log ' re-run with -ClientIpSource direct, or use token authentication' 'WARN'
Write-Log ' a caller can send its own X-Forwarded-For and be believed, so EVERY' 'WARN'
Write-Log ' address-based control is BYPASSABLE from anywhere on the network:' 'WARN'
Write-Log ' - the GE-Enforce client IP allowlist' 'WARN'
Write-Log ' - usb_kiosk_allowed_cidrs (logged-out USB kiosk writes)' 'WARN'
Write-Log ' - printedparts_kiosk_allowed_cidrs (logged-out parts kiosk writes)' 'WARN'
Write-Log ' and with no header at all every client is recorded as 127.0.0.1.' 'WARN'
Write-Log ' the USB kiosk refuses a loopback caller outright rather than' 'WARN'
Write-Log ' admitting one, so that case reads as "not set up", not as access.' 'WARN'
Write-Log ' install URL Rewrite and enable the SHOPDB-CLIENTIP block, or leave the' 'WARN'
Write-Log ' two kiosk allowlists EMPTY and provision each screen with its token' 'WARN'
}
} else {
# Proxy mode. The rule must be OFF here - it would overwrite the real
# client with the proxy's address - and the thing that has to be true
# instead is about the proxy, which this script cannot reach out and
# test. So: check what can be checked, and state the rest plainly.
if ($ruleLive) {
Write-Log 'web.config OVERWRITES X-Forwarded-For but -ClientIpSource is proxy' 'WARN'
Write-Log ' every request will be attributed to the proxy, not the caller:' 'WARN'
Write-Log ' the GE-Enforce and kiosk allowlists then match nothing at all' 'WARN'
Write-Log ' remove the <rewrite> block from web.config' 'WARN'
} else {
Write-Log 'client IP comes from the proxy; no rewrite rule in web.config' 'OK'
}
Write-Log 'PROXY MODE: two things this installer cannot verify for you' 'WARN'
Write-Log ' 1. the proxy must APPEND the caller to X-Forwarded-For (ARR does).' 'WARN'
Write-Log ' waitress takes the LAST hop and puts it in REMOTE_ADDR, so a proxy' 'WARN'
Write-Log ' that passes a caller-sent header through unchanged makes every' 'WARN'
Write-Log ' address control bypassable from anywhere that can reach the proxy.' 'WARN'
Write-Log ' 2. set usb_kiosk_trusted_proxies (Settings > USB) to where the proxy' 'WARN'
Write-Log ' is, BEFORE filling in usb_kiosk_allowed_cidrs. The app does not' 'WARN'
Write-Log ' read forwarding headers itself - waitress already promoted the' 'WARN'
Write-Log ' caller into REMOTE_ADDR, so the allowlist matches the CALLER, not' 'WARN'
Write-Log ' the proxy. What it is for is the opposite case: a request that' 'WARN'
Write-Log ' arrives wearing the PROXY address, because the proxy forwarded' 'WARN'
Write-Log ' nothing. A proxy normally sits on the same plant network the' 'WARN'
Write-Log ' allowlist names, so that request would otherwise be admitted on no' 'WARN'
Write-Log ' evidence at all. Naming the proxy here subtracts it and fails closed.' 'WARN'
}
# The self-hosted API documentation. It is the thing an operator - or the

4787
docs/api-inventory.json Normal file
View File

File diff suppressed because it is too large Load Diff

42074
docs/openapi.json
View File

File diff suppressed because it is too large Load Diff

42
docs/wiki-drafts/ADR-019-Plugin-Dynamic-Settings-Routes.md
View File

@@ -2,7 +2,12 @@
**Status:** Accepted (2026-09-11)
**Context:** Plugin UI extensions, lean builds
**Supersedes:** Hardcoded plugin routes in core router
**Supersedes:** Hardcoded plugin routes in core router
**Last checked against code:** 2026-09-18, branch gh-wj at 35d4fcd41
This draft lives only in `docs/wiki-drafts/`. There is no `docs/adr/` directory
on this branch, so this is the sole ADR-shaped record here and nothing enforces
it except the tests named below. See ADR-Conformance-2026-09-18.md.
## Problem
@@ -63,27 +68,44 @@ class ComputersPlugin(BasePlugin):
### Backend: API Endpoint
Added to `shopdb/core/api/plugins.py`:
In `shopdb/core/api/plugins.py:46-95`, as it stands on gh-wj at 35d4fcd41:
```python
@plugins_bp.route('/settings-routes', methods=['GET'])
@jwt_required(optional=True)
@require_permission('settings.view')
def get_plugin_settings_routes():
"""Return settings routes contributed by all enabled plugins."""
pm = current_app.extensions.get('plugin_manager')
if not pm:
return success_response({'routes': []})
routes = []
for plugin in pm.get_enabled_plugin_instances():
if hasattr(plugin, 'get_settings_routes'):
for plugin_name, plugin in pm.get_all_plugins().items():
# Only include routes from enabled plugins
if not pm.registry.is_enabled(plugin_name):
continue
if not hasattr(plugin, 'get_settings_routes'):
continue
# One broken plugin must not take every settings hub down with it.
try:
plugin_routes = plugin.get_settings_routes()
if plugin_routes:
routes.extend(plugin_routes)
except Exception:
current_app.logger.exception(
'get_settings_routes failed for plugin %s; skipping it',
plugin_name)
continue
routes.extend(renderable_settings_routes(plugin_name, plugin_routes,
current_app.logger))
return success_response({'routes': routes})
```
Earlier revisions of this draft showed a `@jwt_required(optional=True)` handler
iterating `pm.get_enabled_plugin_instances()`. Neither is in the code:
`grep -rn get_enabled_plugin_instances shopdb/ plugins/` returns nothing, so
that block would not have run. It is replaced above rather than kept, because a
copy-paste block that reopens the anonymous read is worse than no block.
**Auth:** the endpoint is gated on `settings.view`. It was written anonymous, on the reasoning that unauthenticated pages (kiosks, TV displays) register plugin tabs before login. Nothing does: its only caller is `ModelCatalogHub.vue`, which sits behind `requiresAuth` and `requiresAdmin`. Open, it handed an anonymous caller the enabled plugin list, the settings hub layout and the internal Vue component paths, so it was closed.
### Frontend: Dynamic Route Registration
@@ -323,7 +345,7 @@ const pluginComponents = import.meta.glob('../.plugins-staged/*/views/*.vue')
**Nested routes:** A plugin could return routes with `children: [...]` for nested settings pages. Router supports this via `addRoute(parentName, childRoute)`.
**Permissions:** The `tab` could include a `permission` field (e.g., `"computers.manage_os"`), and the frontend filters out tabs the user can't access. The backend endpoint is anonymous, so permission checks would be client-side only (acceptable for UI gating; the API endpoint enforces permission anyway).
**Permissions:** The `tab` could include a `permission` field (e.g., `"computers.manage_os"`), and the frontend would filter out tabs the user cannot reach. This would be UI gating only, and the page behind the tab must still enforce its own permission; the `settings.view` gate on `/api/plugins/settings-routes` says who may read the tab list, not who may use a given tab.
**Icons:** Add an `icon` field to the route metadata (e.g., `"mdi-microsoft-windows"`), and the tab renders the icon next to the label.

556
docs/wiki-drafts/ADR-Conformance-2026-09-18.md Normal file
View File

@@ -0,0 +1,556 @@
# ADR Conformance Review - September 18, 2026
**Branch:** gh-wj at 35d4fcd41
**Range reviewed:** 2ac2e6d33..gh-wj, 20 commits, 149 files, +22055 lines
**Method:** read-only. Every statement below cites a file and line that was read.
No test suite was run except `tests/test_plugins/test_kiosk_network_allowlist.py`
(20 passed) and `bash scripts/check-naming-and-style.sh` (exit 0).
## Read this first: the ADRs are not in this repository
`ls docs/adr` fails on gh-wj. There is no ADR directory on this branch. The
whole set, ADR-001 through ADR-018 plus a README, lives on the other lineage at
`docs/adr/ in the product repo` and was read there for this review.
`docs/` on gh-wj holds only `ShopDB-Planner-Tasks-v2.csv`, `deployment/`,
`notes/`, `openapi.json` and `wiki-drafts/`.
Three consequences, and they are the reason this page exists:
1. **Nothing on this branch states the rules.** A person working here has no
ADR to read, so a decision can be contradicted without anyone noticing that
a decision existed. That is exactly what happened to ADR-016 (see below).
2. **Nothing on this branch enforces them except one script.**
`scripts/check-naming-and-style.sh` is the only ADR-aware gate in the repo,
and it names two ADRs: ADR-017 at line 343 and ADR-015 at line 395. Its other
eight checks enforce CONTRIBUTING.md house rules, not ADRs. A handful of
tests cover ADR ground without being described that way:
`tests/test_plugin_migrations.py` (ADR-008 chain shape and ownership),
`tests/test_core/test_plugin_settings_hubs.py` (ADR-019),
`tests/test_plugin_contract.py` (manifest fields),
`tests/test_core/test_public_read_surface.py` (the read gate).
`tests/test_api_inventory_parity.py` skips itself entirely on this branch:
`docs/api-inventory.json` is not tracked here, and its own skip reason says
"docs/ is excluded from publication".
3. **Some violations cannot be fixed from here.** ADR-018 has no
implementation-status note; ADR-016 is still marked "decided, NOT yet
implemented". Both corrections belong in files that are not in this
repository.
The one ADR-shaped file on gh-wj is the ADR-019 draft in this folder, and it is
a draft.
## Result by ADR
| ADR | Subject | Result |
|-----|---------|--------|
| ADR-001 | Asset as platform contract | Two exceptions, both below: the unbumped Asset cert columns (2) and raw SQL over core tables (6) |
| ADR-002 | Plugin versioning | Holds for the bumps. Two stale manifests, and one unbumped core-model change (2) |
| ADR-003 | Plugin distribution | Holds |
| ADR-004 | Deployment topology | Holds |
| ADR-005 | Equipment vs measuringtools | Superseded by ADR-018 |
| ADR-006 | Collector contract | Holds on shape. One gap in a new boundary |
| ADR-007 | Product versioning and releases | VIOLATED (CHANGELOG) |
| ADR-008 | Plugin migration ownership | VIOLATED (plugin chain, core table) |
| ADR-009 | Frontend plugin gating | Holds |
| ADR-010 | Frontend plugin hooks | Holds |
| ADR-011 | Machines rename | Holds |
| ADR-012 | GE-Enforce manifest ownership | Holds |
| ADR-013 | Plugin catalog and lean builds | Holds |
| ADR-014 | Schema-lean per site | Holds |
| ADR-015 | Site-specific configuration | Holds in scope. Scope has a hole |
| ADR-016 | Credential delivery | Unbuilt by its own status, and MISDESCRIBED in four files here |
| ADR-017 | Buildings and levels | Holds |
| ADR-018 | Metrology instruments are machines | Accepted, not implemented |
| ADR-019 | Plugin dynamic settings routes (draft) | Implementation matches the draft |
## What holds, and the evidence
**ADR-001, the import surface.**
`grep -rn "from shopdb\.\(core\|utils\|extensions\|services\)" plugins/
--include=*.py` returns zero hits. The only `shopdb.*` import in `plugins/`
outside `shopdb.api` is `shopdb.plugins.alembic_template` from migration
`env.py` files, which is the shared Alembic runner ADR-008 sanctions. The three
names added to the contract this range - `BusinessUnit`, `ip_in_cidrs`,
`request_ip_allowlisted` - are both defined and listed in `__all__`
(`shopdb/api/__init__.py:190-194`, `:471-486`). The new endpoints tables key off
`assets.assetid` by foreign key rather than introducing a parallel identity.
This covers the names added and kept. It does NOT cover a name added to the
Asset model and then withdrawn again without a bump, which happened in this
range; see finding 2.
**ADR-002, the bumps.** `__contract_version__` moved 0.32.0 to 0.35.0 with a
per-bump rationale naming what was added and why it is additive
(`shopdb/__init__.py:124-152`). Every name those comments claim is genuinely
exported. Every new `BasePlugin` hook is optional with a safe default
(`shopdb/plugins/base.py:313-332`), so minor is the right classification. The
loader still checks `core_version` against `__contract_version__`
(`shopdb/plugins/loader.py:255-258`).
**ADR-004 and ADR-014, per-site instances.** No `siteid` column anywhere in the
new schema. utf8mb4 still pinned in `.env.example`. The two-command deploy
(`flask db upgrade` then `flask plugin upgrade-all`) is unchanged. All four new
endpoints tables are registered in `PLUGIN_TABLE_OWNERS`
(`shopdb/plugins/alembic_template.py:57-63`) in the same change that creates
them, so `flask plugin prune-schema` still leaves a lean site lean.
**ADR-006, the collector shape.** `/api/collector/<pluginname>` is still the
single dynamic dispatch route (`shopdb/core/api/collector.py:229`), the response
envelope is byte-for-byte the ADR's `{status, action, assetid, identityvalue,
warnings}` (`:324-331`), every submission still writes an AuditLog row
(`:313-320`), `/api/collector/_schemas` is still JWT-protected (`:337-341`), and
per-plugin `COLLECTOR_API_KEY_<PLUGIN>` with shared fallback resolves as the ADR
specifies (`:194-203`). Identity integrity under the new credential mode is
enforced in all three schema-declaring plugins - a per-PC credential reports
only for its own asset (`plugins/computers/plugin.py:729-733`,
`plugins/printers/plugin.py:556-559`, `plugins/endpoints/plugin.py:1422-1427`) -
and the dispatcher maps the `PermissionError` to 403 ASSET_MISMATCH
(`collector.py:294-299`).
**ADR-007, the version lock-step.** `__version__ = '0.12.0'`
(`shopdb/__init__.py:158`) matches `frontend/package.json:3`, and `__version__`
is still absent from `shopdb.api`.
**ADR-008 in the forward direction.**
`git diff --stat 2ac2e6d33..gh-wj -- migrations/` is empty. Not one line of
plugin schema went into the core chain. The endpoints chain is 24 revisions with
exactly one root and one head, no dangling `down_revision`, longest revision id
29 characters against the 32-character `alembic_version` cap.
**ADR-009, route gating.** Every plugin-owned route added or touched this range
carries `meta.plugin`, including the two new endpoints settings pages, the USB
kiosk and the printed-parts kiosk. `GET /api/plugins/enabled` stayed
`jwt_required(optional=True)` (`shopdb/core/api/plugins.py:31-42`) through the
read-permission sweep that gated 165 other GETs, so the unauthenticated kiosk
gate still works.
**ADR-010 and ADR-013, frontend boundaries.** The new
`frontend/src/components/NetworkAllowlistPicker.vue` is a CORE component
consumed by plugin views through the `@/` alias
(`plugins/usb/frontend/views/USBSettings.vue:108`,
`plugins/printedparts/frontend/views/PrintedPartsSettings.vue:164`), which is
the established pattern; 97 plugin frontend files already import that way. No
core file gained a hardcoded plugin import.
**ADR-012, GE-Enforce ownership.** The per-PC rollout keeps ShopDB as author of
record: the runbook's JSON snippets are entries to author and publish through
the plugin, not hand edits to the share, and `Set-ShopDBServerPins.ps1` is
staged as a normal PS1 entry with `TargetHostnames`/`TargetMachineNumbers`
gates. The engine is still referenced, not vendored.
**ADR-015 within its enforced scope.** `scripts/check-naming-and-style.sh` exits
0 and the pipeline is not vacuous: a control pattern over the same six
directories returns 1995 lines. The new kiosk network admission is a setting
with a blank default and a documented-neutral example
(`plugins/usb/plugin.py:153-168`), blank means off
(`shopdb/utils/ipallowlist.py:44-46`), and `frontend/vite.config.js:5-16`
replaced a hardcoded dev-proxy address with `VITE_DEV_API_TARGET`.
**ADR-017.** The per-occurrence `mapx`/`levelid` gate and the
`LocationMapTooltip :levelid` gate both pass. No payload in this range emits a
position without its level.
**ADR-019 against the implementation.** `SETTINGS_HUBS_WITH_PLUGIN_TABS =
('model-catalog',)` at `shopdb/plugins/base.py:29`; the loud drop at `:32-54`;
the `settings.view` gate at `shopdb/core/api/plugins.py:47`; the still
unrewritable `` import(`../../${tab.component}`) `` at
`frontend/src/views/settings/ModelCatalogHub.vue:84` with no `vite.config.js`
change to fix it; and a non-vacuous test suite in
`tests/test_core/test_plugin_settings_hubs.py`. The draft in this folder was
corrected on 2026-09-18: its code block still showed
`@jwt_required(optional=True)` and its Future Extensions section still called
the endpoint anonymous, both contradicting its own Auth paragraph and the
shipped code.
## Violations
### 1. ADR-007: no release record for the largest change in the range
`CHANGELOG.md:11`
`git diff --stat 2ac2e6d33..gh-wj -- CHANGELOG.md` returns nothing at all. The
file is byte-identical across all 20 commits, which include per-PC agent
credentials, enrollment, bootstrap tokens, server pinning, kiosk network
admission and three contract bumps. Grepping the whole file for "per-PC",
"X-Endpoint-Credential", "Endpoint credential" and "enrol" finds nothing
relevant. ADR-007 decision point 3 makes `CHANGELOG.md` the canonical
human-readable record and says work in flight accumulates under
`## [Unreleased]`; that block still describes only the pre-range share-link and
read-gate work.
Low severity: no release has been cut in the range, so the obligation point
(ADR-007 release procedure step 3) has not been reached and the debt is fully
repairable before the tag. Operator-facing content does exist, at
`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`, but that is plugin-local
and not where ADR-007 says an adopter looks.
### 2. ADR-008: a plugin migration chain does DDL on the core `assets` table
`plugins/endpoints/migrations/versions/0023_endpoint_credentials.py:73` and `:76`
0023 declares `plugin_name = 'endpoints'` (line 33) and then runs
`op.drop_index(index['name'], table_name='assets')` and
`op.drop_column('assets', column)`. It is cleaning up after itself:
`git show 5cd533239:plugins/endpoints/migrations/versions/0021_client_certificates.py`
shows the shipped 0021 adding three columns to `assets` plus
`idx_assets_cert_fingerprint` from the same plugin chain. `assets` is not, and
cannot be, in `PLUGIN_TABLE_OWNERS['endpoints']`
(`shopdb/plugins/alembic_template.py:57-63`), so the metadata ownership filter
never saw it, and `tests/test_plugin_migrations.py` has no test over which
tables a chain touches.
Low severity: every step is guarded on the live schema (0023:64-76), 0021 and
0022 are now no-ops, the forward direction is clean, and
`tests/test_plugins/test_endpoints_credentials_migration.py:96-140` runs the
chain twice from both a legacy and a fresh schema. The lane crossing is real;
the fleet blast radius is essentially zero.
**The ADR-001/ADR-002 half of this was written up wrong, and that conclusion is
withdrawn here (2026-09-18).** The original text read: "`git show 5cd533239
--stat` touched no file under `shopdb/`, and `grep -rn cert
shopdb/core/models/asset.py` is empty: the Asset *model* never carried those
columns. Only the database did. No contract bump was owed." Both halves of the
evidence are real and the conclusion drawn from them is still false, because
5cd533239 is not the only commit in the range and `grep` only sees the tree as
it stands now. `git show 18e5fdc00:shopdb/core/models/asset.py` puts
`client_cert_fingerprint` (declared `index=True`), `cert_issued_at`,
`cert_expires_at`, `cert_revoked_at` and `cert_revocation_reason` on the CORE
`Asset` model. 18e5fdc00 (2026-09-14, "Fix critical security vulnerabilities in
client certificate authentication") is an ancestor of `gh-wj`; 7480e4fe3
(2026-09-15) deleted all 28 lines again. The grep comes back empty because it
runs after the retraction, not because the contract was never touched.
The ADR-001/ADR-002 finding therefore holds, in a worse form than first stated:
the core Asset contract surface was extended to carry a plugin's feature and
then retracted, with no contract bump in either direction, and for the length of
that window five underscore-spelled columns sat on a core model in a repo whose
column convention forbids underscores. Severity stays low - the window is inside
an unreleased branch, and nothing outside the endpoints plugin ever read the
columns - but the finding is a real ADR-001/ADR-002 breach and not, as written,
a non-finding.
**Now guarded, 2026-09-18.** The finding above ended with "no test over which
tables a chain touches". There is one:
`tests/test_plugin_migrations.py::test_no_plugin_migration_touches_another_lanes_table`.
It AST-scans every `plugins/*/migrations/versions/*.py` for `create_table`,
`drop_table`, `add_column`, `drop_column`, `alter_column`, `create_index`,
`drop_index`, `rename_table`, the five constraint ops (`create_foreign_key`,
`drop_constraint`, `create_unique_constraint`, `create_check_constraint`,
`create_primary_key`) and the `batch_alter_table` forms, resolves each target
table through module-level constants and literal `for` sequences, and fails when
the table belongs to `core` or to a different plugin. Ownership comes from
`PLUGIN_TABLE_OWNERS` and a static scan of `shopdb/core/models/` (including the
three bare `db.Table` association tables), so the map stays the single source of
truth rather than a table list copied into a test.
Raw SQL is scanned too, because `op.execute('ALTER TABLE assets DROP COLUMN
assetnumber')` is DDL that no `op.*` scan can see. `op.execute` and
`exec_driver_sql` have their statement text flattened - f-string holes and
unresolvable names become a sentinel character rather than being dropped - and
`ALTER TABLE x`, `DROP TABLE x`, `CREATE TABLE x`, `RENAME TABLE x TO y`,
`CREATE/DROP INDEX ... ON x` and `TRUNCATE x` name the op's target table.
`UPDATE`, `DELETE`, `INSERT` and `SELECT` are data, not schema, and are not
flagged: three shipped revisions already run cross-checked DML
(`plugins/backups/.../0002_backups_lastseenat.py:37`,
`plugins/backups/.../0003_backups_clear_backfilled_lastseen.py:58`,
`plugins/computers/.../0006_policyactionhold_indefinite.py:78`) and all three
stay green. Keeping the hole in place is what separates the two cases that
matter: `machines 0002` interpolates a CONSTRAINT name into
`f"ALTER TABLE equipment DROP FOREIGN KEY {name}"`, where the table is literal
text and resolves; `f'ALTER TABLE {t} DROP COLUMN x'` puts the hole inside the
table name, resolves to nothing, and fails as UNRESOLVED, which is the obvious
dodge closed.
Four supporting tests keep it honest: an op whose table name cannot be resolved
statically is a failure, not a pass, so a computed name does not slip through; a
non-vacuity test asserts the scan still finds more than 100 ops and still
detects the 0023 crossing; a second non-vacuity test pins two specific raw-SQL
ops out of `machines 0002` and asserts the three DML statements are absent; and
a dead-entry test fails if an allowlist line stops matching anything, so an
excuse cannot outlive what it excused.
The allowlist keys on `(plugin, filename, op, table, column)`, not on the first
four. Keyed on the first four alone, the one shipped `drop_column on assets`
entry would also excuse a brand new `op.drop_column('assets', 'assetnumber')`
appended to that same shipped file - the exact breach the guard exists to stop -
so the six entries name their columns: one `drop_index` (column `None`, the
index name is computed) and one `drop_column` per certificate column. 0021 and
0022 are not listed, because they are stamp-only no-ops and present no DDL to
scan.
Verified non-vacuous against the pre-fix version of the same test module. Three
violations were planted in a scratch copy of `plugins/` (the shipped 0023 is
applied on production and was not edited in place): the five constraint ops on
`assets` from a plugin chain, `op.execute` with both a literal `ALTER TABLE
assets` and an f-string table name, and a new `op.drop_column('assets',
'assetnumber')` inside 0023. The pre-fix guard passed all four of its tests on
all three plants. The current guard fails, naming all seven ops with file, line,
op, table, column and owner, and goes green again once the plants are removed.
**The 0023 index-drop loop was examined and needs no repair.** 0023:71-73 drops
any index on `assets` whose column set intersects the certificate columns and
recreates nothing, which would matter if such an index could carry columns that
still exist. It cannot. The complete set of things that create an index on
`assets` is:
* the core baseline `68b3947ae14f`: `idx_asset_active`, `idx_asset_location`,
`idx_asset_status`, `idx_asset_type_bu`, `ix_assets_assetnumber`,
`ix_assets_serialnumber`
* core `7b02`: `ix_assets_gaugelabreference`, `ix_assets_maintenancereference`
* core `7d33`: `idx_assets_levelid`
* the shipped endpoints `0021`: `idx_assets_cert_fingerprint` over
`client_cert_fingerprint` alone (the shipped `0022` added two columns and no
index)
* `db.create_all` off `shopdb/core/models/asset.py`. The earlier version of this
inventory said that builds "that same set" as the core baseline, on the
strength of the Asset model never carrying a cert column. As corrected above,
between 18e5fdc00 and 7480e4fe3 it did, with `client_cert_fingerprint`
declared `index=True`, so `create_all` in that window ALSO built
`ix_assets_client_cert_fingerprint`. That index is missing from every earlier
telling and is the one real correction to this list.
Re-checked rather than assumed, the conclusion is unchanged.
`ix_assets_client_cert_fingerprint` is single-column over
`client_cert_fingerprint`, which `_drop_certificate_columns` removes in the
statement immediately after the loop, so the index carries no column that
outlives it and there is nothing to recreate. The filter is also unreachable by
accident: every name in `_CERT_COLUMNS` is underscore-spelled, which the column
convention forbids, so no real `assets` column can collide with it. The only two
indexes the loop can ever drop are `idx_assets_cert_fingerprint` and
`ix_assets_client_cert_fingerprint`, both dead by construction. No index was
lost on any database at the reference site and no repair migration is owed; the reasoning is
recorded in the test file beside the allowlist entries so it is not
re-litigated.
### 3. ADR-016: four files here describe a decision ADR-016 never took
`docs/wiki-drafts/README.md:316-323` (corrected 2026-09-18), `CLAUDE.md:119`,
`the assistant quick-reference file, lines 135 and 14`,
`docs/wiki-drafts/Security-Audit-2026-09-11.md:392`
All four attribute "device tokens" to ADR-016. `grep -rni device
docs/adr/ in the product repo*.md` returns no hit for device tokens
in ANY ADR. ADR-016's Decision is a fleet credential store with Fernet at rest,
a targeting and precedence resolver reusing `plugins/geenforce/filters.py`, an
explain endpoint, `credentials.manage`, and `GET /api/collector/credentials`
behind a dedicated `credentials.fetch` scope.
The `docs/wiki-drafts/README.md` copy went further and, under a heading
"ADR-016: Credential Delivery", listed registry keys among the places
credentials must **never** live. ADR-016 says the opposite: its
"Alternatives rejected" section calls registry-only provisioning "a worthwhile
FIRST STEP" and "Recommended as tier one regardless". The ban also contradicted
shipped code on this branch:
`plugins/computers/client/Report-AssetToShopDB.ps1:367-374` reads `CollectorKey`
out of `HKLM:\SOFTWARE\GE\ShopDB`, and
`plugins/endpoints/client/Set-ShopDBServerPins.ps1:4-20` writes the server trust
pins into the same key by design, with
`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:106-111` ("The registry
wins") making that the intended precedence. That section was rewritten on
2026-09-18; the three files outside `docs/wiki-drafts/` still carry the
misattribution and are out of scope for this edit.
Two qualifications, so nobody overcorrects: `PER-PC-CREDENTIALS-ROLLOUT.md:351-361`
deliberately plans to clear `CollectorKey` from both registry views at cutover,
so the shared key in HKLM is being retired anyway; and `ServerPins` are trust
anchors, not credentials.
### 4. ADR-016: the per-PC programme redefines "credential" with no design record
`plugins/endpoints/models/credential.py:1`
ADR-016's mechanism is unbuilt. Grepping `shopdb/` and `plugins/` for
`credentials.fetch`, `collector/credentials` and `Fernet` returns zero hits, and
`plugins/endpoints/plugin.py:154-265` declares only `endpoints.*` permissions -
no `credentials.manage`, no `credentials.fetch`. That matches ADR-016's own
"Implementation status, 2026-08-14". What the 20-commit programme built is a
different subject under the same word: `EndpointCredential` /
`EndpointEnrollRequest` / `EndpointMigrationWave` as the agent's own identity
secret (`models/credential.py:1-22`), `X-Endpoint-Credential` as the agent auth
header (`api/agentauth.py:1-18`), and an unauthenticated `/enroll`
(`api/enrollment.py:289`).
The design itself is sound and consistent with ADR-016's intent: the rollout
removes the cleartext key from the share, and rotation is a settings change plus
a pickup report, which is what ADR-016 wanted. The gap is that nobody wrote it
down as a decision. `grep -ril 'enroll|endpoint credential|X-Endpoint-Credential'`
over the ADR directory on the other lineage matches only ADR-012, which is
unrelated. Low severity and not fixable from here: the missing record belongs on
the lineage that carries `docs/adr/`.
### 5. ADR-018: accepted, unimplemented, with nothing saying so
`shopdb/plugins/alembic_template.py:74`
ADR-018 (ACCEPTED, 2026-09-04) decides that the `measuringtools` plugin is
retired and the `measuring_tool` asset type removed, that machines gains four
calibration columns and machinetypes gains `ismetrology`. None of it exists
here: `plugins/measuringtools/` still ships with its manifest,
`alembic_template.py:74` still registers
`'measuringtools': ('measuringtooltypes', 'measuringtools')`,
`shopdb/core/api/dashboard.py:19`, `:55` and `:93` still count `measuring_tool`
into the asset totals, `shopdb/cli/__init__.py:1021` still seeds it,
`deploy/site-profile-universal.json:12` still lists it in the universal profile,
and `grep -rn ismetrology shopdb/ plugins/` and
`grep -n calibration plugins/machines/models/*.py` both return nothing.
`git log -3 -- plugins/measuringtools/` shows no retirement work.
Unlike ADR-016, ADR-018 carries no implementation-status paragraph, so its text
reads as done. Low severity: the table holds zero rows, so nothing is broken
today, and ADR-018's own argument is that the window is time-boxed - the
migration is free while the 134 staged metrology instruments have not been
entered. gh-wj is the branch that deploys to the reference site, so entering them here is what
closes the window.
### 6. ADR-001 / ADR-002: raw SQL over core tables from a plugin
`plugins/computers/plugin.py:2080`
`db.session.execute(db.text(...))` joining core `assets`, `assetrelationships`
and `assettypes`, inside the `retype` CLI command. `Asset`, `AssetType`,
`AssetRelationship` and `RelationshipType` are all exported and in `__all__`
(`shopdb/api/__init__.py:47-48`, `:424`, `:438-439`), so unlike the
`BusinessUnit` case that the 0.34.0 bump note closed, there is no missing export
to blame, and `plugins/endpoints/api/routes.py:3981-3986` is the same pattern
already corrected. The 0.34.0 note names the reason in the code:
"a raw query is hidden coupling to the core schema that no contract bump can
catch, and it is the kind of thing that gets copied"
(`shopdb/__init__.py:135-142`).
Low severity and arguably generous: ADR-001's stated rule is an import rule,
which raw SQL does not literally breach, and the blast radius is one admin CLI
command with no serving route affected. Predates this range (331d27e07). The
other plugin raw query, `plugins/computers/plugin.py:2128`, touches only
plugin-owned tables and is fine.
### 7. ADR-015: the site-literal scan does not cover the C# agent
`scripts/check-naming-and-style.sh:440`
Lines 436-440 scope the grep to `plugins/ shopdb/ scripts/ deploy/
frontend/src/ tools/` with `--include` on
`.py/.vue/.js/.ps1/.psm1/.sh/.iss/.json/.html`. `agent/` is absent and `.cs` is
not an included extension. This range added roughly 2,500 lines of C# under
`agent/`, all of which ships to a sister site. Re-running the assembled site
patterns by hand over `agent/`, excluding bin and obj, finds one hit: prose in
`agent/ShopDbEndpointAgent/ShopDbEndpointAgent.csproj:108`, which ADR-015
explicitly permits ("Comments and documentation may name a site where it
explains WHY"). So the rule holds today; the gate would not notice if it stopped
holding.
**This is a coverage gap in the rule, not a breach of it.** ADR-015's 2026-08-14
amendment names those six directories as the scan's scope, so the script matches
the ADR precisely. The amendment widened the scan on exactly this reasoning -
the original scope missed every literal that reached a second site, because
those were in the client scripts and the installer - and the C# agent is now
that client channel.
### 8. ADR-002: two manifests declare a floor below the contract name they import
`plugins/usb/manifest.json:8` and `plugins/printedparts/manifest.json:8`
`da91a2915` added `shopdb/utils/ipallowlist.py`, bumped `__contract_version__`
to 0.35.0 with `ip_in_cidrs` and `request_ip_allowlisted` named as the new
surface (`shopdb/__init__.py:143-152`), and made
`plugins/usb/api/routes.py:42` and `plugins/printedparts/api/routes.py:521`
import `request_ip_allowlisted`. Neither manifest was touched: usb still says
`">=0.1.0,<1.0.0"` and printedparts `">=0.16.0,<1.0.0"`. The team does maintain
this field - `066a19052` raised `plugins/endpoints/manifest.json:7` to
`">=0.34.0,<1.0.0"` one commit earlier.
On a 0.34.0-or-older core both plugins would pass the loader's version gate
(`shopdb/plugins/loader.py:141-166`, called at `:258`) and then die with an
ImportError at blueprint registration. Impact at the reference site is nil, because core and
plugins always ship from this one tree.
The ADR-002 framing is weak and should not be quoted as a clean breach: ADR-002
itself says pre-1.0 plugins should declare `">=0.1.0,<1.0.0"`, which is
literally what `plugins/usb/manifest.json:8` declares, while its Neutral section
says bundled plugins' `core_version` is "always the current
`__contract_version__`". The ADR contradicts itself here. This is staleness
against team practice.
## Checked and not a violation
These were examined and did NOT hold up as ADR breaches. They are listed so the
next review does not re-open them.
- **`get_settings_routes` narrowed to a one-hub allowlist without a contract
bump.** `shopdb/core/api/plugins.py:92` now filters through
`renderable_settings_routes`. Nothing observable changed: the only frontend
caller, `frontend/src/views/settings/ModelCatalogHub.vue:73`, already filtered
to its own hub at `:76`, and `frontend/src/router/routes/core.js:62-63` states
in-tree that no other hub page fetches or registers plugin routes. A route
naming another hub was drawn nowhere before and is drawn nowhere now. No
`BasePlugin` ABC signature or `PluginMeta` schema changed, so under ADR-002
this is patch-level and no bump was owed. It is also not silent:
`shopdb/plugins/base.py:46-52` logs an error naming the plugin, the hub and
the supported set, and the same commit added the "Constraints as implemented"
section to the ADR-019 draft plus
`tests/test_core/test_plugin_settings_hubs.py`.
- **`computers` and `printers` manifests declaring `>=0.20.0` against the
contract-0.33.0 key `acceptsagentcredential`.** ADR-002 requires no such
tracking, nothing in `loader.py` implements it, and the claimed fail-open
cannot occur: on a core pinned at 0.32.x the endpoints plugin is refused at
load by its own `>=0.34.0` floor, so no credential is ever issued and there is
no enrolled PC whose shared key an older dispatcher could wrongly accept.
## Drift that no ADR covers
Recorded because it is real and someone will look for it here.
**`docs/openapi.json` is a release behind the code.**
`shopdb/core/api/docs.py:18` pins `_SPEC` to that file and serves it through a
vendored Redoc at `/api/docs`, specifically for the air-gapped production box.
The tracked spec holds 377 paths and `info.version` is still `0.12.0`. It
contains zero paths matching `enroll` or `bootstrap`, while
`git diff 2ac2e6d33..gh-wj -- 'plugins/*/api/*.py' 'shopdb/core/api/*.py'` adds
24 route decorators, including the whole enrollment surface
(`plugins/endpoints/api/enrollment.py:289`, `:580`, `:657`, `:752`, `:773`,
`:827`, `:853`, `:1095`, `:1135`, `:1233`, `:1253`) and the bootstrap-token
surface (`plugins/endpoints/api/bootstrap.py:419`, `:474`, `:485`, `:506`,
`:530`). Where the spec does describe a route that changed, it describes the old
behaviour: `/api/plugins/settings-routes` is documented as
"**Auth:** none (jwt-optional decorator, deliberately anonymous...)" with
`"security": [{}, {"bearerAuth": []}]`, while the shipped handler is
`@require_permission('settings.view')`.
Neither ADR-001 nor ADR-002 mentions `openapi.json` or `api-inventory.json`, so
no stated rule is broken. The drift also cannot be detected or repaired from
this branch: `docs/api-inventory.json` is not tracked here, so
`tests/test_api_inventory_parity.py` skips and `scripts/gen_openapi.py` has no
input.
**`CLAUDE.md` in this repo contradicts the repo.** Line 3 and line 9 describe
the database as "PostgreSQL 15+", while `docker-compose.yml:20` runs
`mysql:8.4` and `.env.example:29` is a `mysql+pymysql://` URL. Line 31 states
"**Columns**: snake_case (e.g., `serial_number`, `last_seen`)", which is the
opposite of `CONTRIBUTING.md:17` ("Column names: lowercase, concatenated,
singular. No dashes, no underscores") and of what the gate enforces at
`scripts/check-naming-and-style.sh:236`. Line 119 carries the ADR-016 device
token misattribution described above. `CLAUDE.md` is outside this folder and was
not edited.
## Scope note
This page records ADR conformance only. The same review produced security
findings against the per-PC credential surface, the kiosk network admission and
the one-line bootstrap; those are not ADR matters and are recorded in
`Security-Audit-2026-09-11.md`, refreshed on the same date and the same commit.
Violation 3 on this page and finding 31 on that one are the same ADR-016
misattribution seen from two angles. The highest of the security findings, for
cross-reference, is that `install.ps1` writes the fleet shared key to
`HKLM\SOFTWARE\GE\ShopDB` with no ACL
(`plugins/endpoints/api/siteagent.py:328`, helper generated at `:394-411`, no
`SetAccessControl` or `RegistrySecurity` anywhere in the generated installer),
where any logged-on user can read it - a consequence the repo states itself at
`.github/GE-ENFORCE-COLLECTOR-FIX.html:377-380`.
## References
- ADR set: `docs/adr/ in the product repo` (NOT on this branch)
- `scripts/check-naming-and-style.sh` - the only ADR-aware gate in this repo
- `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md` - the rollout runbook
- ADR-019-Plugin-Dynamic-Settings-Routes.md - the one ADR-shaped file here
- Security-Audit-2026-09-11.md - the security half of the same review
- Architectural-Findings-2026-09-11.md - re-checked 2026-09-18
- Maintenance-Footprint-Baseline.md - re-checked 2026-09-18

638
docs/wiki-drafts/Architectural-Findings-2026-09-11.md
View File

@@ -1,443 +1,405 @@
# Architectural Findings - September 11, 2026
**Scope:** Full codebase analysis via Gortex semantic code intelligence
**Severity:** 22 findings (6 critical, 6 high, 10 medium)
**Status:** Documented for remediation planning
**Scope:** Full codebase analysis via Gortex semantic code intelligence
**Original severity claim:** 22 findings (6 critical, 6 high, 10 medium)
**Status:** RE-CHECKED against branch gh-wj at 35d4fcd41 on 2026-09-18. Most of
the original findings do not survive a read of the code. This page keeps them,
with the verdict and the evidence, so nobody acts on a claim that was never
true.
## Executive Summary
## How to read this page (added 2026-09-18)
Gortex semantic analysis identified architectural issues in plugin boundaries, dependency management, and data schema consistency. Most critical: circular dependencies between plugins violate the plugin isolation model, and schema drift between SQLAlchemy models and Alembic migrations creates deployment risks.
The September 11 analysis was written from semantic queries, not from reading
the files it names. Eight of its claims cite code that does not exist on this
branch: a `machines` plugin that imports `computers`, a migration that spells a
column `last_reported_date`, a ServiceNow plugin, `MACHINE_TYPES = ['Lathe',
'Mill', 'CNC']`, a hardcoded `['Bldg 300', 'Bldg 400']`, Vuex, an absent CSP
header, an unhandled `requests.post`. Each of those was grepped for on
2026-09-18 and returns nothing.
**Key theme:** Plugin architecture (ADR-001, ADR-008) is well-designed but not consistently enforced. Code grew organically with cross-plugin imports, shared state leakage, and implicit coupling that violates the self-contained plugin contract.
Every finding below now carries one of four verdicts:
- **REFUTED** - the cited code does not exist, or the opposite is in the tree.
- **STALE** - it was true once; the tree has since fixed it.
- **HOLDS (narrowed)** - something real is there, but smaller or different than
the original text claims.
- **UNVERIFIED** - cannot be settled from this repository.
Effort and priority numbers in the original text were estimates with no stated
basis. They are left where they sit for findings that still hold and struck
where the finding does not.
## Critical Findings (P0)
### 1. Circular Dependencies Between Plugins
**Risk:** Plugin load order matters. Enabling/disabling plugins can break others.
**Verdict: REFUTED as stated. HOLDS (narrowed) as a deferred-import coupling.**
**Evidence:**
- `computers` plugin imports from `machines` plugin: `from plugins.machines.models import Machine`
- `machines` plugin imports from `computers` plugin: `from plugins.computers.models import Computer`
- `printers` plugin imports from `computers` plugin
- `endpoints` plugin imports from `computers`, `printers`, and `usb` plugins
**Original claim:** `computers` imports `machines` and `machines` imports
`computers`; import errors at runtime if plugins load in the wrong order; 25+
violations.
**Impact:**
- Violates plugin isolation (ADR-001: plugins are independent extensions)
- Disabling `computers` breaks `machines`, `printers`, `endpoints`
- Plugin dependency chain not declared in plugin metadata
- Import errors at runtime if plugins loaded in wrong order
**What the code shows (2026-09-18):**
**Root cause:** Plugins use each other's models for relationships (e.g., Computer controls Machine).
- `grep -rn "^from plugins\." plugins/ --include=*.py` returns **nothing**.
There is not one module-level cross-plugin import in the tree, so there is no
load-order failure mode and no import cycle at import time.
- `grep -rn "from plugins.computers" plugins/machines/` returns **nothing**.
The `machines` plugin does not reference `computers` at all, so the cycle the
finding is built on does not exist.
- 52 cross-plugin imports do exist, and every one of them is inside a function
body, resolved on call. `plugins/computers/plugin.py:414` and `:1166` reach
`plugins.machines.models` that way; `plugins/endpoints/api/routes.py:235` and
fourteen more reach `plugins.computers.models`.
**Proper pattern:** Relationship via `Asset` base class
```python
# BAD: Direct plugin-to-plugin import
from plugins.machines.models import Machine
**What is actually there:** a mutual reference pair between `computers` and
`endpoints`, both deferred:
# GOOD: Relationship via Asset ID, resolve at runtime
machine_asset_id = db.Column(db.Integer, db.ForeignKey('asset.id'))
machine = relationship('Asset', foreign_keys=[machine_asset_id])
- `plugins/computers/api/policy_routes.py:44` imports
`plugins.endpoints.commands.elevated_permission_for`, unguarded.
- `plugins/endpoints/commands.py:477` imports
`plugins.computers.services.policyresolver.effective_policy_for`, wrapped in
`try/except` that falls back to the base allowlist (`commands.py:475-481`).
# In plugin manifest, declare dependency
dependencies = ['machines'] # Explicit, not implicit
```
So one side of that pair degrades when its partner is absent and the other
raises. The same asymmetry appears at
`plugins/endpoints/api/agentauth.py:118`, where `resolve_hostname` imports
`plugins.computers.models.Computer` with no guard: an endpoints-without-
computers site fails there rather than returning "unknown hostname".
**Remediation:**
1. Audit all `from plugins.<other>` imports (25+ violations found)
2. Replace with Asset-based relationships or declared dependencies
3. Enforce via linter: plugins can import from `shopdb.api`, not `plugins.*`
**Effort:** 60 hours (high impact, requires refactoring multiple plugins)
**Priority:** Q1 2027 (blocks plugin catalog ADR-013 lean builds)
**What to do instead of the original remediation:** nothing about load order.
The open question is narrower and worth one pass: decide whether a deferred
cross-plugin import must be guarded, and guard the unguarded ones. A linter
rule banning `plugins.*` imports outright would break the working policy and
enrollment paths listed above, which are the product's actual design.
### 2. Undeclared Import Dependencies
**Risk:** Plugins import from `shopdb.core` internals instead of stable `shopdb.api` surface.
**Verdict: REFUTED.**
**Evidence:**
- 18 plugins import from `shopdb.core.models` directly (not via `shopdb.api`)
- 12 plugins import from `shopdb.core.utils` (internal helper modules)
- 7 plugins import from `shopdb.core.services` (not part of plugin API)
**Original claim:** 18 plugins import `shopdb.core.models`, 12 import
`shopdb.core.utils`, 7 import `shopdb.core.services`.
**Impact:**
- Plugins break when core refactors internals
- No API versioning or stability guarantee
- Circular imports between core and plugins
**What the code shows:**
`grep -rn "from shopdb\.\(core\|utils\|extensions\|services\)" plugins/
--include=*.py` returns **0 hits**. Not one plugin deep-imports core. The only
`shopdb.*` import in `plugins/` outside `shopdb.api` is
`shopdb.plugins.alembic_template` from migration `env.py` files, which is the
shared Alembic runner ADR-008 sanctions.
**Plugin API contract (ADR-001):** Plugins MUST import from `shopdb.api` only
```python
# GOOD: Stable plugin API
from shopdb.api import db, Asset, BasePlugin, success_response
ADR-001's rule that `shopdb.api` is the only core surface a plugin may import is
being kept. The thing that is NOT enforced is raw SQL against core tables: see
finding 3a below.
# BAD: Internal implementation details
from shopdb.core.models.asset import Asset # Internal path
from shopdb.core.services.search import SearchService # Not exported
```
### 2a. Raw SQL against core tables from a plugin (added 2026-09-18)
**Remediation:**
1. Audit imports: `grep -r "from shopdb.core" plugins/`
2. Export needed classes via `shopdb.api.__init__.py`
3. Update plugin imports to use `shopdb.api`
4. Add linter rule: plugins cannot import `shopdb.core.*`
**Verdict: HOLDS (new, narrow).**
**Effort:** 40 hours
**Priority:** Q4 2026 (enables core refactoring without breaking plugins)
`plugins/computers/plugin.py:2080-2091` runs
`db.session.execute(db.text(...))` joining core `assets`, `assetrelationships`
and `assettypes` inside the `flask computers retype` CLI command. `Asset`,
`AssetType`, `AssetRelationship` and `RelationshipType` are all exported from
`shopdb.api` (`shopdb/api/__init__.py:47-48`, `:424`, `:438-439`), so no
missing export excuses it, and `plugins/endpoints/api/routes.py:3981-3986` is
the same pattern already corrected for `BusinessUnit`. The contract note at
`shopdb/__init__.py:135-142` names the reason: a raw query is coupling to the
core schema that no contract bump can catch.
Blast radius: one admin CLI command, no serving route. Import rule not
literally broken. Worth fixing because it is the kind of thing that gets copied.
### 3. Schema Drift: Models vs Migrations
**Risk:** SQLAlchemy models don't match Alembic migrations. Deployed schema differs from code.
**Verdict: REFUTED as stated. HOLDS (narrowed) to one column.**
**Evidence (examples):**
- `Computer.lastreporteddate` column in model, but migration creates `last_reported_date` (snake_case mismatch)
- `Printer.driver` column added to model but no migration exists
- `AssetRelationship.label` column in migration but not in model
- 14 tables with mismatched column names or missing columns
**Original claim:** `Computer.lastreporteddate` in the model but
`last_reported_date` in the migration; `Printer.driver` with no migration;
`AssetRelationship.label` in a migration but not the model; 14 tables affected.
**Impact:**
- Fresh install (runs migrations) has different schema than ORM expects
- Queries fail: `Column 'lastreporteddate' does not exist in table`
- Adding data via ORM writes to wrong column
- Impossible to tell what production schema actually is (migrations are source of truth, but models are what dev sees)
**What the code shows:**
**Root cause:**
- Naming convention changed (ADR: lowercase concatenated, no underscores)
- Manual migrations written with old convention
- No validation that migrations match models
- `grep -rn "last_reported_date" --include=*.py .` returns **nothing**. The
model spells it `lastreporteddate` (`plugins/computers/models/computer.py:85`)
and so does every migration.
- `grep -rPn "sa\.Column\(\s*'[a-z]+_[a-z_]+'" plugins/*/migrations/versions/*.py
migrations/versions/*.py` returns **nothing**. There is no snake_case column
name in any migration in the tree.
- There is no `shopdb/models/` directory to hold the claimed
`AssetRelationship.label`; core models live under `shopdb/core/`.
**Remediation:**
1. Generate fresh migration from current models: `flask db migrate`
2. Compare to existing migrations, find diffs
3. Write fixup migration to align schema with models
4. Add test: `test_migrations_match_models()` validates ORM vs DB schema
5. Enforce naming convention in migration template
**The one real instance:** `plugins/notifications/models/notification.py:158`
declares `alertscope = db.Column(db.JSON, nullable=True)` while the migration
that creates it, `plugins/notifications/migrations/versions/0006_add_fleet_alerts.py:45`,
creates `sa.Column('alertscope', sa.Text, nullable=True)` and its own docstring
at line 8 calls it "the alertscope TEXT field (JSON string)". Commit f484be8a3
fixed the two neighbouring columns to agree with 0006 and left this one. Nothing
breaks on MySQL 8 - the SQLAlchemy JSON type serialises into TEXT and reads back
- and consumers already normalise both shapes
(`plugins/endpoints/api/routes.py:4011-4020`). The cost is that the model is not
a true description of the deployed column, which is what autogenerate reads.
**Effort:** 80 hours (high risk, requires careful schema analysis)
**Priority:** Q4 2026 (CRITICAL - blocks confidence in production schema)
**Related gap that is real:** the naming gate cannot see migrations at all.
`scripts/check-naming-and-style.sh:236` greps
`^\s+[a-z]+_[a-z_]+\s*=\s*db\.Column`, which only matches a model attribute
assignment; migrations write `sa.Column('name', ...)` with the name as a
positional string. That is how the shipped build of
`plugins/endpoints/migrations/versions/0021_client_certificates.py` created
`client_cert_fingerprint`, `cert_issued_at` and `cert_expires_at` and passed the
gate (`git show 5cd533239:plugins/endpoints/migrations/versions/0021_client_certificates.py`,
lines 20-28). The tree is conformant today, because 0021 and 0022 are now
no-ops and 0023 renamed the survivor; the gate simply would not notice the next
one.
### 4. Single Points of Failure
**Risk:** Core services have no failover or redundancy.
**Verdict: UNVERIFIED from this repository, and partly STALE.**
**Evidence:**
- Single database instance (no replication)
- Single Flask app instance (gunicorn workers on one server)
- Single Docker host (no orchestration)
- No load balancing
Replication, load balancing and orchestration are properties of the reference site
deployment, not of this tree, so nothing here can confirm or deny them. Two
parts of the original text are wrong about what the repo ships:
**Impact:**
- Database outage = full system outage
- App server failure = downtime until manual restart
- Hardware failure = data loss without recent backup
- Restart policy is configured, not absent: `docker-compose.yml:24`, `:104` and
`:120` set `restart: unless-stopped`, and three services carry `healthcheck:`
blocks (`:38`, `:112`, `:219`).
- Backups are no longer "manual mysqldump weekly, stored on the same server".
See finding 4a.
**Mitigation (current):**
- Weekly manual backups (mysqldump)
- VM snapshots before deploys
- Docker restart on failure (docker-compose restart policy)
The underlying point - one database, one app host, air gap limits the options -
is a statement about the site, and belongs in a deployment document rather than
a code findings page. The 200+ hour estimate has no basis in anything measured.
**Proper solution:** High availability architecture
- Database: MySQL master-replica replication
- App: Multiple gunicorn instances behind nginx load balancer
- Orchestration: Docker Swarm or Kubernetes for auto-restart
- Monitoring: Automated health checks + alerting
### 4a. Backups (added 2026-09-18, correcting the original text)
**Effort:** 200+ hours (infrastructure redesign)
**Priority:** Q2 2027 (production risk, but air-gapped environment has limited HA options)
**Verdict: STALE. The repo ships a backup system.**
**Note:** Air-gap deployment model limits cloud HA solutions. On-premise HA requires hardware investment.
`scripts/backup/` contains `backup.sh`, `restore.sh`, `status.sh`, `watch.sh`,
`simulate-restore.sh` and `test-restore-datarev.sh`. Retention is enforced by
restic (`scripts/backup/backup.sh:134-141`: `--keep-daily 5 --keep-weekly 4
--keep-monthly 6 --prune`), destinations include SMB and S3 as well as local
(`scripts/backup/README.md`), each set carries `manifest.json` with schema
revisions for compatibility checking, and restore runs from the Settings >
Backups UI with a schema compatibility check. A restore-simulation harness is in
the tree.
### 5. Plugin Migration Chain Validation Gaps
**Risk:** ADR-008 (plugin migration ownership) not enforced by tooling.
**Verdict: STALE as written, and the credited fix is the wrong artifact.**
**Evidence:**
- Manual inspection required to verify linear chains
- Plugin migrations can create branches (multiple head revisions)
- `PLUGIN_TABLE_OWNERS` order not validated against actual dependencies
Validation is in the test suite, in this repository, and has been for longer
than the `/migration-check` skill: `tests/test_plugin_migrations.py` holds nine
tests covering owners-versus-models, chain shape, no-op anchors and idempotency,
and pins each chain's head in `EXPECTED_HEAD_REVISION`
(`tests/test_plugin_migrations.py:125`). `tests/test_plugins/test_endpoints_credentials_migration.py`
runs the 0020-0023 chain twice from both a legacy and a fresh schema, plus
downgrade and re-upgrade. The endpoints chain today is 24 revisions with one
root and one head and no dangling `down_revision`.
**Example violation:**
```
plugins/computers/migrations/versions/
001_initial.py (down_revision=None)
002_add_access.py (down_revision=001)
003_add_policies.py (down_revision=001) # BRANCH! Two revisions point to 001
```
The branch example in the original text (`003_add_policies.py` with
`down_revision=001`) is invented; no such file exists under
`plugins/computers/migrations/versions/`, which holds 6 revisions.
**Impact:**
- Alembic upgrade fails with "Multiple head revisions" error
- Plugin can't be enabled until migration chain fixed
- Hard to diagnose (requires reading migration files manually)
**Proper validation:** `/migration-check` skill (created 2026-09-11)
- Checks: Each plugin has linear chain (no branches)
- Checks: `down_revision` references exist
- Checks: `PLUGIN_TABLE_OWNERS` order matches declared dependencies
- Status: Implemented, run via pre-commit hook
**Effort:** DONE (skill created)
**Priority:** Resolved 2026-09-11
**The real gap, which the original text missed:** no test asserts that a plugin
chain touches only the tables it owns. `plugins/endpoints/migrations/versions/0023_endpoint_credentials.py`
declares `plugin_name = 'endpoints'` (line 33) and then runs
`op.drop_index(..., table_name='assets')` at line 73 and `op.drop_column('assets',
column)` at line 76 against the core `assets` table, cleaning up after the
shipped 0021 which had added those columns from the same plugin chain. `assets`
is not, and cannot be, in `PLUGIN_TABLE_OWNERS['endpoints']`
(`shopdb/plugins/alembic_template.py:57-63`), so the metadata ownership filter
never saw it. Every step is guarded on the live schema and the forward direction
is clean, so nothing is broken; nothing stops the next one either.
### 6. Implicit Database Connection Sharing
**Risk:** Plugins share `db.session` without transaction isolation.
**Verdict: REFUTED as a blanket claim.**
**Evidence:**
- Multiple plugins write to database in same request
- No explicit transaction boundaries
- Rollback in one plugin affects others
Savepoints are in use where concurrency actually bites:
`plugins/endpoints/plugin.py:1318`, `:1387` and `:1692` all wrap work in
`with db.session.begin_nested():`, and the application-delta path reconciles a
unique-constraint collision inside that savepoint rather than sinking the
check-in. The original text's example - a printers rollback silently discarding
a computers write in the same request - is not backed by any cited call site,
and none was found.
**Example:**
```python
# computers plugin writes Computer
computer = Computer(hostname='PC1')
db.session.add(computer)
# printers plugin writes Printer (same session!)
printer = Printer(name='Printer1')
db.session.add(printer)
# If printer validation fails and rolls back, computer write is lost too
db.session.rollback()
```
**Impact:**
- Plugin A's failure can undo Plugin B's work
- No atomicity guarantee per-plugin
- Hard to debug (cross-plugin transaction interactions)
**Best practice:** Explicit transaction scopes per plugin
```python
from shopdb.api import db
with db.session.begin_nested(): # Savepoint for this plugin
computer = Computer(hostname='PC1')
db.session.add(computer)
db.session.flush()
# If exception here, only this savepoint rolls back
```
**Remediation:**
1. Audit plugin DB writes: find implicit `db.session.add()` without savepoint
2. Wrap in `begin_nested()` savepoints or explicit `commit()`
3. Document transaction patterns in PLUGIN-DEVELOPMENT.md
**Effort:** 40 hours
**Priority:** Q1 2027 (data integrity risk, but low frequency in practice)
A per-plugin transaction policy is still not written down anywhere. That is a
documentation item for PLUGIN-DEVELOPMENT.md, not a critical data-integrity
finding.
## High Priority Findings (P1)
### 7. Frontend Plugin Import Violations
**Risk:** Plugin frontends import from each other, violating ADR-013 isolation.
**Verdict: STALE. Enforced and passing.**
**Evidence:**
- `plugins/printers/frontend/` imports from `plugins/computers/frontend/`
- 5 plugins import shared components via relative paths `../../other-plugin/`
**ADR-013 rule:** Plugin frontends can import:
- Core via `@/` alias (e.g., `@/components/Shared.vue`)
- Own tree via `./` or `../` within plugin
- NOT other plugin trees
**Proper pattern:**
```javascript
// BAD: Cross-plugin import
import PCSelect from '../../computers/frontend/components/PCSelect.vue'
// GOOD: Core provides shared component
import PCSelect from '@/components/PCSelect.vue'
// OR: Plugin exposes via API, frontend fetches data
const pcs = await axios.get('/api/computers/')
```
**Enforcement:** `check-naming-and-style.sh` now checks for escaping imports
```bash
grep -rPn "(import|from)\s+['\"]([^'\"]*\.\./\.\./|[^'\"]*/plugins/)" \
plugins/*/frontend/
```
**Remediation:**
1. Move shared components to `frontend/src/components/`
2. Update plugin imports to use `@/` alias
3. Run pre-commit check to catch violations
**Effort:** 20 hours
**Priority:** Q4 2026 (blocks lean builds)
**Status:** Linter rule added 2026-09-11 (commit 4943e182c3)
`scripts/check-naming-and-style.sh` runs "Checking for cross-plugin / escaping
imports in plugin frontends" and the whole script exits 0 on this branch. The
established pattern is a core component consumed through the `@/` alias: 97
plugin frontend files already import that way, and the kiosk allowlist editor
added in this range follows it
(`frontend/src/components/NetworkAllowlistPicker.vue`, consumed at
`plugins/usb/frontend/views/USBSettings.vue:108` and
`plugins/printedparts/frontend/views/PrintedPartsSettings.vue:164`).
### 8. Hardcoded Site Literals in Plugins
**Risk:** Site-specific data (building names, machine types, printer models) hardcoded in plugin code.
**Verdict: REFUTED as stated. HOLDS (narrowed) as a gate coverage gap.**
**Evidence (violations of ADR-015):**
- `plugins/machines/models.py` has `MACHINE_TYPES = ['Lathe', 'Mill', 'CNC']` hardcoded
- `plugins/printers/api/routes.py` validates `building in ['Bldg 300', 'Bldg 400']`
- 12 plugins have site-specific literals
`grep -rn "MACHINE_TYPES" . --include=*.py` and
`grep -rn "Bldg 300\|Bldg 400" . --include=*.py` both return **nothing**. The
ADR-015 block in `scripts/check-naming-and-style.sh:392-462` runs and passes,
and a control pattern over the same directories returns 1995 lines, so the
pipeline is not vacuous.
**ADR-015 rule:** Site data MUST live in:
- `shopdb/core/settings/` - Settings table defaults
- `shopdb/core/seed/` - Initial data SQL
- `scripts/site_imports/` - Import tools for site-specific data
**Proper pattern:**
```python
# BAD: Hardcoded site data
MACHINE_TYPES = ['Lathe', 'Mill', 'CNC']
# GOOD: Setting with neutral default
machine_types = Setting.get('machines_allowed_types', default='[]')
MACHINE_TYPES = json.loads(machine_types)
```
**Enforcement:** `check-naming-and-style.sh` scans for literal building names, FQDNs, network ranges
**Remediation:**
1. Extract literals to Settings table
2. Update plugins to read from Settings
3. Seed data with site-specific values during setup
**Effort:** 30 hours
**Priority:** Q1 2027 (blocks multi-site deployments)
**The narrowed finding:** the ADR-015 scan is scoped at
`scripts/check-naming-and-style.sh:436-440` to `plugins/ shopdb/ scripts/
deploy/ frontend/src/ tools/` with `--include` on
`.py/.vue/.js/.ps1/.psm1/.sh/.iss/.json/.html`. `agent/` is not in that list and
`.cs` is not an included extension, so roughly 2,500 lines of C# added in this
range - `AgentCredentials.cs`, `AgentHttpFactory.cs`, `EnrollmentService.cs`,
`AgentPipeServer.cs`, `ProtectedFileSystem.cs`, `StrictModePreamble.cs`, plus
`agent/appsettings.json` - ship to a sister site unscanned. Re-running the
assembled site patterns by hand over `agent/` (excluding bin/obj) finds one hit,
prose in `agent/ShopDbEndpointAgent/ShopDbEndpointAgent.csproj:108`, which
ADR-015 permits. The tree is clean; the gate would not notice if it stopped
being clean. Note this is a gap in ADR-015's own stated scope, not a breach of
it: the 2026-08-14 amendment names those six directories.
### 9. Missing Error Handling on External Calls
**Risk:** Plugins don't handle failures when calling other services.
**Verdict: REFUTED.**
**Evidence:**
- ServiceNow plugin: no timeout on HTTP requests, no retry logic
- LDAP sync: no handling of connection failures
- Email plugin: assumes SMTP always succeeds
The plugins the finding names do not exist. `ls plugins/` gives: applications,
backups, computers, employees, endpoints, geenforce, guides, knowledgebase,
machines, measuringtools, network, notifications, printedparts, printers,
slides, tools, usb, warranty. There is no ServiceNow plugin, no LDAP sync
plugin and no email plugin; mail is core (`shopdb/utils/mailer.py`).
**Impact:**
- Hung requests block worker threads
- Transient failures cause permanent data loss
- No user feedback when external service down
Every outbound HTTP call in the tree carries a timeout and an exception handler:
**Best practice:** Timeouts, retries, circuit breakers
```python
# BAD: No timeout
response = requests.post(servicenow_url, data=payload)
# GOOD: Timeout + retry
response = requests.post(servicenow_url, data=payload, timeout=10)
if response.status_code >= 500:
# Retry with exponential backoff
time.sleep(2 ** attempt)
```
**Remediation:**
1. Audit external calls: `grep -r "requests\.(get|post)" plugins/`
2. Add timeouts (default 10s)
3. Add retry logic for transient failures (5xx errors)
4. Log failures for monitoring
**Effort:** 30 hours
**Priority:** Q4 2026 (production stability)
- `plugins/warranty/services/providers.py:129` and `:157` - `timeout=HTTP_TIMEOUT`,
`except requests.RequestException` raising `WarrantyLookupError`.
- `plugins/printers/services/zabbix_service.py:95` and `:126` -
`timeout=self.REACHABLE_TIMEOUT` / `self.API_TIMEOUT`, caught.
- `shopdb/utils/mailer.py:238` - `timeout=10`, caught, returns `(False, error)`
and never raises.
### 10-12. Additional High Findings
See memory file `architectural_analysis_gortex_2026_09_11.md` for:
- Inconsistent permission checking across plugins
- Plugin state leakage via module-level globals
- Missing database connection pool tuning
The original text deferred these to a memory file rather than citing code:
"inconsistent permission checking across plugins", "plugin state leakage via
module-level globals", "missing database connection pool tuning".
The third is REFUTED. The pool is sized on purpose and env-driven:
`shopdb/config.py:20-22` (`DB_POOL_SIZE` 10, `DB_MAX_OVERFLOW`, `DB_POOL_TIMEOUT`
5) and `:48-72`, with the reasoning for the non-default `pool_timeout` written
in place, and `docker-compose.yml:172-193` documents the
`workers x (pool_size + max_overflow)` arithmetic against MySQL's
`max_connections`.
The other two were not re-checked, because neither cites a file. They should be
restated with a file:line or dropped.
## Medium Priority Findings (P2)
### 13. No Rate Limiting on API Endpoints
**Risk:** Collector endpoints can be flooded, exhausting worker threads.
**Verdict: HOLDS (narrowed). Login is limited; two kiosk surfaces are not.**
**Mitigation:** See Security-Audit-2026-09-11.md (login rate limiting planned Q1 2027)
Login rate limiting exists and is not "planned Q1 2027": `shopdb/core/api/auth.py:35`
`_login_ratelimited()`, keyed per host at `:52`, applied at `:80`, with
`tests/test_core/test_auth_ratelimit.py` over it. Enrollment has its own limiter
(`plugins/endpoints/api/ratelimit.py`, 300 per 60 seconds per IP applied at
`plugins/endpoints/api/enrollment.py:306`).
What has none: the USB kiosk routes. `grep` for `limiter` or `ratelimit` across
`plugins/usb/` returns nothing, while `plugins/usb/api/selfhosted.py:346-371`
resolves a posted badge to an employee name and returns it. That is an
unauthenticated, unthrottled badge-to-name lookup for any caller inside the
configured kiosk CIDRs.
Both limiter counters live in per-process `SimpleCache`
(`plugins/endpoints/api/ratelimit.py:10-11`), so at the shipped four gunicorn
workers the effective budget is four times the configured maximum.
### 14. Frontend State Management Inconsistencies
**Risk:** Some components use Vuex, others use local state, no clear pattern.
**Verdict: REFUTED.**
**Impact:** Hard to track state flow, data duplication, synchronization bugs
**Recommendation:** Standardize on Pinia (Vue 3 state management) or document when to use local vs global state
`grep -rn "vuex" frontend/src/` returns **nothing**, and `frontend/package.json:28`
declares `"pinia": "^2.1.0"`. There is no Vuex-versus-Pinia split to
standardise.
### 15. No API Versioning
**Risk:** Breaking API changes affect all clients (frontend, agent, integrations) simultaneously.
**Verdict: HOLDS.**
**Current:** All endpoints at `/api/`, no version in path
**Recommended:** `/api/v1/` with deprecation policy for v0 -> v1 migration
**Effort:** 60 hours (requires API redesign)
**Priority:** v1.0 release
`grep -rn "api/v1" shopdb/ --include=*.py` returns nothing; every route is
mounted under `/api/`. This is a v1.0 decision, not a defect, and ADR-002 already
carries the versioning story for the plugin contract. The 60-hour estimate has
no stated basis.
### 16-22. Additional Medium Findings
See memory file for:
- Missing request ID tracing across plugins
- No structured logging format
- Plugin discovery relies on filesystem scan (slow)
- Frontend bundle size (2.1MB) larger than ideal
- No content security policy headers
- Database connection pool too small for peak load
- Missing health check endpoints for monitoring
Re-checked on 2026-09-18:
## Remediation Priority
- **No content security policy headers: REFUTED as stated.** `shopdb/__init__.py:340`
sets `Content-Security-Policy: frame-ancestors 'self'` on every response, plus
`X-Content-Type-Options`, `X-Frame-Options` and `Referrer-Policy` at `:334-337`;
uploads carry a stricter `default-src 'none'; sandbox`
(`shopdb/utils/uploads.py:13`), pinned by `tests/test_core/test_security_headers.py`.
The accurate, narrower statement is that there is no `script-src` policy, and
the code says so in place and says why.
- **Missing health check endpoints: REFUTED.** `shopdb/core/api/dashboard.py:318`
is `GET /api/dashboard/health`.
- **Database connection pool too small: REFUTED.** See 10-12 above.
- **No request ID tracing: HOLDS.** No request-id plumbing was found in
`shopdb/__init__.py`.
- **No structured logging format: not re-checked.**
- **Plugin discovery relies on a filesystem scan: HOLDS, by design.** ADR-003
decides filesystem discovery under `plugins/`. "Slow" was never measured.
- **Frontend bundle size 2.1MB: UNVERIFIED.** No build was run for this page.
| Finding | Severity | Effort (hrs) | Priority | Status |
|---------|----------|--------------|----------|--------|
| Schema drift (models vs migrations) | CRITICAL | 80 | Q4 2026 | Planned |
| Circular plugin dependencies | CRITICAL | 60 | Q1 2027 | Planned |
| Undeclared import dependencies | CRITICAL | 40 | Q4 2026 | Planned |
| Migration chain validation | CRITICAL | 0 | DONE | /migration-check skill |
| Frontend plugin imports | HIGH | 20 | Q4 2026 | Linter added |
| Site literals in plugins | HIGH | 30 | Q1 2027 | Planned |
| Missing error handling | HIGH | 30 | Q4 2026 | Planned |
| Implicit DB session sharing | HIGH | 40 | Q1 2027 | Planned |
| Single points of failure | CRITICAL | 200+ | Q2 2027 | Infrastructure |
| API versioning | MEDIUM | 60 | v1.0 | Planned |
## Remediation Priority (revised 2026-09-18)
## Testing Performed
| Finding | Verdict | What is actually left to do |
|---------|---------|------------------------------|
| Circular plugin dependencies | REFUTED / narrowed | Decide whether deferred cross-plugin imports must be guarded; guard `agentauth.py:118` and `policy_routes.py:44` |
| Undeclared core imports | REFUTED | Nothing |
| Schema drift | REFUTED / narrowed | Align `notification.py:158` with migration 0006; teach the naming gate to read `sa.Column('...')` |
| Plugin chain touches core tables | HOLDS (new) | Add a test that a plugin chain writes only owned tables |
| Migration chain validation | STALE | Nothing; `tests/test_plugin_migrations.py` is the gate |
| Implicit DB session sharing | REFUTED | Document the transaction pattern in PLUGIN-DEVELOPMENT.md |
| Frontend plugin imports | STALE | Nothing; gate passes |
| Site literals in plugins | REFUTED / narrowed | Extend the ADR-015 scan to `agent/` and `.cs` |
| Missing external-call error handling | REFUTED | Nothing |
| Rate limiting | HOLDS (narrowed) | Throttle the USB kiosk badge lookup; decide whether per-process counters are acceptable at 4 workers |
| Vuex/Pinia split | REFUTED | Nothing |
| API versioning | HOLDS | v1.0 decision |
| Single points of failure | UNVERIFIED | Belongs in a deployment document |
## Testing Performed (2026-09-18 re-check)
**Gortex semantic analysis:**
```bash
gortex explore "find circular dependencies between plugins"
gortex explore "find undeclared imports from shopdb.core"
gortex explore "find schema mismatches between models and migrations"
# Every claim above was checked with a grep or a read on gh-wj at 35d4fcd41.
grep -rn "^from plugins\." plugins/ --include=*.py # 0 hits
grep -rn "from shopdb\.\(core\|utils\|extensions\|services\)" \
plugins/ --include=*.py # 0 hits
grep -rPn "sa\.Column\(\s*'[a-z]+_[a-z_]+'" \
plugins/*/migrations/versions/*.py migrations/versions/*.py # 0 hits
grep -rn "MACHINE_TYPES\|Bldg 300\|Bldg 400" . --include=*.py # 0 hits
grep -rn "vuex" frontend/src/ # 0 hits
grep -rn "api/v1" shopdb/ --include=*.py # 0 hits
bash scripts/check-naming-and-style.sh # exit 0
```
**Migration chain validation:**
```bash
/migration-check # Runs pytest tests/test_plugin_migrations.py
# PASS: All plugin migration chains are linear
```
**Frontend import validation:**
```bash
bash scripts/check-naming-and-style.sh
# Scans for plugin-to-plugin imports (../../ escaping)
# PASS: No violations after cleanup
```
## Recommendations
**Immediate (Q4 2026):**
1. Fix schema drift (models vs migrations) - CRITICAL
2. Enforce plugin API imports (no shopdb.core) - enables refactoring
3. Add error handling to external service calls - stability
**Short-term (Q1 2027):**
1. Break circular plugin dependencies - enables lean builds
2. Remove site literals from plugins - enables multi-site
3. Fix implicit DB session sharing - data integrity
**Long-term (Q2-Q4 2027):**
1. High availability architecture (if feasible in air-gap)
2. API versioning for v1.0 release
3. Frontend optimization (bundle size, state management)
No test suite was run for this re-check; every verdict above is static.
## References
- Gortex MCP Server: Semantic code intelligence for architectural analysis
- ADR-001: Asset as Platform Contract (plugin isolation model)
- ADR-003: Plugin Distribution (filesystem discovery)
- ADR-008: Plugin Migration Ownership (linear chains, PLUGIN_TABLE_OWNERS)
- ADR-013: Plugin Catalog and Lean Builds (frontend isolation)
- ADR-015: Site-Specific Configuration (no literals in code)
- Memory file: `.claude/memory/architectural_analysis_gortex_2026_09_11.md`
- ADR-Conformance-2026-09-18.md: the ADR-by-ADR result of the 2026-09-18 review,
including the fact that the ADR set is not in this repository

216
docs/wiki-drafts/Maintenance-Footprint-Baseline.md
View File

@@ -1,8 +1,37 @@
# Maintenance Footprint Baseline - 2026-09-11
**Total annual burden:** 936 hours/year
**Risk level:** 8 critical + 10 high operational risks
**Total annual burden:** 936 hours/year (estimate, see the note below)
**Risk level:** originally stated as 8 critical + 10 high operational risks
**Optimization potential:** 147-205 hours/year savings with automation
**Last checked against code:** 2026-09-18, branch gh-wj at 35d4fcd41
## About the numbers (added 2026-09-18)
Every hour figure on this page is an estimate with no recorded measurement
behind it. No time log, ticket export or deploy log is cited anywhere, and none
was found in the repository. Treat the hours as an ordering of where effort
goes, not as data. The same applies to the row counts and growth rates in the
Database Management section: no query output is cited and none could be taken
from a repository checkout.
What CAN be checked from the repository is which of the recommendations have
since been built. Several have. The sections below are annotated
accordingly, dated 2026-09-18, and the stale risk-matrix rows are corrected.
## What the tree actually contains (2026-09-18)
| Thing | Count | How counted |
|-------|-------|-------------|
| Plugins | 18 directories, 17 with a `manifest.json` | `ls -d plugins/*/`; `applications` carries no manifest |
| Plugin migration revisions | 70 across 17 chains | `ls plugins/*/migrations/versions/*.py` |
| Largest single chain | `endpoints`, 24 revisions | one root, one head, no branch |
| Core migration revisions | 58 | `ls migrations/versions/*.py` |
| Test files | 223 | `find tests -name "test_*.py"` |
| Top-level scripts | 29 (.sh/.py/.ps1/.mjs), plus `scripts/backup`, `scripts/dev`, `scripts/migration`, `scripts/loadtest`, `scripts/site_imports` | `ls scripts/` |
| Automation skills | 13 | `ls <assistant-config>/skills/*.md` minus README |
The "13 automation skills" claim later on this page is the one headline figure
that checks out exactly.
## Executive Summary
@@ -29,6 +58,19 @@ Baseline assessment of ShopDB operational burden across air-gapped production en
- Savings: 52 hrs/year (from 52 hrs to ~0 hrs, fully automated)
- Status: PLANNED for Q4 2026
**2026-09-18 update: half built, and the half that is built is manual.**
Retention exists as a service and a CLI, not as a cron job and not as a stored
procedure. `shopdb/core/services/auditretention.py` provides `configureddays()`,
`stats()` and `purge()`; `flask audit stats` (`shopdb/cli/__init__.py:1891`)
reports size against the configured window and `flask audit purge`
(`:1911`) deletes, with a dry-run preview first and `--days` to override for one
run. The window is the `audit_retentiondays` setting
(`shopdb/core/api/settings.py:1049`), and its default is **`'0'`, which means
keep everything forever**: nothing is deleted until an operator sets a value.
There is no scheduler in the application and no cron entry in the repository, so
the purge runs when somebody runs it. The remaining work is a value in Settings
> Site & Facility and a host cron line, not code.
**Endpoint events cleanup:** 26 hrs/year (HIGH RISK)
- Current: 450K rows, 127MB, no retention
- Growth: Process events every 30 seconds, app launches, USB connects
@@ -38,10 +80,32 @@ Baseline assessment of ShopDB operational burden across air-gapped production en
- Implementation: Similar cron job as audit log
- Savings: 26 hrs/year (fully automated)
**2026-09-18 update: partly built, per table rather than globally.** Operator
reports carry a retention window defaulting to 7 days
(`plugins/endpoints/api/routes.py:3619-3620`,
`endpoints.operatorreport.retentiondays`, 0 or less meaning keep forever), and
endpoint health transitions are trimmed to a high-water mark on write, keeping
every unresolved row regardless of age
(`plugins/endpoints/services/health.py:326`). The row counts and the 127MB
figure above were never sourced and are not verifiable from a checkout.
**Database backups:** 78 hrs/year (MEDIUM RISK)
- Current: Manual mysqldump weekly, stored on same server
- Current (as stated 2026-09-11): Manual mysqldump weekly, stored on same server
- Risk: No offsite copies, no restore testing, backup growth unbounded
- Backup size: 2.1GB compressed, growing 15% per quarter
- Backup size: 2.1GB compressed, growing 15% per quarter (unsourced)
**2026-09-18 update: STALE. This is built, and it is what the recommendation
below asks for.** `scripts/backup/` holds `backup.sh`, `restore.sh`, `status.sh`,
`watch.sh`, `simulate-restore.sh` and `test-restore-datarev.sh`, plus a
`Dockerfile.backup` and a compose file. Retention is enforced by restic:
`scripts/backup/backup.sh:134-141` runs `restic forget --keep-daily 5
--keep-weekly 4 --keep-monthly 6 --prune`. Destinations are configured in
Settings > Backups and include local, SMB and AWS S3, so offsite is a setting
rather than a manual copy. Each backup set carries `db.sql.gz`, the `instance/`
tree and a `manifest.json` of schema revisions, and restore validates schema
compatibility before it runs. A restore-simulation harness is in the tree.
Backup credential redaction landed in the same commit as audit retention
(`scripts/redact_backup_secrets.py`).
**Recommended:** Automated backup with rotation
- Daily incremental, weekly full, monthly archive
@@ -69,7 +133,7 @@ Baseline assessment of ShopDB operational burden across air-gapped production en
**Air-gap coordination:** 312 hrs/year (CRITICAL BURDEN)
- Current: Manual bundle creation, USB transfer, deploy in 3 environments
- Environments: dev PC (gitea) <-> work PC (GitHub) <-> VM (prod)
- Environments: dev PC (the internal git server) <-> work PC (GitHub) <-> VM (prod)
- Steps per deploy: 14 manual steps, 45-90 minutes each
- Frequency: 2-3 deploys/week = 104-156 deploys/year
@@ -111,6 +175,13 @@ logging:
```
- Savings: 52 hrs/year (fully automated, from manual cleanup)
**2026-09-18 update: STILL OPEN, and still the cheapest item on this page.**
`grep -c logging docker-compose.yml` returns 0: there is no `logging:` block on
any service. Three services do carry `restart: unless-stopped`
(`docker-compose.yml:24`, `:104`, `:120`) and three carry `healthcheck:` blocks
(`:38`, `:112`, `:219`), so the container lifecycle is managed; log size is not.
The 500MB/week growth figure was never sourced.
**Image rebuilds:** 52 hrs/year
- Current: Manual rebuild when dependencies change
- Frequency: ~20 rebuilds/year (Python deps, base image updates)
@@ -197,9 +268,57 @@ logging:
| Docker log disk fill | HIGH | High (3-6 mo) | Container crashes | Log rotation Q4 2026 |
| Deployment coordination errors | HIGH | Medium (2-3/year) | Failed deploys, downtime | Automation scripts implemented |
| Migration conflicts (ADR-008) | HIGH | Low | Plugin blocked | Migration-check skill |
| Agent certificate validation disabled | CRITICAL | Low | MITM attack | Per-endpoint cert pinning Q4 2026 |
| No login rate limiting | HIGH | Medium | Brute force attack | Rate limiting Q1 2027 |
| Unauthenticated collector endpoints | CRITICAL | N/A | FIXED 2026-09-11 | Device token auth |
| Agent certificate validation disabled | CORRECTED 2026-09-18 | n/a | n/a | Built. See note below |
| No login rate limiting | CORRECTED 2026-09-18 | n/a | n/a | Built. See note below |
| Unauthenticated collector endpoints | CORRECTED 2026-09-18 | n/a | n/a | Per-PC endpoint credentials, not device tokens. See note below |
| Docker log disk fill | HIGH | unmeasured | Container crashes | STILL OPEN, no `logging:` block in docker-compose.yml |
### Risk matrix corrections (2026-09-18)
**Agent certificate validation.** Not disabled. `AgentHttpFactory` is the only
`HttpClient` construction site in the agent, every channel runs
`ServerPinSet.Validate`, and `Validate` has no path returning true for an
unmatched pin (`agent/ShopDbEndpointAgent.Core/AgentHttpFactory.cs:291-331`). A
failed registry re-read returns null and keeps the existing pins rather than
silently unpinning (`:352-386`). The three SYSTEM PowerShell reporters refuse to
send at all when `ServerPins` is set and the check cannot be enabled
(`plugins/computers/client/Report-AssetToShopDB.ps1:228-247`, and the same block
in `Report-PrintersToShopDB.ps1` and `upgrade-shopdbagent.ps1:436-447`). The one
`ServerCertificateValidationCallback = {$true}` left in the tree is
`test-vm-connectivity.ps1:44`, a developer connectivity probe.
**Login rate limiting.** Built, and not "Q1 2027":
`shopdb/core/api/auth.py:35` `_login_ratelimited()`, keyed per host at `:52`,
applied at `:80`, with `tests/test_core/test_auth_ratelimit.py` over it. Note
the counter lives in per-process `SimpleCache`, so at four gunicorn workers the
effective budget is four times the configured maximum.
**Collector authentication.** The mechanism is a per-PC endpoint credential,
not a device token and not ADR-016. An enrolled PC sends
`X-Endpoint-Credential` (`plugins/endpoints/api/agentauth.py:1-18`); the server
stores only a sha256 hash plus a 12-character display prefix
(`plugins/endpoints/models/credential.py`); once a PC holds an active
credential the shared key is refused for it
(`plugins/endpoints/api/agentauth.py:129-134`). ADR-016's own mechanism -
`credentials.fetch`, `GET /api/collector/credentials`, Fernet at rest - is
**not built**: grepping `shopdb/` and `plugins/` for `credentials.fetch`,
`collector/credentials` and `Fernet` returns zero hits, matching ADR-016's own
"Implementation status, 2026-08-14". The rollout runbook is
`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`.
One boundary the runbook does not state: turning the shared key off does not
close the four legacy core collector routes. `/pc`
(`shopdb/core/api/collector.py:349`), `/apps` (`:387`), `/heartbeat` (`:459`)
and `/bulk` (`:494`) are decorated `@require_api_key`, which at `:154-164`
checks only `COLLECTOR_API_KEY` or a `collector.ingest`-scoped token and never
consults `agent_shared_key_refused`. Anything still holding the env shared key
can rewrite logged-in user, last boot time, serial number and installed-app
inventory for any hostname. Regenerating the key in the UI
(`plugins/endpoints/api/routes.py:1864-1884`) mints a new managed token and
clears the env fallback flag but does not change `COLLECTOR_API_KEY`, and
`_env_fallback_disabled(None)` returns False (`collector.py:60-74`), so the
legacy routes keep honouring the env key. Rotating `COLLECTOR_API_KEY` in
`.env` is the step that closes it.
## Automation Savings Summary
@@ -313,11 +432,23 @@ rsync -av $BACKUP_DIR/ /mnt/offsite-backups/shopdb/
### 5. check-naming-and-style.sh Optimization (DONE)
**Effort:** 2 hours
**Savings:** Script runtime 5x faster (10s -> 2s)
**Status:** Committed 2026-09-11 (commit 4943e182c3)
**Status:** In the tree and on by default. `USE_GIT_LS_FILES` defaults to `1`
(`scripts/check-naming-and-style.sh:24`) and the script takes the
`git ls-files -z` path whenever it is inside a work tree (`:26`, `:159`, `:183`,
`:218`), filtering out `migrations/versions/`, `frontend/dist/`, `staticdocs/`
and `.plugins-staged/`. Setting `USE_GIT_LS_FILES=0` falls back to recursive
grep with `--exclude-dir` for non-git checkouts. The whole script exits 0 on
gh-wj at 35d4fcd41.
**Impact:** 10-15 minutes saved per commit = ~26 hrs/year for team
**The 5x / 10s-to-2s claim is unsourced** and no timing was recorded for this
page. The savings line below it follows from it and is equally unsourced.
**Coverage note added 2026-09-18:** the root-anchored `^migrations/versions/`
filter does not exclude `plugins/*/migrations/versions/`, so plugin migrations
ARE scanned. They are not caught by the column rule, though: that rule greps
`^\s+[a-z]+_[a-z_]+\s*=\s*db\.Column` (`:236`), which only matches a model
attribute assignment, while migrations write `sa.Column('name', ...)`. The
column name is created in the one file the naming gate cannot read.
### 6. Pre-commit Hook Integration
@@ -349,17 +480,36 @@ pytest tests/ -x || exit 1
## Long-Term Improvements (Planned)
### 1. Agent Delta Updates (Q1 2027)
### 1. Agent Delta Updates - BUILT (2026-09-18 update)
**Current:** Agent POSTs full inventory (11KB) every 30 seconds
**Proposed:** POST only deltas (added/removed apps, IP changes) = ~500 bytes
**Original text:** "Agent POSTs full inventory (11KB) every 30 seconds.
Proposed: POST only deltas." Both halves are now wrong.
**Savings:**
- 99% reduction in network traffic (17GB/day -> 170MB/day for 500 endpoints)
- 99% reduction in database writes (only changed data)
- Performance: Collector API load reduced from 1.44M requests/day to ~14K writes/day
**The cadence was never 30 seconds.** The agent default is 900 seconds with up
to 60 seconds of jitter: `agent/ShopDbEndpointAgent/appsettings.json`
(`"cadenceseconds": 900`, `"jitterseconds": 60`) and
`agent/ShopDbEndpointAgent.Core/AgentConfig.cs:14` and `:31`. The 17GB/day and
1.44M-requests/day figures were derived from the wrong interval and should not
be quoted.
**Effort:** 40 hours (agent changes + server changes + testing)
**The delta is implemented, on the server side.** The agent still reports the
full installed-software list each cycle, but each entry carries a SHA-256
fingerprint of `displayname|displayversion|publisher`, and
`_replace_applications` (`plugins/endpoints/plugin.py:1122-1140`) compares
fingerprint sets instead of delete-all-insert-all: adds for new fingerprints,
deletes for departed ones, unchanged rows untouched (`:1277-1289`). The
docstring states the effect as roughly 100 database operations per check-in
down to about 2. It is guarded: fingerprint format validation, a delete-count
check that warns rather than aborting, a memory cap above 5000 apps, a legacy
fall-back for agents that send no fingerprints, and collision reconciliation
inside a savepoint (`:1318`, `:1387`). The unique `(assetid, appfingerprint)`
index was added NULL-filled by migration
`plugins/endpoints/migrations/versions/0020_*`, so index creation cannot fail on
a populated production table.
Remaining network-side work, if anyone wants it, is making the agent send only
the changed entries rather than the full list. The database win is already
taken.
### 2. Automated Staging Tests (Q4 2026)
@@ -369,14 +519,17 @@ pytest tests/ -x || exit 1
**Savings:** 50-100 hrs/year (from deployment coordination)
**Effort:** 20 hours (test suite + CI integration)
### 3. Per-Endpoint Certificate Pinning (Q4 2026)
### 3. Per-Endpoint Certificate Pinning - BUILT (2026-09-18 update)
**Current:** Certificate validation disabled (security risk)
**Proposed:** Each endpoint validates server cert against pinned thumbprint
**Savings:** Eliminates CRITICAL security risk
**Effort:** 40 hours (agent changes + deployment script + mass agent update)
**Status:** Planned (see Security-Audit-2026-09-11.md)
Pins are SPKI SHA-256, validated as 64 lowercase hex on the way in, enforced on
every agent channel, and owned by the registry rather than by `appsettings.json`
(`plugins/endpoints/client/Set-ShopDBServerPins.ps1` writes
`HKLM:\SOFTWARE\GE\ShopDB` `ServerPins`; the precedence is stated in
`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`, section "The registry
wins"). A CA-flagged certificate is pinned but deliberately NOT imported into
`LocalMachine\Root`, and the build warns the admin through an
`X-ShopDB-Build-Warning` header. See the risk-matrix correction above for the
enforcement call sites.
## Monitoring & Alerting
@@ -399,8 +552,11 @@ pytest tests/ -x || exit 1
## References
- Security Audit 2026-09-11: 4 critical + 6 high vulnerabilities, some fixed, some planned
- Architectural Findings 2026-09-11: 22 findings (6 critical, 6 high) from Gortex analysis
- Automation Skills: 13 skills created in `.claude/skills/` directory
- Deployment Documentation: `docs/deployment/` and `.claude/setup/` guides
- Architectural Findings 2026-09-11: re-checked 2026-09-18; most of the 22
findings are refuted or stale, see that page for the verdict on each
- ADR-Conformance-2026-09-18: which ADRs were checked in the 2026-09-18 review
and which hold
- Automation Skills: 13 skills created in the assistant skills directory
- Deployment Documentation: `docs/deployment/` and the assistant setup guides
- ADR-004: Deployment Topology (air-gapped architecture)
- ADR-008: Plugin Migration Ownership (linear chain requirement)

205
docs/wiki-drafts/README.md
View File

@@ -1,6 +1,18 @@
# Wiki Page Drafts - 2026-09-11
# Wiki Page Drafts - 2026-09-11, revised 2026-09-18
This directory contains drafted wiki pages based on work completed Q3 2026. Copy these to the GitHub wiki at https://github.com/ge-aero/shopdb-flask/wiki.
This directory contains drafted wiki pages based on work completed Q3 2026. Copy
these to the GitHub wiki at https://github.com/ge-aero/shopdb-flask/wiki.
**Revised 2026-09-18, branch gh-wj at 35d4fcd41.** A full review re-checked every
page here against the code. `Architectural-Findings-2026-09-11.md` was
substantially wrong and now carries a verdict and evidence per finding.
`Maintenance-Footprint-Baseline.md` is annotated where the tree has moved on.
`Security-Audit-2026-09-11.md` was refreshed with a verdict per finding and
thirteen new ones. `ADR-019-...md` had two internal contradictions corrected. A
new page, `ADR-Conformance-2026-09-18.md`, was added. The SECURITY.md draft below
had its ADR-016 section replaced: it described a decision ADR-016 never took and
banned the registry storage the reference site fleet depends on. Nothing here is ready to
paste without reading the 2026-09-18 notes inside each file first.
## New Pages to Create
@@ -19,6 +31,15 @@ This directory contains drafted wiki pages based on work completed Q3 2026. Copy
**Related commits:** fe09d2745e, d59d0c3194
**2026-09-18:** corrected against the shipped code. The "Backend: API Endpoint"
code block showed `@jwt_required(optional=True)` iterating
`get_enabled_plugin_instances()`; neither exists. It now shows
`shopdb/core/api/plugins.py:46-95` as it stands, gated on `settings.view`. The
Future Extensions section no longer calls the endpoint anonymous. Everything
else in the draft checked out, including the three constraints. Note
`docs/openapi.json` still documents this endpoint as anonymous, so publishing
the ADR without regenerating the spec leaves the two disagreeing.
---
### 2. Security Audit - September 11, 2026
@@ -54,12 +75,23 @@ This directory contains drafted wiki pages based on work completed Q3 2026. Copy
- Security/compliance: 78 hrs/year (scanning + updates)
**Automation:**
- 13 Claude Code skills created
- 13 assistant automation skills created (this figure checks out: `ls <assistant-config>/skills/*.md`)
- 147-205 hrs/year saved (from 936 baseline)
- 7 quick wins identified (4 hours - 6 hours effort each)
**Risk matrix:** 8 critical + 10 high operational risks
**2026-09-18:** annotated. Every hour figure and row count on that page is an
unsourced estimate and the page now says so. Four recommendations have since
been built and the page records what and where: audit-log retention (service,
CLI and setting, but default 0 and no scheduler), backups (restic with
5/4/6 retention, SMB and S3 destinations, restore simulation), agent delta
updates, and per-endpoint certificate pinning. Three risk-matrix rows were
stale and are corrected: certificate validation is not disabled, login rate
limiting exists, and the collector fix is per-PC endpoint credentials rather
than "device token auth". Docker log rotation is still genuinely missing:
`grep -c logging docker-compose.yml` returns 0.
---
### 4. Architectural Findings - September 11, 2026
@@ -83,6 +115,41 @@ This directory contains drafted wiki pages based on work completed Q3 2026. Copy
**Remediation timeline:** Q4 2026 - Q4 2027
**2026-09-18: do not publish this page as it stood.** Re-checking it against
gh-wj refuted most of it. There is no module-level cross-plugin import anywhere
(`grep -rn "^from plugins\." plugins/ --include=*.py` = 0 hits) and `machines`
does not import `computers` at all, so finding 1's cycle does not exist. No
plugin deep-imports `shopdb.core` (0 hits), so finding 2 is refuted outright. No
migration in the tree spells a column in snake_case, so finding 3's examples are
refuted. `MACHINE_TYPES`, `Bldg 300`, Vuex, a ServiceNow plugin and an
unhandled `requests.post` do not exist. The page has been rewritten to carry a
verdict and the evidence per finding, keeping the four that survive in narrowed
form.
---
### 5. ADR Conformance - September 18, 2026
**File:** `ADR-Conformance-2026-09-18.md`
**Wiki path:** `ADRs/ADR-Conformance-2026-09-18`
**Summary:** ADR-by-ADR result of the 2026-09-18 review of gh-wj at 35d4fcd41.
Fourteen ADRs hold with cited evidence; eight items are recorded as violations
or gaps, each with a file and line.
**Reads first, before the table:** the ADR set does not exist on this branch.
`ls docs/adr` fails on gh-wj; ADR-001 through ADR-018 live only on the other
lineage. Nothing here states the rules and nothing here enforces them except
`scripts/check-naming-and-style.sh`, which names exactly two ADRs (ADR-017 at
line 343, ADR-015 at line 395). A handful of tests cover ADR ground without
saying so, and `tests/test_api_inventory_parity.py` skips itself entirely
because `docs/api-inventory.json` is not tracked here.
**Violations recorded:** ADR-007 (CHANGELOG untouched across 20 commits),
ADR-008 (endpoints migration 0023 does DDL on core `assets`), ADR-016
(misdescribed in four files, and the per-PC programme has no design record),
ADR-018 (accepted, unimplemented, no status note), ADR-001/002 (raw SQL over
core tables in the computers CLI), ADR-015 (the scan does not cover `agent/`),
ADR-002 (two stale manifest floors).
---
## Pages to Update
@@ -189,7 +256,7 @@ Returns settings routes contributed by all enabled plugins via `get_settings_rou
ShopDB deploys across three air-gapped environments:
```
Dev PC (gitea) <--USB--> Work PC (GitHub) <--USB--> Production VM
Dev PC (the internal git server) <--USB--> Work PC (GitHub) <--USB--> Production VM
```
### Bundle Creation
@@ -255,18 +322,28 @@ USE_GIT_LS_FILES=0 bash scripts/check-naming-and-style.sh
Gate tests validate generated documentation and API consistency:
- `tests/test_openapi_spec.py` - OpenAPI spec completeness
- `tests/test_public_read_surface.py` - Public endpoint security
- `tests/test_authz.py` - Authorization decorators
- `tests/test_plugin_migrations.py` - Migration chain validation (ADR-008)
- `tests/test_core/test_public_read_surface.py` - public endpoint security
- `tests/test_core/test_authz.py` - authorization decorators
- `tests/test_plugin_migrations.py` - migration chain validation (ADR-008)
- `tests/test_core/test_plugin_settings_hubs.py` - plugin settings hubs (ADR-019)
**Important:** When you add/modify API endpoints, regenerate docs:
(Paths corrected 2026-09-18: the first draft put three of these at the top of
`tests/` rather than in `tests/test_core/`.)
**Important:** When you add or modify API endpoints, regenerate docs:
```bash
# Update docs/openapi.json to reflect new endpoints
# (Manual until docs/api-inventory.json exists)
git add docs/openapi.json
```
**Caveat for the wj branch (2026-09-18):** the parity gate cannot run here.
`tests/test_api_inventory_parity.py` skips itself when `docs/api-inventory.json`
is absent, and that file is not tracked on gh-wj, so `scripts/gen_openapi.py`
has no input either. `docs/openapi.json` on this branch is 377 paths at
`info.version` 0.12.0 and documents none of the 24 routes added since
2ac2e6d33. Regenerating it needs the inventory file from the other lineage.
**Enforcement:** `scripts/check-gates-ran.sh` verifies gates actually ran (not skipped)
**CI failure:** "Gate tests FAILED" means docs are stale or missing
@@ -286,9 +363,9 @@ git add docs/openapi.json
```bash
# These MUST be in .gitignore
.claude/settings.json
.claude/settings.local.json
.claude/mcp-servers.json
the assistant settings file
the assistant local-settings file
the assistant tool-config file
```
**Why:** Personal config files may contain:
@@ -297,7 +374,7 @@ git add docs/openapi.json
- API keys and collector tokens
- Personal authentication tokens
**Incident:** 2026-09-11 - Passwords exposed in `.claude/mcp-servers.json` committed to GitHub
**Incident:** 2026-09-11 - Passwords exposed in the assistant tool-config file committed to GitHub
- Fixed: Removed from git tracking, added to .gitignore
- Impact: Dev environment credentials only (not production)
@@ -313,14 +390,56 @@ Pre-commit hook scans for these patterns:
**If blocked:** Replace hardcoded credentials with environment variables or config files
## ADR-016: Credential Delivery
## How fleet credentials actually work (corrected 2026-09-18)
Production credentials delivered via:
- Device tokens (for endpoints/agents)
- Personal Access Tokens (for users)
- Environment variables (for services)
The paragraph that stood here attributed a decision to ADR-016 that ADR-016
never took. It is replaced with what the code does. Do not reinstate it.
**Never:** Hardcoded in source, registry keys, git history
**What was wrong.** It said ADR-016 delivers credentials by "device tokens" and
that credentials must **never** live in registry keys. ADR-016 says neither.
Grepping every ADR for device tokens returns nothing. ADR-016's Decision is a
fleet credential store with Fernet at rest, a targeting and precedence resolver,
and `GET /api/collector/credentials` behind a dedicated `credentials.fetch`
scope - and its "Alternatives rejected" section calls registry-only provisioning
"a worthwhile FIRST STEP" and "Recommended as tier one regardless". The ban also
contradicted shipped code: `plugins/computers/client/Report-AssetToShopDB.ps1:367-374`
reads the collector key from `HKLM:\SOFTWARE\GE\ShopDB`, and
`plugins/endpoints/client/Set-ShopDBServerPins.ps1:4-20` writes the server trust
pins to the same key by design. Publishing the old text would have told the next
person to delete values the fleet depends on.
**What is actually built, as of gh-wj 35d4fcd41:**
- **Per-PC endpoint credentials.** An enrolled PC authenticates with
`X-Endpoint-Credential` (`plugins/endpoints/api/agentauth.py`). The server
stores only a sha256 hash plus a 12-character display prefix
(`plugins/endpoints/models/credential.py`); the secret exists once, in the
201 response to `/enroll`. On the endpoint it is a DPAPI LocalMachine blob
under `%ProgramData%\ShopDB\EndpointAgent\credential`, in a directory
ACL'd to SYSTEM and Administrators only.
- **Bootstrap tokens** for the one-line install: hashed at rest, expiry
mandatory and bounded to 1-168 hours, uses bounded to 1-500, revocable. The
package they serve never carries the collector key.
- **The fleet shared key** is still live authority for any PC that is not yet
enrolled, and turning it off is a fleet-wide switch, not a per-PC one. The
rollout sequence is `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`.
- **ADR-016's own mechanism is NOT built.** `credentials.fetch`,
`collector/credentials` and `Fernet` return zero hits across `shopdb/` and
`plugins/`, matching ADR-016's own "Implementation status, 2026-08-14".
**Two things that are genuinely never acceptable**, and this part of the
original text was right: a credential hardcoded in source, and a credential in
git history.
**Registry storage is not itself the problem; an unprotected key is.**
`install.ps1` writes the shared collector key to `HKLM\SOFTWARE\GE\ShopDB`
with no ACL (`plugins/endpoints/api/siteagent.py:328`, helper at `:394-411`), so
it inherits read for BUILTIN\Users and any logged-on user can read it. The repo
states that consequence itself at
`.github/GE-ENFORCE-COLLECTOR-FIX.html:377-380`. The rollout plans to clear that
value at cutover.
**See also:** ADR-Conformance-2026-09-18.md, violation 3.
```
---
@@ -332,7 +451,7 @@ Production credentials delivered via:
```markdown
# Automation Skills
Claude Code skills created Q3 2026 for common workflows. Save 147-205 hrs/year from 936 hrs baseline.
Assistant automation skills created Q3 2026 for common workflows. Save 147-205 hrs/year from 936 hrs baseline.
## Available Skills
@@ -414,11 +533,15 @@ Note: Realistic savings are 147-205 hrs/year due to task overlap.
## Summary
**4 new wiki pages drafted:**
1. ADR-019: Plugin Dynamic Settings Routes (architectural decision)
2. Security Audit 2026-09-11 (4 critical + 6 high findings)
3. Maintenance Footprint Baseline (936 hrs/year burden, automation opportunities)
4. Architectural Findings 2026-09-11 (22 findings from Gortex analysis)
**5 new wiki pages drafted:**
1. ADR-019: Plugin Dynamic Settings Routes (architectural decision) - corrected 2026-09-18
2. Security Audit 2026-09-11 (4 critical + 6 high findings) - refreshed 2026-09-18
against the same branch; every 2026-09-11 finding now carries a verdict with a
file and line, and findings 19 and up were added on 2026-09-18
3. Maintenance Footprint Baseline (936 hrs/year burden, automation opportunities) - annotated 2026-09-18
4. Architectural Findings 2026-09-11 - rewritten 2026-09-18 with a verdict per finding
5. ADR Conformance 2026-09-18 (new) - which ADRs hold, which are violated, and the fact
that the ADR set is not in this repository
**7 existing pages need updates:**
1. PLUGIN-DEVELOPMENT.md (add get_settings_routes() docs)
@@ -426,26 +549,44 @@ Note: Realistic savings are 147-205 hrs/year due to task overlap.
3. DEPLOYMENT.md (add air-gap workflow)
4. CONTRIBUTING.md (add optimized pre-commit checks)
5. TESTING.md (add gate tests explanation)
6. SECURITY.md (create new page for credential security)
6. SECURITY.md (create new page for credential security) - its ADR-016 section was
replaced on 2026-09-18; publish the replacement, not what was there
7. Automation-Skills.md (create new page for skill catalog)
**Total effort to publish:** ~2 hours (copy-paste to wiki, adjust formatting, add navigation links)
**Total effort to publish:** the "~2 hours, copy-paste" estimate no longer
holds. Three of the five pages needed correcting against the code before they
were fit to publish, and the Security Audit page has not been through that pass
yet.
---
## Next Steps
1. Review drafts for accuracy (have I missed anything?)
1. Read the 2026-09-18 notes inside each page before pasting anything. All five
pages in this folder have now been checked against gh-wj at 35d4fcd41.
2. Copy to GitHub wiki at https://github.com/ge-aero/shopdb-flask/wiki
3. Update wiki navigation/sidebar to include new pages
4. Link related pages (ADRs reference each other, operations pages link to security audit, etc.)
5. Notify team of new documentation
3. Update wiki navigation/sidebar to include the new pages
4. Link related pages. In particular, `Security-Audit-2026-09-11.md` finding 31
and `ADR-Conformance-2026-09-18.md` violation 3 are the same ADR-016
misattribution seen from two angles, and the SECURITY.md draft above is the
copy that was fixed.
5. Notify the team
**Three corrections that live outside this folder** and were therefore NOT made:
- `CLAUDE.md:3` and `:9` say PostgreSQL 15+; `docker-compose.yml:20` runs
`mysql:8.4` and `.env.example:29` is a `mysql+pymysql://` URL.
- `CLAUDE.md:31` says columns are snake_case; `CONTRIBUTING.md:17` says
lowercase concatenated with no underscores, and that is what
`scripts/check-naming-and-style.sh:236` enforces.
- `CLAUDE.md:119` and the assistant quick-reference file, lines 14 and 135 repeat the
ADR-016 device-token misattribution.
---
## Related Memory Files
These wiki pages are based on memory files in `.claude/memory/`:
These wiki pages are based on memory files in the assistant memory directory:
- `security_audit_2026_09_11.md`
- `maintenance_footprint_baseline.md`
- `architectural_analysis_gortex_2026_09_11.md`

605
docs/wiki-drafts/Security-Audit-2026-09-11.md
View File

@@ -1,393 +1,406 @@
# Security Audit - September 11, 2026
# Security Audit - September 11, 2026, refreshed September 18, 2026
**Scope:** Endpoints plugin, C# agent, system tray application
**Severity:** 4 critical, 6 high, 8 medium findings
**Status:** 3 critical fixed, 1 critical planned, 6 high in progress
**Scope:** Endpoints plugin, C# agent, system tray application, and (new on 2026-09-18) the USB and printed-parts kiosks
**Branch reviewed:** `gh-wj` at `35d4fcd41` (2026-09-16). Range re-read for this refresh: `2ac2e6d33..gh-wj`, 38 commits, 2026-09-11 to 2026-09-16
**Severity, 2026-09-11 set:** 4 critical, 6 high, 8 medium
**Severity, added 2026-09-18:** 1 high, 4 medium, 9 low
**Status:** see the remediation summary. Every 2026-09-11 finding below now carries a verdict read off the code on this branch, with a file and line.
This page covers two dates. The findings numbered 1 to 18 were raised on 2026-09-11 and their status lines were rewritten on 2026-09-18. The findings numbered 19 and up were raised on 2026-09-18 and did not exist on the earlier page.
## What changed since 2026-09-11
Two programmes landed in this range and both move the security boundary.
**Per-PC agent credentials.** The agent no longer has to authenticate with one fleet-wide shared key. Each PC can hold its own secret: `plugins/endpoints/models/credential.py:36` stores only a sha256 hash and a 12-character display prefix, `plugins/endpoints/api/agentauth.py` makes `X-Endpoint-Credential` the agent auth header, and `plugins/endpoints/api/enrollment.py:289` adds an unauthenticated `/enroll` that a PC uses to ask for one. On the endpoint the secret is a DPAPI LocalMachine blob written through a SYSTEM and Administrators only DACL (`agent/ShopDbEndpointAgent.Core/AgentCredentials.cs:203`, `agent/ShopDbEndpointAgent.Core/ProtectedFileSystem.cs:219`). Once a PC holds an active credential the server refuses the shared key for it (`plugins/endpoints/api/agentauth.py:129`). The operator sequence for the cutover is `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`.
**Kiosk network admission.** Commit `da91a2915` added a second way for an unauthenticated kiosk caller to be admitted: a source address inside a configured CIDR list, alongside the existing per-device token. Both the USB kiosk (`plugins/usb/api/routes.py:798`) and the printed-parts kiosk (`plugins/printedparts/api/routes.py:551`) call `request_ip_allowlisted`, backed by the new `shopdb/utils/ipallowlist.py`. Both lists default to empty, which matches nothing (`shopdb/utils/ipallowlist.py:44`), and neither can override the kiosk's own enabled flag. This widens an unauthenticated write surface from a handful of provisioned screens to every PC on the named networks, so the correctness of the client IP the server sees now matters more than it did.
Also in range: the agent's server certificate pinning was completed, the tray stopped holding secrets, a one-line bootstrap install was added, and the 2026-09-12 audit reports were deliberately removed from the product tree (`8eee6f19d`) because they described weaknesses in detail on the branch that deploys to production. This page keeps that spirit: findings are located precisely enough to fix, and no exploit recipe is written out.
## Executive Summary
Comprehensive security review of the endpoint management subsystem (plugin + C# agent + tray app) identified critical authentication gaps, credential exposure risks, and performance bottlenecks that expose the system to unauthorized access and operational failure.
The headline of the 2026-09-11 page, unauthenticated inventory writes, is closed on the ingest routes that actually exist on this branch: `POST /api/collector/<pluginname>` requires either a per-PC credential or a collector key, and neither branch of `shopdb/core/api/collector.py:251` reaches the plugin without one.
**Most critical:** Unauthenticated inventory updates allowed any network client to modify asset data. **Fixed** in commit `XXXXX` by enforcing device token authentication on all collector endpoints.
The 2026-09-11 page's second headline, the collector key sitting unprotected in the registry, was still open when this review ran and was **fixed the same day**: the generated installer now restricts `HKLM\SOFTWARE\GE\ShopDB` to SYSTEM and Administrators in both views (finding 19). The key itself stays on purpose until every consumer is on a per-PC credential, and bays installed before this change keep the old DACL until they are reinstalled or upgraded. The per-PC credential work in this range is the exit from the shared key entirely, but only for PCs that have enrolled.
**Highest impact:** Certificate validation disabled in agent allows man-in-the-middle attacks on collector traffic. **Planned** for Q4 2026 with per-endpoint certificate authentication.
The third headline, no certificate validation in the agent, is **fixed** and was fixed properly, not worked around.
## Critical Findings (P0)
### 1. Unauthenticated Inventory Updates [FIXED]
### 1. Unauthenticated Inventory Updates [CLOSED, ON THE ROUTES THAT EXIST]
**Risk:** Any client on the network could POST to `/api/endpoints/inventory` and modify asset records without authentication.
**2026-09-18 verdict:** closed, but not by the mechanism the 2026-09-11 page described, and the evidence it published cannot be reproduced.
**Impact:**
- Attacker could inject false inventory data
- Legitimate collector reports could be overwritten
- Audit trail shows "unknown" as modifier
- Data integrity compromised
None of the four paths named in the original fix (`/api/endpoints/inventory`, `/api/endpoints/heartbeat`, `/api/endpoints/{id}/apps`, `/api/endpoints/{id}/snapshots`) exists on this branch. Endpoint ingest is the ADR-006 generic collector: `@collector_bp.route('/<pluginname>', methods=['POST'])` at `shopdb/core/api/collector.py:229`. That route admits a caller in exactly two ways and neither is anonymous:
**Root cause:** Missing `@requires_device_token` decorator on inventory endpoints.
- a per-PC secret in `X-Endpoint-Credential`, resolved to an asset id or rejected 401 `CREDENTIAL_INVALID` (`shopdb/core/api/collector.py:255-264`)
- otherwise `_check_collector_auth`, which takes a per-plugin or shared collector key, or a `collector.ingest` scoped managed token, from the header only (`shopdb/core/api/collector.py:154`)
**Fix:** Commit `XXXXX` added device token authentication to:
- `/api/endpoints/inventory` (POST)
- `/api/endpoints/heartbeat` (POST)
- `/api/endpoints/{id}/apps` (POST)
- `/api/endpoints/{id}/snapshots` (POST)
A credential caller also cannot write for another PC: the three schema-declaring plugins raise `PermissionError` on an asset mismatch and the dispatcher maps it to 403 `ASSET_MISMATCH` (`shopdb/core/api/collector.py:294`, `plugins/computers/plugin.py:729`, `plugins/printers/plugin.py:556`, `plugins/endpoints/plugin.py:1422`).
**Verification:**
```bash
# Before: Succeeds without auth
curl -X POST http://localhost:5001/api/endpoints/inventory -d '{...}'
**What is still open:** four legacy collector routes take the shared key alone and are not covered by the new refusal. See finding 20.
# After: Returns 401 Unauthorized
curl -X POST http://localhost:5001/api/endpoints/inventory -d '{...}'
# {"status":"error","data":{"error":{"code":"UNAUTHORIZED","message":"Device token required"}}}
**Residual risk, restated:** the original text said device tokens sit in agent config files protected by Windows ACLs. On this branch the per-PC credential is not in a config file at all; it is a DPAPI LocalMachine blob in `%ProgramData%\ShopDB\EndpointAgent\credential` written through a protected DACL (`agent/ShopDbEndpointAgent.Core/AgentCredentials.cs:207-215`). The fleet shared key is the one that is still readable, and that is finding 19.
# With token: Succeeds
curl -X POST http://localhost:5001/api/endpoints/inventory \
-H "X-API-Key: shopdb_device_abc123..." -d '{...}'
```
### 2. Deployment Credential Exposure in Registry [STILL OPEN]
**Residual risk:** Device tokens stored in agent config files on endpoints. An attacker with file access can read and reuse the token. Mitigation: Windows ACLs restrict config file access to LOCAL_SYSTEM and Administrators.
**2026-09-18 verdict:** still open, and the described fix did not ship. Promoted to the highest open severity and restated in full as finding 19.
### 2. Deployment Credential Exposure in Registry [MITIGATED]
The 2026-09-11 page said the deployment script would store the token in an ACL'd config file, remove the registry key, and have the agent read the file rather than the registry. On this branch:
**Risk:** Deployment scripts wrote collector API key to HKLM registry in plaintext, readable by any user via `reg query`.
- the generated installer still writes the key to the registry with no ACL: `plugins/endpoints/api/siteagent.py:328`, through a helper that does only `CreateSubKey` plus `SetValue` in both views (`plugins/endpoints/api/siteagent.py:394-411`). There is no `SetAccessControl` or `RegistrySecurity` anywhere in that file.
- the collection script still reads the key back out of the registry: `plugins/computers/client/Report-AssetToShopDB.ps1:367-375`, both the native and the WOW6432Node view.
- the registry path in the original write-up (`HKLM\SOFTWARE\ShopDB\Agent`, value `CollectorApiKey`) never matched this product. The real location is `HKLM\SOFTWARE\GE\ShopDB`, value `CollectorKey`.
**Impact:**
- Any logged-in user (including non-admin) could read the API key
- Key valid for all collector operations (inventory, heartbeat, app updates)
- No expiration or rotation mechanism
**What did change, and it matters:** there is now a keyless build. When the package is built without the shared key the installer emits no `-CollectorKey` parameter and no key write at all (`plugins/endpoints/api/siteagent.py:331-334`), and the PC enrolls for its own credential instead. The rollout runbook plans to clear `CollectorKey` from both registry views at cutover (`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:351-361`). So the exposure ends when the fleet finishes enrolling, not before.
**Evidence:**
```powershell
PS> reg query "HKLM\SOFTWARE\ShopDB\Agent" /v CollectorApiKey
CollectorApiKey REG_SZ shopdb_pat_b4a3fe2a01abc6bc1cb685280e65a8c35d52ec43
```
**Note on the registry generally:** the same key is also where the agent's server trust pins live, by design (`plugins/endpoints/client/Set-ShopDBServerPins.ps1:4-20`, precedence stated at `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:106-111`). Pins are trust anchors, not secrets. Do not read "get the key out of the registry" as "the registry must hold nothing".
**Fix:** Deployment script updated to:
1. Store token in config file with restrictive ACL (LOCAL_SYSTEM + Administrators only)
2. Remove registry key after migration
3. Agent reads from config file, not registry
### 3. No Certificate Validation in Agent [FIXED]
**Status:** Mitigated on new deployments. Legacy endpoints still have registry key. Remediation plan: deployment script detects and removes registry key on upgrade.
**2026-09-18 verdict:** fixed, and fixed by pinning rather than by re-enabling default validation and hoping.
**Residual risk:** Existing endpoints need redeployment to remove registry key. Scheduled for Q4 2026 mass agent update.
`agent/ShopDbEndpointAgent.Core/AgentHttpFactory.cs` is the only place the agent constructs an HTTP client, and every channel goes through `pins.Validate(cert, errors, channel)` (`AgentHttpFactory.cs:52`). `Validate` (`AgentHttpFactory.cs:291`) has no path that returns true for an unmatched pin: unpinned falls back to `SslPolicyErrors.None` (`AgentHttpFactory.cs:302`), pinned requires an SPKI sha256 or legacy thumbprint match (`AgentHttpFactory.cs:314`). A failed registry re-read returns null and keeps the existing pins rather than silently unpinning.
### 3. No Certificate Validation in Agent [PLANNED]
The three SYSTEM PowerShell reporters refuse to send at all when pins are configured but the check cannot be enabled, including under pwsh (`plugins/computers/client/Report-AssetToShopDB.ps1:228-247`, and the same block in `plugins/printers/client/Report-PrintersToShopDB.ps1:125` and `plugins/endpoints/api/templates/upgrade-shopdbagent.ps1:110`).
**Risk:** Agent's HTTP client disables SSL certificate validation:
The blanket accept-any-certificate callback the original finding quoted survives in exactly one place, a developer connectivity probe that is not shipped to endpoints: `test-vm-connectivity.ps1:44`.
```csharp
ServicePointManager.ServerCertificateValidationCallback =
(sender, cert, chain, sslPolicyErrors) => true; // ACCEPTS ANY CERT
```
### 4. Hardcoded Admin Credentials [PARTLY FIXED]
**Impact:**
- Man-in-the-middle attack possible on collector traffic
- Attacker can intercept inventory data (hostnames, IPs, installed apps, usernames)
- Attacker can inject false commands or updates
**2026-09-18 verdict:** partly fixed. The production path is blocked; the credential is still in the source tree.
**Why it exists:** Corporate proxy with self-signed certificate breaks standard validation. Quick fix was to disable validation entirely.
`flask seed test-user` still creates `admin` / `admin123` (`shopdb/cli/__init__.py:636`), and the string appears four times in that file. What changed is that the command now refuses to run outside DEBUG or TESTING and says why (`shopdb/cli/__init__.py:619-623`), pointing the operator at `flask seed admin`, which generates a password.
**Proper fix (PLANNED):** Per-endpoint certificate pinning
1. Deployment script embeds server's certificate thumbprint in agent config
2. Agent validates that presented cert matches pinned thumbprint
3. Pinned cert stored in protected config file (LOCAL_SYSTEM ACL)
4. Rotation: new deployment updates pinned thumbprint
The file the original finding quoted, `tests/fixtures/users.py`, does not exist on this branch, so the described fix to it cannot be confirmed.
**Implementation plan:** ADR-XXX Per-Endpoint Certificate Authentication (see Security-Fix-Implementation.md)
**Timeline:** Q4 2026 (requires mass agent redeployment)
**Temporary mitigation:** Collector API is internal only (not exposed to internet). Attacker must be on corporate network with proxy access. Still unacceptable for long-term.
### 4. Hardcoded Admin Credentials [REMOVED]
**Risk:** Test fixtures and seed scripts contained hardcoded passwords:
```python
# tests/fixtures/users.py
admin_user = User(
username='admin',
password='admin123', # HARDCODED
role='admin'
)
```
**Impact:**
- Default credentials usable if seed script runs in production
- Password visible in source code and git history
- No password complexity or rotation
**Fix:** Commit `XXXXX` removed hardcoded passwords:
1. Test fixtures use `os.environ.get('TEST_PASSWORD', generate_random())`
2. Seed scripts prompt for admin password on first run
3. Password stored hashed (bcrypt) in database, not in code
4. Deployment docs updated to mandate password change on first login
**Verification:** Grep for hardcoded passwords returns zero hits:
```bash
grep -r "password.*=.*['\"][^'\"]\{8,\}" --include="*.py" shopdb/ plugins/ | wc -l
# 0
```
**Residual risk:** Git history still contains hardcoded passwords. Requires git-filter-repo to fully purge. Scheduled for next major release (v1.0).
**Residual risk:** unchanged. A dev-only well-known credential still ships in the tree, and git history still holds the older forms.
## High Priority Findings (P1)
### 5. Collector Key Exposure in Committed Files
### 5. Collector Key Exposure in Committed Files [FIXED, ONE CLAIM CORRECTED]
**Risk:** Collector API key (`shopdb_pat_...`) committed to `.claude/skills/*.md` files and pushed to GitHub.
**2026-09-18 verdict:** the redaction holds. `git ls-files | xargs grep -l "shopdb_pat_"` returns nothing on this branch, and the assistant settings file, the assistant local-settings file and the assistant tool-config file are ignored (`.gitignore:174-176`).
**Impact:**
- Key visible in public/private GitHub repository
- Valid until manually rotated
- Grants full collector access (inventory, heartbeat, app updates)
**Correction:** the original page's prevention line is not true of this branch. The committed pre-commit hook runs the naming and style gate only (`.githooks/pre-commit:5`), and `scripts/check-naming-and-style.sh` has no credential pattern check at all. Its eleven checks are non-ASCII, banned shorthand, three snake_case rules, frontend API params, plugin frontend imports, the shared code-library module, ADR-017 map levels and ADR-015 site literals (`scripts/check-naming-and-style.sh:151-395`). Nothing blocks a commit that reintroduces a key.
**Evidence:**
```bash
git log -S "shopdb_pat_b4a3fe2a" --oneline
ce27cb5eef Add 9 comprehensive automation skills
```
### 6. Password Exposure in MCP Config [FIXED AS FAR AS THIS BRANCH GOES]
**Fix:** Commit `1e8c6a513a` redacted key and added to .gitignore:
1. Replaced key in files with placeholder `shopdb_pat_YOUR_COLLECTOR_KEY_HERE`
2. Added `.claude/settings*.json` and `.claude/mcp-servers.json` to .gitignore
3. Removed files from git tracking (`git rm --cached`)
**2026-09-18 verdict:** the assistant tool-config file is ignored (`.gitignore:176`) and is not in `git ls-files`. The accepted risk recorded in 2026-09-11, that the exposed dev passwords were not rotated, is unchanged and still accepted.
**Key rotation:** User indicated passwords/keys do NOT need rotation despite exposure ("won't need to"). Accepted risk: exposed key remains valid.
### 7. Missing Rate Limiting on Login [FIXED]
**Prevention:** Pre-commit hook scans for `shopdb_pat_` pattern and blocks commit if found.
**2026-09-18 verdict:** fixed, in two layers.
### 6. Password Exposure in MCP Config
- Per-IP fixed-window limiter: `_login_ratelimited` at `shopdb/core/api/auth.py:35`, applied at `shopdb/core/api/auth.py:80`, defaults 30 hits per 300 seconds.
- Per-account lockout: after `MAX_FAILED_LOGINS` consecutive bad passwords the account is locked (`shopdb/core/api/auth.py:113-118`), and a locked account is rejected before the password is checked so a valid password cannot reset the clock mid-lockout (`shopdb/core/api/auth.py:100`).
**Risk:** `.claude/mcp-servers.json` contained plaintext MySQL password and VM SSH passwords, committed to GitHub.
**Known limit, stated in the code:** under the default SimpleCache the limiter's counter is per-process, so with N gunicorn workers the effective budget is N times the configured maximum (`shopdb/core/api/auth.py:39-42`). The per-account lockout is the control that does not have that weakness. Both are keyed on the client address, which is why the client-IP plumbing in finding 23 matters.
**Impact:**
- MySQL dev database password: `shopdb-dev-password`
- VM SSH password for SupportUser: `vos313`
- VM SSH password for shopfloor account: `vos313`
- Passwords visible in git history
### 8. SQL Injection in Asset Search [STILL A FALSE POSITIVE]
**Fix:** Commit `1e8c6a513a` removed from git tracking and added to .gitignore.
**2026-09-18 verdict:** re-checked, still no injection.
**Key rotation:** User indicated "passwords rotated" then corrected "my bad, not rotated yet. but won't need to". Accepted risk: exposed passwords remain valid.
The raw SQL that exists is parameterised. The USB external-database path builds a WHERE clause out of fixed fragments and passes every value as a parameter (`plugins/usb/api/routes.py:277-299`); the only interpolated text is the fragment list itself. `plugins/employees/api/routes.py:113` binds through `%s`.
**Scope:** Dev environment only. Production uses different credentials not stored in repo.
**One related note that is not injection:** `plugins/computers/plugin.py:2080` runs a raw `db.text` query joining the core `assets`, `assetrelationships` and `assettypes` tables inside the `flask computers retype` CLI. No user input reaches it, so it is not a security finding. It is hidden coupling to the core schema, which is an architecture item, not one for this page.
### 7. Missing Rate Limiting on Login
### 9. Session Token Storage in LocalStorage [UNCHANGED]
**Risk:** No rate limiting or account lockout on failed login attempts.
**2026-09-18 verdict:** unchanged. The JWT and the refresh token are still written to and read from localStorage (`frontend/src/stores/auth.js:6-7`, `frontend/src/stores/auth.js:43-44`). No HttpOnly cookie path was added in this range.
**Impact:**
- Brute force attack on login endpoint
- No delay or throttling after failed attempts
- Audit log fills with failed attempts but no automated response
Partial compensating control added since: the app now sets `X-Content-Type-Options`, `X-Frame-Options`, `Referrer-Policy` and a `frame-ancestors 'self'` CSP on every response (`shopdb/__init__.py:334-341`). That is not a script-src policy and the code says so deliberately.
**Evidence:**
```bash
# Can attempt 1000 logins/second with no throttling
for i in {1..1000}; do
curl -X POST http://localhost:5001/api/auth/login \
-d '{"username":"admin","password":"wrong"}' &
done
```
### 10. Audit Log Unbounded Growth [MECHANISM BUILT, OFF BY DEFAULT]
**Mitigation options:**
1. Flask-Limiter: `@limiter.limit("5 per minute")` on login endpoint
2. Account lockout: Lock account after 5 failed attempts for 15 minutes
3. CAPTCHA after 3 failed attempts
**2026-09-18 verdict:** the tooling now exists; it does nothing unless an operator turns it on and runs it.
**Status:** Planned for Q1 2027. Not blocking for air-gapped production (internal network only).
- Service: `shopdb/core/services/auditretention.py`
- Admin API: `GET /api/auditlogs/retention` (`shopdb/core/api/auditlogs.py:195`) and the purge route at `shopdb/core/api/auditlogs.py:211`
- CLI: `shopdb/cli/__init__.py:1894` and `shopdb/cli/__init__.py:1918`
### 8. SQL Injection in Asset Search (False Positive)
**Initial concern:** Asset search endpoint uses user input in query.
**Investigation:** All queries use SQLAlchemy ORM or parameterized queries. String concatenation not found.
**Verification:**
```bash
grep -rn "execute.*+\|execute.*format\|execute.*%" \
--include="*.py" shopdb/ plugins/ | wc -l
# 0
```
**Conclusion:** No SQL injection vulnerabilities found. Marked false positive.
### 9. Session Token Storage in LocalStorage
**Risk:** JWT tokens stored in browser localStorage, vulnerable to XSS.
**Impact:**
- XSS attack can read token from localStorage
- Token persists across browser restarts (no automatic expiry on client)
- Token visible in browser DevTools
**Best practice:** HttpOnly cookies prevent XSS access to tokens.
**Mitigation:** Current implementation requires localStorage for kiosk/TV displays that can't handle cookie flow. Acceptable for internal network deployment.
**Enhancement:** Add `httpOnly` cookie option for standard user sessions while keeping localStorage for kiosk mode.
**Status:** Planned for v1.0 (requires auth refactor).
### 10. Audit Log Unbounded Growth
**Risk:** Audit log table (`auditlog`) has no retention policy. Grows unbounded.
**Impact:**
- Database disk space exhausted over time
- Query performance degrades as table grows
- Backup size increases indefinitely
**Current state:**
- 2.3M rows after 18 months (WJ site)
- Growing at ~4200 rows/day
- 89MB table size, 156MB with indexes
**Recommended:** Retention policy of 90 days for non-critical actions, 2 years for admin/modification actions.
**Status:** Planned for Q4 2026. See MAINTENANCE-DEPLOYMENT-CHECKLIST.md for implementation.
`configureddays()` returns 0 when `audit_retentiondays` is unset, and 0 means keep forever (`shopdb/core/services/auditretention.py:123-133`). There is no scheduler in that module, so nothing purges on its own. The row and size figures quoted in 2026-09-11 come from the reference site database and cannot be confirmed from the repository.
## Medium Priority Findings (P2)
### 11. Endpoint Event Accumulation
### 11. Endpoint Event Accumulation [STILL OPEN]
**Risk:** `endpointevents` table accumulates every app launch, process change, USB connect. No retention.
**2026-09-18 verdict:** still open. `EndpointEvent` is `endpointevents` (`plugins/endpoints/models/endpoint.py:333`) and there is no retention, purge or prune code anywhere under `plugins/endpoints/`. The audit-log retention service added in this range covers `auditlog` only. The row and size figures quoted in 2026-09-11 are site measurements and are not verifiable here.
**Impact:**
- 450K rows after 18 months
- 127MB table size
- High-churn data (process events every 30 seconds)
### 12. Missing HTTPS Enforcement in Dev [CHANGED; THE STATED MITIGATION IS GONE]
**Recommended:** Retention of 30 days for events, 90 days for snapshots.
**2026-09-18 verdict:** the specific mitigation the page described no longer exists, and that is an improvement. `NODE_TLS_REJECT_UNAUTHORIZED` appears nowhere in the tree. The Vite dev proxy target is now an environment variable defaulting to `http://localhost:5001`, and TLS verification for an https target is off unless `VITE_DEV_SECURE=true` (`frontend/vite.config.js:5-18`). Dev only; production still terminates TLS in front of the app.
### 12. Missing HTTPS Enforcement in Dev
### 13. Docker Log Rotation Not Configured [STILL OPEN]
**Risk:** Vite dev server runs HTTP, not HTTPS. Cookies sent without Secure flag.
**2026-09-18 verdict:** still open. There is no `logging:` block anywhere in `docker-compose.yml`. Both hops log to stdout (`nginx.conf:36`, `docker-compose.yml:213-214`), so the host's Docker log driver defaults apply unchanged. This now has a second consequence: see finding 25.
**Impact:** Dev environment only. Production uses HTTPS via gunicorn + nginx.
### 14-18. Additional Medium Findings [PARTLY ADDRESSED, SOURCE NOW MISSING]
**Mitigation:** `start-dev.cmd` sets `NODE_TLS_REJECT_UNAUTHORIZED=0` for Vite proxy.
**2026-09-18 verdict:** the file this section pointed at is gone. the assistant memory directory does not exist on this branch; commit `8eee6f19d` moved the 2026-09-12 audit reports and scan output out of the product tree on purpose. The detail behind findings 14 to 18 is therefore not readable from here.
### 13. Docker Log Rotation Not Configured
Of the five items summarised in the original bullet list, two can be answered from the code:
**Risk:** Docker container logs grow unbounded. Can fill host disk.
- **No content security policy headers:** partly addressed. `shopdb/__init__.py:334-341` sets a security header block including `Content-Security-Policy: frame-ancestors 'self'`, and uploads carry a much stricter `default-src 'none'; sandbox` (`shopdb/utils/uploads.py:13`, applied at `shopdb/utils/uploads.py:35`). A real script-src policy is still not set, deliberately.
- **Directory listing enabled in nginx:** no `autoindex` directive appears in `nginx.conf`, and nginx defaults to off.
**Mitigation:** Configure log rotation in docker-compose.yml:
```yaml
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
```
The remaining three, input validation gaps, world-readable backup files and old TLS versions, cannot be confirmed or refuted without the removed source document. Treat them as unverified rather than as fixed.
**Status:** Planned for Q4 2026.
## New findings, 2026-09-18
### 14-18. Additional Medium Findings
### High
See full audit report in `.claude/memory/security_audit_2026_09_11.md` for:
- Missing input validation on some endpoints
- No content security policy headers
- Directory listing enabled in nginx (dev only)
- Backup files world-readable (dev only)
- Old TLS versions enabled (corporate proxy requirement)
### 19. The fleet shared key is written to the registry with no ACL (FIXED 2026-09-18; the key itself is transitional by design)
**Risk:** for a shared-key build, the generated `install.ps1` writes the collector key to `HKLM\SOFTWARE\GE\ShopDB\CollectorKey` (`plugins/endpoints/api/siteagent.py:328`) through a helper that does only `CreateSubKey` plus `SetValue` in both registry views (`plugins/endpoints/api/siteagent.py:394-411`). No `RegistrySecurity`, no `SetAccessControl`. The key inherits read for BUILTIN\Users. The repository states this outcome itself, in a file added in this same range: `.github/GE-ENFORCE-COLLECTOR-FIX.html:377-380`.
**Why it is still authority:** `authenticate_agent` accepts `X-API-Key` whenever `shared_key_allowed()` (`plugins/endpoints/api/agentauth.py:184-192`, default true at `plugins/endpoints/api/agentauth.py:67`), and `shared_key_refused_for` refuses it only for a hostname whose asset holds an **active** credential (`plugins/endpoints/api/agentauth.py:129-134`). So for any not-yet-enrolled, revoked or re-enroll-blocked PC the shared key still claims queued commands (`plugins/endpoints/api/routes.py:3352`) and downloads script bodies and installer packages (`plugins/endpoints/api/routes.py:3511`, `:3550`, `:3589`).
**Impact:** any operator logged on to a bay can read the fleet key and act as any un-enrolled endpoint: claim and answer its queued commands, write false compliance records, and fetch hosted installers and script bodies. It also means revoke plus re-enroll-block leaves a compromised PC holding a working credential of the older kind.
**Contrast:** the per-PC credential got the opposite treatment in the same range. DPAPI LocalMachine inside a SYSTEM and Administrators only DACL (`agent/ShopDbEndpointAgent.Core/AgentCredentials.cs:203-217`, `agent/ShopDbEndpointAgent.Core/ProtectedFileSystem.cs:219-232`), and `appsettings.json` is re-ACLed at runtime (`agent/ShopDbEndpointAgent/AgentPipeServer.cs:494-522`). The registry key alone was left open.
**Status, 2026-09-18: fixed.** The generated installer now calls `Protect-HklmShopdbKey` after provisioning, which breaks inheritance, drops the inherited entries rather than copying them, and states SYSTEM (S-1-5-18) and Administrators (S-1-5-32-544) by SID in both registry views (`plugins/endpoints/api/siteagent.py`). It runs for the key-less package too, because `ServerPins` live in the same key and the registry is authoritative for pinning. A bay that cannot take the ACL still installs, and says plainly that the key stays readable there. Four tests cover it in `tests/test_plugins/test_endpoints_siteagent_build.py`, including the write-then-lock ordering.
**The key's continued existence is a decision, not an oversight.** The shared key stays until `upgrade-shopdbagent.ps1`, `Report-AssetToShopDB.ps1` and `Report-PrintersToShopDB.ps1` are off it, because rollback dies without it; the site is mid-migration to per-bay credentials and the key-less package (`includesharedkey=False`) is already the path for new installs. What this finding was about was the DACL, and that is now closed. The remaining work is the migration itself, and finally clearing `CollectorKey` from both views as the runbook plans (`plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:351-361`).
**Not closed by this:** bays that already carry the key keep their inherited DACL until they are reinstalled or upgraded through a package built after this change.
### Medium
### 20. Turning the shared key off does not close the four legacy collector routes
`generic_collect` enforces the new rule: 401 `SHARED_KEY_REFUSED` when `agent_shared_key_refused(identityvalue)` is true (`shopdb/core/api/collector.py:279-284`), which `plugins/endpoints/api/agentauth.py:129-134` makes true fleet-wide once `plugin.endpoints.sharedkeyallowed` is off.
Four legacy routes never consult it. `/pc` (`shopdb/core/api/collector.py:349`), `/apps` (`:387`), `/heartbeat` (`:459`) and `/bulk` (`:494`) are decorated `@require_api_key`, which checks only `COLLECTOR_API_KEY` or a `collector.ingest` scoped token (`shopdb/core/api/collector.py:154-164`). `/pc` then writes `lastreporteddate`, `loggedinuser`, `lastboottime` and the asset serial number for any hostname it can resolve (`shopdb/core/api/collector.py:367-376`); `/apps` and `/bulk` rewrite installed-app inventory.
Two details that make this worse than the runbook implies. `_env_fallback_disabled(pluginname)` returns False when `pluginname` is None (`shopdb/core/api/collector.py:60-74`), so these routes keep honouring the `.env` `COLLECTOR_API_KEY` even after `plugin.endpoints.envkeyfallback` is switched off. And "Regenerate key" mints a new `collector.ingest.endpoints` token and clears the env fallback but never changes `COLLECTOR_API_KEY` (`plugins/endpoints/api/routes.py:1864-1884`), so runbook Phase 6 step 4 does not close this either.
**Bounded by:** the UI-managed agent key is scoped `collector.ingest.endpoints` (`plugins/endpoints/api/routes.py:1834`) and `_collector_scopes_for(None)` allows only the generic `collector.ingest` (`shopdb/core/api/collector.py:46-57`), so a PC holding only that token is already refused on all four routes. The exposure needs a copy of the env shared key, and the reachable damage is inventory-field integrity. No command issuance and no credential minting.
ADR-006 says the legacy `/pc` endpoint is removed before v1.0. At `__version__` 0.12.0 keeping it is not yet a violation, but it is a hole in a boundary this branch advertises.
### 21. Rejecting an enrollment request is not durable
`enroll_endpoint` is unauthenticated by design. When a row for (hostname, serialnumber) already exists, it returns `rejected` only when the presented nonce matches the stored one (`plugins/endpoints/api/enrollment.py:383`). A REJECTED row presented with a **different** nonce falls into the non-pending branch at `plugins/endpoints/api/enrollment.py:390` and is rewritten into a fresh cycle: new nonce, new `requestedat`, `decidedat` and `decidedby` cleared (`:393-396`). Execution then falls through to the match block, and with the fleet shared key present it is auto-issued a credential outright (`plugins/endpoints/api/enrollment.py:426-427`). Both gates default on (`plugins/endpoints/api/agentauth.py:67`, `:71`). Without the key the request simply lands back in the admin approval queue, indistinguishable from a first-time request.
There is no second lever: `reject_enroll_request` sets no durable flag (`plugins/endpoints/api/enrollment.py:753-770`), and `reenrollblocked` lives on `EndpointCredential` (`plugins/endpoints/models/credential.py:56`), which `revoke_endpoint_credential` 404s on when no credential row exists (`plugins/endpoints/api/enrollment.py:801-805`). So a PC that never enrolled cannot be blocked.
**Bounded by:** auto-issue additionally requires a Computer record whose hostname matches and whose Asset serial matches case-insensitively (`plugins/endpoints/api/enrollment.py:405-421`), so this does not grant a credential for an arbitrary asset. The existing test covers only the same-nonce case (`tests/test_plugins/test_endpoints_credentials.py:250`).
### 22. The one-line bootstrap stages its script in a world-writable path and runs it elevated
`COMMAND_SCRIPT_PATH` is a literal `C:\Windows\Temp\shopdb-bootstrap.ps1` (`plugins/endpoints/api/bootstrap.py:68`), and the emitted command downloads to that path and then executes it, with a `Remove-Item -Force` against the same path in the finally block (`plugins/endpoints/api/bootstrap.py:164-169`). The module docstring and the script header both say the line is pasted into an elevated PowerShell (`plugins/endpoints/api/bootstrap.py:3-5`, `:209-213`). The same literal is published in `plugins/endpoints/AGENT-CONTRACT.md:794` and `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:296`.
`C:\Windows\Temp` is not admin-only: its default DACL grants BUILTIN\Users create-file plus CREATOR OWNER full control on what they create, and overwriting a file's contents does not reset its owner or DACL. Nothing verifies the script after download; the sha256 checks in the script body cover the downloaded ZIP's binaries, not the script itself.
**Bounded by:** this needs an attacker who can already log on interactively to that specific bay and a tech who later pastes the one-liner on that same PC. The bootstrap path is documented as the one-off route (`plugins/endpoints/api/bootstrap.py:7-11`), not how managed bays get the agent.
**Fix direction:** stage under a path only Administrators and SYSTEM can write, or verify the script against a server-stamped hash before executing it.
### 23. Kiosk network admission rests on a header the client can send, and proxy mode verifies nothing
`request_ip_allowlisted` reads `client_ip()`, which is `strip_port(request.remote_addr)` (`shopdb/utils/clientip.py:53`). On the Windows deployment `remote_addr` is not the TCP peer: `deploy/windows/web.config:31` launches waitress with `--trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for --trusted-proxy-count=1`, and HttpPlatformHandler always connects from loopback, so waitress always parses `X-Forwarded-For` and promotes the selected hop into `REMOTE_ADDR`.
The control that makes that hop honest is the IIS rewrite rule that overwrites the header with `{REMOTE_ADDR}`, and it ships commented out (`deploy/windows/web.config:90-102`). The installer enables and then verifies it only under `-ClientIpSource direct`: the whole "CLIENT-IP RULE IS NOT IN EFFECT" check sits inside `if ($ClientIpSource -eq 'direct')` at `deploy/windows/installer/shopdb-install.ps1:2509-2523`. In proxy mode the rule is deliberately left off (`:2041-2046`) and **nothing checks that a proxy is actually in front**. The module docstring says the same thing about the failure mode: it does not fail closed (`shopdb/utils/ipallowlist.py:17-26`).
**Bounded by:** `-ClientIpSource` defaults to `direct` (`deploy/windows/installer/shopdb-install.ps1:129`), which installs URL Rewrite, uncomments the block and verifies it live, so a default install is covered. With a real proxy in proxy mode the attack also fails, because waitress takes the last hop and ARR appends the true peer after any client-supplied value. Both kiosk allowlists default to empty and both kiosks are additionally gated on their enabled flag (`plugins/usb/api/routes.py:798-801`, `plugins/printedparts/api/routes.py:551-557`).
**The real gap:** proxy mode performs no verification at all. A server set to proxy mode with no proxy really there, or a hand-maintained `web.config` where the rewrite block was never uncommented, admits an attacker-chosen address. The same header also satisfies the GE-Enforce IP path (`plugins/geenforce/api/routes.py:82-108`), which is pre-existing and not introduced by `da91a2915`, and it re-keys the login rate limit and account lockout on a value the caller chose.
### Low
### 24. Revoke does not stop a live agent; it falls back to the shared key
`plugins/endpoints/models/credential.py:7-10` says `reenrollblocked` keeps a revoked PC from enrolling again, listing "a compromised PC" among the uses. The containment does not exist. On a 401 the agent deletes its credential and clears it (`agent/ShopDbEndpointAgent.Core/AgentCredentials.cs:107-135`), then every later request takes the shared-key branch (`agent/ShopDbEndpointAgent.Core/AgentHttpFactory.cs:119-120`). Server side, `shared_key_refused_for` refuses the shared key only when an active credential exists (`plugins/endpoints/api/agentauth.py:129-134`), and revoke has just cleared that (`plugins/endpoints/api/enrollment.py:773-825`).
This is documented intended behaviour, not an undetected gap: `plugins/endpoints/api/agentauth.py:11-14` states it, and `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md:379-380` says outright that the server admits the shared key again. What needs fixing is the model comment, which implies a containment the design never provides. There is no per-PC deny until the shared key is off fleet-wide.
### 25. The bootstrap token travels in the query string and is written to both access logs
`_resolve_token` takes the whole authorization from the URL (`plugins/endpoints/api/bootstrap.py:182`) for both token routes (`plugins/endpoints/api/bootstrap.py:60-61`). Both hops log the full request line: `nginx.conf:36` is `access_log /dev/stdout combined`, and the gunicorn access format uses `%(r)s` (`docker-compose.yml:213-214`). The IIS deployment logs `cs-uri-query` in its W3C log by default too.
The same range deliberately stopped logging the setup token for exactly this reason (`shopdb/__init__.py`, setup-token block), and the audit rows this module writes carry only `tokenprefix` (`plugins/endpoints/api/bootstrap.py:192-195`).
**Bounded by:** what a replayed token buys is bounded and carries no secret. `download_bootstrap_package` is explicitly keyless (`plugins/endpoints/api/bootstrap.py:532-535`) and the script it serves contains only the base URL and pins. The reader also needs container or IIS log access, which is already privileged.
### 26. The unauthenticated enroll queue can be filled to its cap
`enroll_endpoint` takes no auth and writes a row for every unseen (hostname, serialnumber) pair (`plugins/endpoints/api/enrollment.py:350-358`). The bounds are a per-IP fixed window of 300 per 60 seconds (`plugins/endpoints/api/enrollment.py:65-66`) and a pending cap of 500 (`plugins/endpoints/api/enrollment.py:255`). Junk rows count toward the cap: an unknown hostname returns `_pending(row, REASON_UNKNOWN_HOSTNAME)` at `plugins/endpoints/api/enrollment.py:411`, which leaves the row PENDING. There is no bulk-reject, prune or stale-row expiry in that module. The limiter's counter is per-process (`plugins/endpoints/api/ratelimit.py:10-11`), so the real budget is the configured maximum times the worker count.
**Bounded by:** while `sharedkeyallowed` is on (the default), a PC that cannot enroll is not offline; it keeps reporting on the fleet key. This only becomes an outage for new bays once the shared key is turned off at the end of the rollout.
### 27. The kiosk confirms an employee name for any badge it is handed
`kiosk_session` normalises the posted badge and returns the resolved employee name in the response body (`plugins/usb/api/selfhosted.py:346-371`, resolver at `:34-57`); the external mode does the same (`plugins/usb/api/routes.py:904-960`). The route carries no auth decorator (`plugins/usb/api/routes.py:905-906`) and there is no rate limiter anywhere in either file. `kiosk_others` additionally returns the holder name for every checked-out drive to the same caller, gated only on `usb_kiosk_allow_return_for_others`, which defaults true (`plugins/usb/plugin.py:169-172`).
Before `da91a2915` this needed the per-device token, so it was confined to screens an administrator had provisioned. It is now reachable by anything inside `usb_kiosk_allowed_cidrs`.
**Bounded by:** the kiosk must be enabled and the caller must be inside a deliberately configured CIDR. The data is badge-to-name, which is printed on the badges themselves and visible to any signed-in user, and the surface is read-only. An unrecognised badge answers 422 rather than silently.
### 28. The kiosk allowlist settings accept a world-open range with no server-side validation
`usb_kiosk_allowed_cidrs` (`plugins/usb/plugin.py:154-168`) and `printedparts_kiosk_allowed_cidrs` (`plugins/printedparts/plugin.py:176-188`) are plain string settings with no validator, and the settings PUT path applies none. `ip_in_cidrs` parses each entry with `ipaddress.ip_network(part, strict=False)` (`shopdb/utils/ipallowlist.py:55`), so `0.0.0.0/0` and `::/0` parse and match everything. The only shape check is client-side and cosmetic (`frontend/src/components/NetworkAllowlistPicker.vue:84-87`), and `0.0.0.0/0` satisfies its regex.
The realistic route in is an administrator who cannot get the allowlist to match, most likely because the client-IP rule is missing and every caller reads as loopback, and who widens the range to make it work.
### 29. The kiosk provisioning link hands out the token in a URL
`kiosk_provisioning_link` returns `/usb-kiosk?token=<token>` (`plugins/usb/api/routes.py:849`) and the printed-parts twin returns `/parts-kiosk?token=<token>` (`plugins/printedparts/api/routes.py:578`). Both are gated on `settings.edit` and both write an audit row first (`plugins/usb/api/routes.py:846-848`, `plugins/printedparts/api/routes.py:574-576`). The token then lands in the IIS or nginx access log, browser history and any clipboard history on the admin's PC, and nothing rotates it after a reveal.
The same file elsewhere states the contrasting reasoning: `kiosk_session` is a POST specifically so the badge does not end up in a proxy access log the way a query string would (`plugins/usb/api/routes.py:914-916`).
**Qualification:** the query-string model is not introduced here. It is the mechanism the token setting itself prescribes (`plugins/usb/plugin.py:146-151`), so the new route only surfaces a URL the product already documented. This is hardening, not a regression.
### 30. Every deployment warning about the client-IP rule still names only GE-Enforce
Three operator-facing places enumerate what depends on the `X-Forwarded-For` rewrite rule, and none was updated when the kiosk allowlists were added: `deploy/windows/installer/shopdb-install.ps1:2520-2522`, `deploy/windows/installer/shopdb-preflight.ps1:322-325` and `deploy/windows/web.config:51-58`. None names `usb_kiosk_allowed_cidrs` or `printedparts_kiosk_allowed_cidrs`, which are now the only allowlists in the product gating unauthenticated writes. The dependency is stated only in the settings descriptions (`plugins/usb/plugin.py:161-168`, `plugins/printedparts/plugin.py:183-187`), which the person building the server does not read.
**One claim not to carry forward:** the remediation line offered at `deploy/windows/installer/shopdb-install.ps1:2522`, "or use token authentication", is correct for the kiosks. The per-device token is a genuine alternative to the allowlist on both.
### 31. This page, and three other documents, attributed a credential policy to ADR-016 that ADR-016 does not contain
The References section of the 2026-09-11 version of this page described ADR-016 as "Credential Delivery (device token authentication)". That is wrong, and the same misattribution is repeated at `docs/wiki-drafts/README.md:316-323`, `CLAUDE.md:119` and the assistant quick-reference file line 135. Grepping the ADR set for device tokens returns nothing in any ADR.
The worst form is in the SECURITY.md page that `docs/wiki-drafts/README.md:429` queues for publication. Under "ADR-016: Credential Delivery" it lists device tokens, personal access tokens and environment variables as the delivery mechanisms and then bans, at `docs/wiki-drafts/README.md:323`, "Hardcoded in source, registry keys, git history". ADR-016 decides none of that. Its Decision is a fleet credential store with Fernet at rest, a targeting and precedence resolver reusing `plugins/geenforce/filters.py`, an explain endpoint and `GET /api/collector/credentials` behind a dedicated `credentials.fetch` scope. Its own alternatives section recommends registry-only provisioning as a worthwhile first step.
The blanket ban on registry keys also contradicts shipped code on this branch: `plugins/computers/client/Report-AssetToShopDB.ps1:367-374` reads the collector key from `HKLM:\SOFTWARE\GE\ShopDB`, and `plugins/endpoints/client/Set-ShopDBServerPins.ps1:4-20` writes the server trust pins there by design.
**Status:** the References section of this page is corrected below. The other three are not, and the SECURITY.md draft should not be published as written. Note that `gh-wj` carries no `docs/adr` directory at all, so there is no ADR text on this branch to correct a reader who follows the wiki.
**Related, and unverifiable from this branch:** ADR-016's own mechanism is still unbuilt. Grepping `shopdb/` and `plugins/` for `credentials.fetch`, `collector/credentials` and `Fernet` returns nothing, and `plugins/endpoints/plugin.py:163-253` declares only `endpoints.*` permissions. The per-PC credential programme in this range is a different subject under the same word: the agent's own identity secret, not a store that delivers third-party credentials to a PC. The design is sound and consistent with ADR-016's intent, but nothing on this branch records it as a decision. That record belongs on the lineage that holds `docs/adr`.
### 32. Kiosk check-in records no wipe or scan state, so the new CMMC audit columns stay blank
`35d4fcd41` added `sanitized` and `scanned_viruses` to the USB audit report on the stated grounds that custody without the state of the media on return is half the story. Both values are written only on the authenticated return path (`plugins/usb/api/selfhosted.py:275-280`, `plugins/usb/api/routes.py:562-571`). The kiosk returns write neither: `kiosk_checkin` updates only check-in time, SSO and name (`plugins/usb/api/selfhosted.py:527-531`), and the external kiosk insert lists only badge, device, action, timestamp and locker (`plugins/usb/api/routes.py:1157-1162`).
This is an honest blank, not a false record: `auditflag` renders the NULL as an empty cell rather than "no" (`plugins/usb/api/selfhosted.py:547-559`), and the tri-state is deliberate. But `da91a2915` has just widened the kiosk from a few provisioned screens to every PC on the allowed networks, so it is about to become the path most returns take.
**Correction to note:** the signed-in path does not answer the scan question either. `USBDetail.vue:226-230` and `USBList.vue:267-271` send only `sanitized`, never `scanned_viruses`, so `plugins/usb/api/routes.py:564` always receives None and that column is blank for every return through any first-party UI.
## Verified as sound
Read on 2026-09-18 and found correct. Recorded so the next review does not re-litigate them.
- **Credential storage on the server.** Only a sha256 hash is persisted, plus a 12-character display prefix. The secret exists once, in the 201 enroll response (`plugins/endpoints/api/enrollment.py:212-222`). No admin route re-reads or re-displays it; `issue_credential` has exactly two callers, both inside `enroll_endpoint`.
- **Cross-asset binding.** A credential caller cannot act for another PC. `bind_hostname` returns `ASSET_MISMATCH` for a hostname resolving elsewhere (`plugins/endpoints/api/agentauth.py:206-231`), and every agent-facing route goes through it.
- **Admin surface authorization.** Approve, reject, revoke, allow-reenroll, the settings PUT, the migration wave and all three bootstrap-token routes are `@require_role('admin')` (`plugins/endpoints/api/enrollment.py:642`, `:755`, `:773`, `:827`, `:852`, `:1134`; `plugins/endpoints/api/bootstrap.py:420`, `:475`, `:486`).
- **Tray holds no secret.** The tray reaches the SYSTEM agent over a named pipe with a SYSTEM and Administrators full, INTERACTIVE read-write ACL and FirstPipeInstance anti-squat (`agent/ShopDbEndpointAgent/AgentPipeServer.cs:126-146`); no pipe operation returns the credential (`:246-256`).
- **Bootstrap token scope and lifetime.** Hashed at rest, expiry mandatory and bounded to 1 to 168 hours, uses bounded to 1 to 500, revocable, and unknown, malformed, expired, revoked and exhausted all answer an identical 404 so the URL confirms nothing (`plugins/endpoints/models/bootstrap.py:197-212`, `plugins/endpoints/api/bootstrap.py:109-111`, `:176-189`). The package it serves never carries the collector key.
- **GE-Enforce payload scoping.** `/payload/<sha256>` enforces the token's resource scope before the ETag shortcut, so a bound token cannot even confirm another scope's hash exists (`plugins/geenforce/api/routes.py:252-283`).
- **`GET /api/plugins/settings-routes` is no longer anonymous.** It moved to `@require_permission('settings.view')` (`shopdb/core/api/plugins.py:47`) and was removed from the public-read allowlist with the reasoning recorded (`tests/test_core/test_public_read_surface.py:29-34`).
- **The kiosk admission gate fails closed in every direction it claims to.** Empty setting matches nothing, an unparseable caller address matches nothing, a malformed CIDR entry is skipped rather than widening the match (`shopdb/utils/ipallowlist.py:44-57`), and the allowlist cannot override the enabled flag. Device-token comparison uses `hmac.compare_digest` in both kiosks (`plugins/usb/api/routes.py:762`, `plugins/printedparts/api/routes.py:536`). An unadmitted caller gets 404, not 403.
- **No secrets committed in the range.** Every `sdbe_` and `sdbb_` hit in the diff is a prefix constant, a regex, a doc example or a test fixture. `certs/` remains ignored except its README, openssl.cnf and the two regeneration scripts.
## Performance Findings
### Agent Polling Inefficiency (99.9% improvement opportunity)
### Agent Polling Inefficiency [LARGELY ADDRESSED]
**Current behavior:** Agent polls `/api/endpoints/inventory` every 30 seconds to POST full inventory (hostname, IP, apps, drives, processes).
**2026-09-18 verdict:** the behaviour the 2026-09-11 page measured no longer matches the code.
**Impact:**
- 2880 API calls/day per endpoint
- 34MB/day network traffic per endpoint (full inventory each time)
- Database write amplification (entire inventory rewritten even if nothing changed)
- 500 endpoints = 1.44M API calls/day, 17GB/day traffic
- The cadence is not 30 seconds. `agent/ShopDbEndpointAgent/appsettings.json:5` ships `cadenceseconds: 900` with 60 seconds of jitter and 30 seconds of startup jitter, so roughly 96 check-ins per day per endpoint rather than 2880.
- Application inventory is a delta, not a full rewrite. `plugins/endpoints/plugin.py:1187` onward builds an incoming fingerprint map, validates each fingerprint as 64 lowercase hex, and applies only the difference; a genuine unique-key collision is caught inside a savepoint and reconciled row by row (`plugins/endpoints/plugin.py:1301-1336`, `:1359-1410`) rather than sinking the check-in.
- Two hotfixes in this range removed row-level locks and hot-path logging from the collector check-in path (`affe455c5`, `1ed0c4d21`).
**Optimization:** Delta updates
1. Agent calculates inventory delta locally (added/removed apps, changed IP, etc.)
2. POST only changes: `{"added_apps": [...], "removed_apps": [...], "changed_ip": "10.1.2.3"}`
3. Server applies delta to existing record
4. Fallback to full inventory on agent startup or if delta rejected
**Expected improvement:**
- 99% fewer writes (only ~1% of polls have changes)
- 99.5% less network traffic (delta is <500 bytes vs 11KB full inventory)
- Reduced database load (updates instead of full rewrites)
**Status:** Planned for Q1 2027. Requires agent + server changes.
The traffic and write-volume figures in the 2026-09-11 text were site measurements against the old cadence and should not be carried forward.
## Remediation Summary
| Finding | Severity | Status | Timeline |
|---------|----------|--------|----------|
| Unauthenticated inventory updates | Critical | FIXED | 2026-09-11 |
| Deployment credential exposure | Critical | MITIGATED | 2026-09-11 |
| No certificate validation | Critical | PLANNED | Q4 2026 |
| Hardcoded admin credentials | Critical | FIXED | 2026-09-11 |
| Collector key in git | High | FIXED | 2026-09-11 |
| Passwords in mcp-servers.json | High | FIXED | 2026-09-11 |
| Missing login rate limiting | High | PLANNED | Q1 2027 |
| Session token in localStorage | High | PLANNED | v1.0 |
| Audit log retention | High | PLANNED | Q4 2026 |
| Endpoint events retention | High | PLANNED | Q4 2026 |
| Docker log rotation | Medium | PLANNED | Q4 2026 |
| Agent delta updates | Performance | PLANNED | Q1 2027 |
| Finding | Severity | Status as of 2026-09-18 | Evidence |
|---------|----------|-------------------------|----------|
| 1. Unauthenticated inventory updates | Critical | CLOSED on the routes that exist | `shopdb/core/api/collector.py:251` |
| 2. Collector key unprotected in registry | Critical | STILL OPEN, see finding 19 | `plugins/endpoints/api/siteagent.py:328` |
| 3. No certificate validation in agent | Critical | FIXED by SPKI pinning | `agent/ShopDbEndpointAgent.Core/AgentHttpFactory.cs:291` |
| 4. Hardcoded admin credentials | Critical | PARTLY FIXED, dev-only guard added | `shopdb/cli/__init__.py:619` |
| 5. Collector key in git | High | FIXED; no pre-commit secret scan exists | `.gitignore:174`, `.githooks/pre-commit:5` |
| 6. Passwords in mcp-servers.json | High | FIXED on this branch; keys not rotated | `.gitignore:176` |
| 7. Missing login rate limiting | High | FIXED, limiter plus account lockout | `shopdb/core/api/auth.py:35`, `:113` |
| 8. SQL injection in asset search | High | FALSE POSITIVE, re-confirmed | `plugins/usb/api/routes.py:277` |
| 9. Session token in localStorage | High | UNCHANGED | `frontend/src/stores/auth.js:6` |
| 10. Audit log retention | High | BUILT, off by default, no scheduler | `shopdb/core/services/auditretention.py:123` |
| 11. Endpoint events retention | Medium | STILL OPEN, no purge code exists | `plugins/endpoints/models/endpoint.py:333` |
| 12. HTTPS enforcement in dev | Medium | CHANGED, TLS bypass removed | `frontend/vite.config.js:5` |
| 13. Docker log rotation | Medium | STILL OPEN | `docker-compose.yml` (no logging block) |
| 14-18. Additional medium findings | Medium | PARTLY ANSWERED, source document removed | `shopdb/__init__.py:334`, `8eee6f19d` |
| 19. Shared key in registry with no ACL | High | FIXED 2026-09-18 (installed bays until reinstall) | `plugins/endpoints/api/siteagent.py`, `Protect-HklmShopdbKey` |
| 20. Legacy collector routes ignore the refusal | Medium | OPEN | `shopdb/core/api/collector.py:349` |
| 21. Enroll rejection is not durable | Medium | OPEN | `plugins/endpoints/api/enrollment.py:390` |
| 22. Bootstrap stages elevated script in Windows\Temp | Medium | OPEN | `plugins/endpoints/api/bootstrap.py:68` |
| 23. Kiosk admission spoofable in proxy mode | Medium | OPEN | `deploy/windows/installer/shopdb-install.ps1:2509` |
| 24. Revoke falls back to the shared key | Low | BY DESIGN, model comment wrong | `plugins/endpoints/models/credential.py:8` |
| 25. Bootstrap token in query string | Low | OPEN | `plugins/endpoints/api/bootstrap.py:182` |
| 26. Enroll queue can be filled to cap | Low | OPEN | `plugins/endpoints/api/enrollment.py:350` |
| 27. Badge to name oracle on the kiosk | Low | OPEN | `plugins/usb/api/selfhosted.py:346` |
| 28. Allowlist accepts 0.0.0.0/0 | Low | OPEN | `plugins/usb/plugin.py:154` |
| 29. Provisioning link carries the token in a URL | Low | OPEN, pre-existing mechanism | `plugins/usb/api/routes.py:849` |
| 30. Client-IP warnings name only GE-Enforce | Low | OPEN | `deploy/windows/installer/shopdb-install.ps1:2520` |
| 31. ADR-016 misattributed in four documents | Low | ONE FIXED HERE, three open | `docs/wiki-drafts/README.md:316` |
| 32. Kiosk check-in records no wipe or scan | Low | OPEN | `plugins/usb/api/selfhosted.py:463` |
| Agent polling inefficiency | Performance | LARGELY ADDRESSED | `agent/ShopDbEndpointAgent/appsettings.json:5` |
## Testing Performed
**Authentication enforcement:**
```bash
# Inventory endpoint now requires device token
curl -X POST http://localhost:5001/api/endpoints/inventory
# 401 Unauthorized
This refresh was a read of the code at `35d4fcd41`, not a live exercise against a server. What was run:
curl -X POST http://localhost:5001/api/endpoints/inventory \
-H "X-API-Key: shopdb_device_test123" -d '{...}'
# 200 OK
**The naming and style gate:**
```bash
bash scripts/check-naming-and-style.sh
# exit 0, all checks pass
```
Its ADR-015 site-literal grep is not vacuous: a control pattern over the same six directories returns 1995 lines.
**Kiosk network allowlist suite:**
```bash
pytest tests/test_plugins/test_kiosk_network_allowlist.py
# 20 passed
```
**Credential scanning:**
**Tracked-secret scan:**
```bash
# No hardcoded passwords in source
bash scripts/check-naming-and-style.sh
# PASS (includes credential pattern checks)
# No collector keys in tracked files
git ls-files | xargs grep -l "shopdb_pat_" | wc -l
# 0
```
**Registry key cleanup:**
```powershell
# Verify registry key removed
Get-ItemProperty "HKLM:\SOFTWARE\ShopDB\Agent" -Name CollectorApiKey -ErrorAction SilentlyContinue
# (none found)
```
### Gate coverage gaps found while testing
- **The ADR-015 site-literal scan does not look at the C# agent.** The grep is scoped to `plugins/ shopdb/ scripts/ deploy/ frontend/src/ tools/` with `--include` on .py, .vue, .js, .ps1, .psm1, .sh, .iss, .json and .html (`scripts/check-naming-and-style.sh:434-440`). `agent/` is absent and `.cs` is not an included extension. Roughly 2500 lines of C# were added under `agent/` in this range, all of which ships to a sister site. Re-running the assembled site patterns over `agent/` by hand returns one hit, prose in `agent/ShopDbEndpointAgent/ShopDbEndpointAgent.csproj:108`, which ADR-015 explicitly permits. The tree is clean; the gate simply would not notice if it stopped being clean. ADR-015's own 2026-08-14 amendment widened the scan on exactly this reasoning, because the literals that reached a second site were in the client scripts and the installer. The C# agent is now that client channel.
- **The gate has no credential pattern check.** See finding 5. The 2026-09-11 page's claim that it includes one is not true of this branch.
## Recommendations
**Immediate (P0):**
1. Deploy per-endpoint certificate pinning (Q4 2026)
2. Mass agent update to remove registry keys (Q4 2026)
3. Rotate exposed collector keys (user accepted risk, not rotating)
**Do first:**
1. ACL the collector key registry value in the generated installer, or finish enrollment and clear it (finding 19).
2. Make the shared-key refusal cover `/pc`, `/apps`, `/heartbeat` and `/bulk`, or retire them (finding 20).
3. Make an admin rejection durable so a new nonce cannot restart the cycle (finding 21).
4. Stage the bootstrap script somewhere a standard user cannot pre-create, or verify it against a server-stamped hash (finding 22).
5. Verify in proxy mode that a proxy is really in front, the way direct mode already verifies the rewrite rule (finding 23).
**Short-term (P1):**
1. Implement audit log retention (Q4 2026)
2. Configure Docker log rotation (Q4 2026)
3. Add login rate limiting (Q1 2027)
**Then:**
1. Do not publish the SECURITY.md wiki page as drafted; it attributes a policy to ADR-016 that ADR-016 does not contain and bans the registry storage the fleet depends on (finding 31).
2. Name the two kiosk allowlists in the installer, preflight and web.config warnings (finding 30).
3. Add server-side validation that rejects a world-open range on both kiosk allowlist settings (finding 28).
4. Move the bootstrap token out of the query string, or accept it and document the log exposure (finding 25).
5. Bound or expire pending enrollment rows so the queue cannot be held at its cap (finding 26).
6. Correct the `reenrollblocked` comment so it does not claim containment of a compromised PC (finding 24).
**Long-term (P2):**
1. Implement agent delta updates (Q1 2027)
2. Migrate sessions to HttpOnly cookies (v1.0)
3. Add CSP headers (v1.0)
4. Purge credentials from git history (v1.0)
**Carried over from 2026-09-11 and still open:**
1. Retention for `endpointevents` (finding 11).
2. Docker log rotation (finding 13).
3. A retention value and a schedule for the audit log, now that the tooling exists (finding 10).
4. HttpOnly cookie sessions for non-kiosk users (finding 9).
5. Recover or re-derive the detail behind findings 14 to 18, which left the tree with `8eee6f19d`.
**Extend the gates:**
1. Add `agent/` and `.cs` to the ADR-015 site-literal scan.
2. Add a credential pattern check to the gate so the pre-commit hook actually blocks a committed key.
## References
- Security Fix Implementation Guide: `.claude/memory/security_fix_implementation.md`
- Maintenance Baseline: 936 hrs/year operational burden, includes security tasks
- ADR-016: Credential Delivery (device token authentication)
- ADR-XXX: Per-Endpoint Certificate Authentication (planned)
- Per-PC credential rollout runbook: `plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md`
- Agent contract: `plugins/endpoints/AGENT-CONTRACT.md`
- ADR-006: Collector API, the single dynamic dispatch route and its response contract
- ADR-015: Site-specific configuration, and the 2026-08-14 amendment that set the scan's scope
- ADR-016: Credential Delivery. A fleet credential store with Fernet at rest and a `credentials.fetch` scope. It is **not** about device tokens, and it does not ban registry storage. Still unbuilt on this branch. See finding 31.
- The ADR set is not in this repository. It lives on the product lineage; `gh-wj` has no `docs/adr` directory.
- The 2026-09-12 detailed audit reports were removed from the product tree by `8eee6f19d` and are kept with the site's working records. the assistant memory directory does not exist here, so the 2026-09-11 page's pointers into it are dead.

129
frontend/src/components/NetworkAllowlistPicker.vue Normal file
View File

@@ -0,0 +1,129 @@
<template>
<div class="allowlist-picker">
<div v-if="loading" class="picker-note">Loading networks...</div>
<div v-else-if="subnets.length" class="subnet-list">
<label v-for="subnet in subnets" :key="subnet.subnetid" class="subnet-row">
<input type="checkbox" :value="subnet.cidr"
:checked="isSelected(subnet.cidr)"
@change="toggle(subnet.cidr)" />
<span class="subnet-cidr">{{ subnet.cidr }}</span>
<span class="subnet-name">{{ subnet.name }}</span>
<span v-if="subnet.vlannumber" class="subnet-vlan">
VLAN {{ subnet.vlannumber }}<template v-if="subnet.vlanname"> - {{ subnet.vlanname }}</template>
</span>
</label>
</div>
<p v-else class="picker-note">
{{ unavailableMessage }}
</p>
<label class="manual-label">Allowed networks (CIDR, comma separated)</label>
<input :value="modelValue" type="text" class="form-control"
placeholder="empty = device token only"
@input="$emit('update:modelValue', $event.target.value)" />
<p v-if="extraEntries.length" class="picker-note">
Not from the network inventory: {{ extraEntries.join(', ') }}
</p>
<p v-if="invalidEntries.length" class="picker-warn">
Not valid CIDR and will never match: {{ invalidEntries.join(', ') }}
</p>
</div>
</template>
<script setup>
// Picks networks for a kiosk allowlist from the network plugin's subnet
// inventory, and stays usable without it.
//
// Stores resolved CIDR STRINGS, not subnet ids. The setting has to keep working
// at a site that does not run the network plugin, and must not break when an
// inventory row is edited or deleted. The trade is that a CIDR edited in the
// inventory leaves a stale copy here - which is why entries that match no
// subnet are listed back to the administrator rather than hidden.
import { ref, computed, onMounted } from 'vue'
import { fetchAllPages } from '@/api'
const props = defineProps({
modelValue: { type: String, default: '' },
})
const emit = defineEmits(['update:modelValue'])
const subnets = ref([])
const loading = ref(true)
const unavailableMessage = ref('No subnets found. Enter networks by hand below.')
function entries (raw) {
return String(raw || '').split(',').map(part => part.trim()).filter(Boolean)
}
const selected = computed(() => entries(props.modelValue))
function isSelected (cidr) {
return selected.value.includes(cidr)
}
function toggle (cidr) {
const current = selected.value
const next = current.includes(cidr)
? current.filter(entry => entry !== cidr)
: [...current, cidr]
emit('update:modelValue', next.join(', '))
}
// Entries the administrator typed that no subnet record matches. Not an error -
// a range may simply not be inventoried - but worth showing, because a stale
// entry left behind by an inventory edit looks exactly like this.
const extraEntries = computed(() => {
const known = new Set(subnets.value.map(subnet => subnet.cidr))
return selected.value.filter(entry => !known.has(entry))
})
// Shape check only, mirroring the server: anything the server cannot parse is
// skipped there, so it silently admits nobody. Better to say so here.
const CIDR_SHAPE = /^\d{1,3}(\.\d{1,3}){3}\/\d{1,2}$/
const invalidEntries = computed(
() => selected.value.filter(entry => !CIDR_SHAPE.test(entry) && !entry.includes(':')))
onMounted(async () => {
try {
subnets.value = await fetchAllPages('/network/subnets')
} catch (error) {
// The network plugin may be disabled at this site, or the signed-in user
// may not hold network.view. Neither stops an administrator configuring
// the allowlist by hand, so degrade instead of failing.
subnets.value = []
unavailableMessage.value =
'Network inventory unavailable. Enter networks by hand below.'
} finally {
loading.value = false
}
})
</script>
<style scoped>
.subnet-list {
display: flex;
flex-direction: column;
gap: 0.25rem;
max-height: 220px;
overflow-y: auto;
border: 1px solid var(--border-color, #ccc);
border-radius: 4px;
padding: 0.5rem;
margin-bottom: 0.5rem;
}
.subnet-row {
display: flex;
align-items: baseline;
gap: 0.5rem;
font-weight: 400;
}
.subnet-cidr { font-family: monospace; min-width: 12rem; }
.subnet-name { font-weight: 600; }
.subnet-vlan { color: var(--text-light); font-size: 0.85rem; }
.manual-label { display: block; font-weight: 600; margin-top: 0.5rem; }
.picker-note { color: var(--text-light); font-size: 0.85rem; margin: 0.25rem 0; }
.picker-warn { color: var(--warning-dark); font-size: 0.85rem; margin: 0.25rem 0; }
</style>

112
frontend/src/views/settings/AuthenticationSettings.vue
View File

@@ -133,6 +133,59 @@
</template>
</div>
<!-- Collector API credentials. This page is where a site manages who
may authenticate, and the shared collector key is a credential like
any other - it just belongs to scripts rather than people. It is
rendered here because there is no generic key/value settings editor
in this UI: without a control the seeded row can only be flipped by
editing the database by hand, which on an air-gapped prod box means
it is not a setting at all. -->
<div class="setting-group">
<h3>Collector API</h3>
<p class="setting-description">
The four legacy collector routes (<code>/api/collector/pc</code>,
<code>/apps</code>, <code>/heartbeat</code>, <code>/bulk</code>) name
the PC they write in the request body, so the shared collector key is
an any-host write key on them: whoever holds it can rewrite the
logged-in user, last boot time, serial number and installed apps of
any hostname. A collector-scoped managed token is unaffected.
</p>
<div class="setting-row">
<label class="toggle-label">
<span>Refuse the shared key on legacy collector routes</span>
<button
class="toggle-btn"
:class="{ active: legacySharedKeyRefused }"
@click="toggleLegacySharedKey"
:disabled="legacySharedKeySaving"
>
<span class="toggle-slider"></span>
</button>
</label>
<small class="input-hint toggle-hint">
Off by default, which is what every site does today. Turning it on
stops <code>Report-AssetToShopDB.ps1</code> and
<code>Report-PrintersToShopDB.ps1</code> reporting through those
routes until they are moved to per-PC credentials, along with
anything else still calling them with the shared key. Every
shared-key call is logged as "Legacy collector route used with the
shared key" whether this is on or off - read those warnings for a
while first, or turning it on is a guess. On the shipped IIS deploy
that line names the route but not the caller: the X-Forwarded-For
rewrite rule in web.config ships disabled, so every request reaches
the app from 127.0.0.1. Count the routes, not the addresses, unless
that rule is on. The two GE-Enforce
reporters that post to <code>/api/collector/computers</code> and
<code>/api/collector/printers</code> are not affected. This should
follow the move to per-PC credentials, not lead it.
</small>
</div>
<div v-if="legacySharedKeyError" class="error-message">
{{ legacySharedKeyError }}
</div>
</div>
<div class="setting-group">
<h3>User Management</h3>
<p class="setting-description">
@@ -151,14 +204,66 @@
</template>
<script setup>
import { onMounted, computed } from 'vue'
import { onMounted, computed, ref } from 'vue'
import { useSystemSettings } from '../../composables/systemSettings'
import { settingsApi } from '../../api'
import { apiError } from '../../utils/apiError'
const {
settings, saving, error, success,
loadSettings, saveSetting, toggleSetting,
} = useSystemSettings()
// collector.legacysharedkeyrefused is read and written here directly rather
// than through useSystemSettings: that composable only keeps keys listed in
// its own reactive map, and this one is core-owned but belongs to no page it
// already serves. Default false = today's behaviour, so anything but an
// explicit true renders off.
const LEGACY_SHARED_KEY_SETTING = 'collector.legacysharedkeyrefused'
const legacySharedKeyRefused = ref(false)
const legacySharedKeySaving = ref(false)
const legacySharedKeyError = ref('')
function isTruthySetting(value) {
// A row seeded boolean comes back as a real true; one created by hand
// through the settings API before the key was declared can be the string.
return value === true || value === 'true'
}
async function loadLegacySharedKey() {
try {
const { data } = await settingsApi.get(LEGACY_SHARED_KEY_SETTING)
legacySharedKeyRefused.value = isTruthySetting(data.data?.value)
} catch (e) {
// 404 = never seeded on this install. The backend default is off.
if (e.response?.status !== 404) console.error(e)
legacySharedKeyRefused.value = false
}
}
async function toggleLegacySharedKey() {
const next = !legacySharedKeyRefused.value
if (next && !window.confirm(
'Refuse the shared collector key on the legacy routes?\n\n'
+ 'Report-AssetToShopDB.ps1, Report-PrintersToShopDB.ps1 and anything '
+ 'else still using the shared key on /pc, /apps, /heartbeat or /bulk '
+ 'stop reporting until they move to per-PC credentials.')) {
return
}
legacySharedKeySaving.value = true
legacySharedKeyError.value = ''
try {
// PUT upserts, so this also works where the row was never seeded.
await settingsApi.update(LEGACY_SHARED_KEY_SETTING, next)
legacySharedKeyRefused.value = next
} catch (e) {
legacySharedKeyError.value = apiError(e, 'Failed to save setting')
console.error(e)
} finally {
legacySharedKeySaving.value = false
}
}
// Per-page connection status (stays with the page, not the composable).
const samlStatus = computed(() => {
if (!settings.saml_enabled) return 'inactive'
@@ -178,5 +283,8 @@ const samlMessage = computed(() => {
return 'Settings saved - waiting on SAML support, local accounts still in use'
})
onMounted(loadSettings)
onMounted(() => {
loadSettings()
loadLegacySharedKey()
})
</script>

92
plugins/endpoints/AGENT-CONTRACT.md
View File

@@ -791,27 +791,62 @@ One-line install and hands a technician one command, which is run in an
ELEVATED PowerShell on a PC with no agent:
```
powershell -ExecutionPolicy Bypass -c "try { Invoke-WebRequest -UseBasicParsing '<baseurl>/api/endpoints/siteagent/bootstrap.ps1?t=<token>' -OutFile 'C:\Windows\Temp\shopdb-bootstrap.ps1'; & 'C:\Windows\Temp\shopdb-bootstrap.ps1' } finally { Remove-Item 'C:\Windows\Temp\shopdb-bootstrap.ps1' -Force -ErrorAction SilentlyContinue }"
powershell -ExecutionPolicy Bypass -c "try { New-Item -ItemType Directory -Force -Path 'C:\Program Files\ShopDB\bootstrap' | Out-Null; icacls 'C:\Program Files\ShopDB\bootstrap' /inheritance:r /grant '*S-1-5-32-544:(OI)(CI)F' '*S-1-5-18:(OI)(CI)F' | Out-Null; if ((Get-Variable LASTEXITCODE -ValueOnly) -ne 0) { Write-Host 'Refusing to run: could not restrict C:\Program Files\ShopDB\bootstrap to administrators (icacls failed). Fix the directory or delete it, then run the command again.' -ForegroundColor Red; exit 6 }; Remove-Item 'C:\Program Files\ShopDB\bootstrap\shopdb-bootstrap.ps1' -Force -ErrorAction SilentlyContinue; Invoke-WebRequest -UseBasicParsing '<baseurl>/api/endpoints/siteagent/bootstrap.ps1' -Headers @{'X-Endpoint-Bootstrap-Token'='<token>'} -OutFile 'C:\Program Files\ShopDB\bootstrap\shopdb-bootstrap.ps1'; if ((Get-Acl 'C:\Program Files\ShopDB\bootstrap\shopdb-bootstrap.ps1').GetOwner([Security.Principal.SecurityIdentifier]).Value -notin @('S-1-5-32-544','S-1-5-18',[Security.Principal.WindowsIdentity]::GetCurrent().User.Value)) { Write-Host 'Refusing to run: the staged bootstrap script is not owned by an administrator.' -ForegroundColor Red; exit 5 }; & 'C:\Program Files\ShopDB\bootstrap\shopdb-bootstrap.ps1' } finally { Remove-Item 'C:\Program Files\ShopDB\bootstrap\shopdb-bootstrap.ps1' -Force -ErrorAction SilentlyContinue }"
```
Download then run, never `irm ... | iex`: Microsoft Defender scores the
piped-to-iex shape as `Trojan:Win32/Commando.A!ml` and kills the process before
a line of it runs (reproduced 5 of 5 on a default install, no ASR rules), and
every bay runs Defender. The shape of the line is load-bearing too:
every bay runs Defender. The rest of the shape is load-bearing too:
- a literal temp path and no `$` anywhere, because pasting into an
already-running PowerShell would otherwise expand `$p` and `$env:TEMP` in the
PARENT and hand the child a broken line - as written it means the same in
cmd.exe and in PowerShell;
- the token rides in the `X-Endpoint-Bootstrap-Token` HEADER. A query string
lands verbatim in the IIS and waitress access logs, and a live install token
has no business sitting in a log file;
- staging is `C:\Program Files\ShopDB\bootstrap`, **not**
`C:\Windows\Temp`. That directory grants Users create-file and gives CREATOR
OWNER full control of what they create, so any standard user on the bay can
pre-plant `C:\Windows\Temp\shopdb-bootstrap.ps1` and wait for a tech to
paste the command, which then runs the planted content elevated;
- the line re-asserts an admin-only DACL on the directory it creates
(`icacls /inheritance:r` granting only the well-known SIDs `S-1-5-32-544` and
`S-1-5-18`, never group names, which a rename or a localized install would
break), and CHECKS that icacls succeeded: it is a native command, so it
reports failure only in an exit code and `$ErrorActionPreference` never
applies to it. Exit 6 with a red message, rather than downloading into a
directory that still inherits whatever the parent grants;
- it refuses (exit 5) to run a staged file owned by anyone but an
administrator, `SYSTEM`, or the elevated caller, which is what a pre-planted
file looks like;
- a literal staging path and no `$` anywhere - including the exit-code read,
which is `Get-Variable LASTEXITCODE -ValueOnly`. Pasting into an
already-running PowerShell would otherwise expand every `$` in the PARENT and
hand the child a broken line. As written it means the same in cmd.exe and in
PowerShell;
- single quotes for every inner literal, so the outer `-c "..."` is never
closed early;
- `try`/`finally`, so the dropped file is removed even when the script exits
non-zero or the download itself fails. The wrapper's own exit code is the
removal's, so read the script's printed result rather than `%ERRORLEVEL%`.
The same URL is offered in the UI as a **Download .ps1** link, for a tech who
would rather save the file and run it. Fetching it does not spend a use either
way.
The same script is offered in the UI as a **Download .ps1** link, for a tech
who would rather save the file and run it. That link is the one caller that
cannot send a header, so it uses `?t=<token>&via=download`. Fetching the script
does not spend a use either way.
The `?t=` form without `via=download` is accepted ONLY so a bay provisioned
before this change - one still running a command pasted when the query form was
the published shape - keeps working. It is deprecated: every such fetch logs a
warning naming the caller's IP, and that warning is how the rollout finds bays
whose command needs re-issuing. Do not publish it, do not paste it into a new
bay, and do not build it by hand. A site that has re-issued all of its live
commands turns the query form off with
`plugin.endpoints.bootstrapquerytoken` = false, which also disables the
Download .ps1 link.
The key is declared in the endpoints plugin's `get_settings_defaults()`, so it
seeds as a boolean row, and it is flipped from Settings > Endpoints > Build
site agent > One-line install ("Accept the token in the URL"). It is read
through the core settings API, not a route of this plugin's own.
The token is only an authorization to FETCH the installer. The package it
serves carries no shared key, whatever `plugin.endpoints.sharedkeyallowed`
@@ -822,26 +857,41 @@ Admin routes (admin role, audited):
- `POST /api/endpoints/siteagent/bootstrap/tokens` `{"hours": 24, "maxuses": 25}`
(`hours` 1-168, `maxuses` 1-500). Returns
`{"secret", "command", "scripturl", "token"}`; `scripturl` is the same token
URL the command downloads, for the UI's Download .ps1 link.
The secret is in the response exactly once; only its sha256 and a 12-character
`tokenprefix` are stored.
`{"secret", "command", "scripturl", "token"}`. `command` carries the token in
the header; `scripturl` is the same route for the UI's Download .ps1 link, in
the `?t=<secret>&via=download` form a browser link can actually use. The
`via=download` marker is what keeps an admin's own click out of the
un-migrated-bay warning count - it is a hint, not authorization, and nothing
is granted by it; what matters is that a command generated before this change
cannot carry it. The secret is in the response exactly once; only its sha256
and a 12-character `tokenprefix` are stored.
- `GET /api/endpoints/siteagent/bootstrap/tokens` - prefix, created, expires,
`usecount` / `maxuses`, `revokedat`, `lastusedat`, `lastusedip` and a `state`
of `active`, `expired`, `exhausted` or `revoked`. Never a secret.
- `POST /api/endpoints/siteagent/bootstrap/tokens/<tokenid>/revoke`
Token-authenticated routes. No login; the token is the whole authorization and
travels in the query string as `t`, because a PC with no agent has neither a
credential nor a user session to put in a header:
Token-authenticated routes. No login; the token is the whole authorization,
because a PC with no agent has neither a credential nor a user session. It
travels in the `X-Endpoint-Bootstrap-Token` header:
| Route | Spends a use | Returns |
|---|---|---|
| `GET /api/endpoints/siteagent/bootstrap.ps1?t=...` | no | `text/plain` PowerShell, generated per token |
| `GET /api/endpoints/siteagent/bootstrap/package?t=...` | yes | the keyless agent package ZIP |
| `GET /api/endpoints/siteagent/bootstrap.ps1` | no | `text/plain` PowerShell, generated per token |
| `GET /api/endpoints/siteagent/bootstrap/package` | yes | the keyless agent package ZIP |
Both also accept `?t=<secret>` while
`plugin.endpoints.bootstrapquerytoken` is true (the default), for bays
provisioned before the header form and for the UI Download .ps1 link. A query
token without `via=download` logs a deprecation warning against the caller's
IP; the Download link's own fetches log at info instead and are audited as
`(query-string token, admin download link)`, so the warning count stays a list
of bays to re-issue. With the setting false, a query token answers **400**
naming the header.
Both answer **404** for a token that is unknown, malformed, expired, revoked or
exhausted, so the URL confirms nothing about which tokens exist. Both are
exhausted, and for a request that presents no token at all - so the URL
confirms nothing about which tokens exist, and a bare probe cannot learn
whether this site still accepts the query form. Both are
rate-limited per IP (fixed window, `ENDPOINTS_BOOTSTRAP_RATELIMIT_MAX` in
`ENDPOINTS_BOOTSTRAP_RATELIMIT_WINDOW_SECONDS`, 429 `RATE_LIMITED` over
budget) and both write an audit row against `EndpointBootstrapToken` carrying
@@ -851,7 +901,9 @@ retried paste must not burn the download budget.
The generated script:
- refuses to run unelevated (exit 1) and says to reopen PowerShell as
administrator;
administrator (the wrapper line has already refused with exit 6 if the
staging directory could not be locked down, or exit 5 if the staged file was
owned by someone else);
- refuses when the agent service already exists (exit 2) and says a managed bay
should get the agent from GE-Enforce, unless `SHOPDB_BOOTSTRAP_FORCE` is set
to anything but `0` in the same session;

75
plugins/endpoints/api/agentauth.py
View File

@@ -9,9 +9,15 @@ and, when the route names a PC, binds the caller through bind_hostname().
X-API-Key The shared fleet key (or a collector-scoped managed
token). Accepted only while
plugin.endpoints.sharedkeyallowed is on, and never for a
PC that holds an active credential. A PC blocked
from re-enrolling has no active credential, so an
older agent on it keeps working on the shared key.
PC that holds an active credential. A PC whose
credential was REVOKED falls back to the shared key,
which is how a rollback to an older agent works and how
the un-migrated fleet keeps reporting; a site that has
finished migrating turns
plugin.endpoints.revokeddeniessharedkey on and a revoked
PC is then refused until it enrolls again. A PC that
never enrolled has no credential row at all and is
never touched by either rule.
Only the sha256 of a credential is stored. The full secret exists once, in the
enroll response.
@@ -37,6 +43,12 @@ DISPLAY_PREFIX_LENGTH = 12
SHARED_KEY_SETTING = 'plugin.endpoints.sharedkeyallowed'
AUTO_ENROLL_SETTING = 'plugin.endpoints.autoenroll'
# Off by default, which is what this site does today: revoking a credential
# drops the PC back to the shared key. Turning it on makes a revoke a real
# eviction - the PC reports nothing until it enrolls again - so it is only safe
# once the fleet is off the shared key, and it also closes the rollback path
# that revoke + blockreenroll exists for.
REVOKED_DENIES_SHARED_KEY_SETTING = 'plugin.endpoints.revokeddeniessharedkey'
CODE_CREDENTIAL_INVALID = 'CREDENTIAL_INVALID'
CODE_SHARED_KEY_REFUSED = 'SHARED_KEY_REFUSED'
@@ -70,6 +82,12 @@ def auto_enroll_enabled():
return Setting.get(AUTO_ENROLL_SETTING, default=True) is not False
def revoked_denies_shared_key():
"""True when a revoked credential also bars the PC from the shared key."""
return Setting.get(REVOKED_DENIES_SHARED_KEY_SETTING,
default=False) is True
def active_credential_for(assetid):
from ..models import EndpointCredential
@@ -81,6 +99,33 @@ def active_credential_for(assetid):
.first())
def shared_key_denied_for(assetid):
"""True when this asset may not report on the shared key.
An active credential always denies it: the PC has its own secret and a
shared-key call for it is either an older agent or somebody else.
A revoked credential denies it only while revokeddeniessharedkey is on.
Today it does not, on purpose: revoke is also how an admin rotates a
credential or rolls a PC back to an agent that only knows the shared key
(revoke + blockreenroll), and denying by default would silence that PC
instead. A PC that never enrolled has no credential row and is never denied,
which is what keeps the un-migrated fleet reporting.
"""
from ..models import EndpointCredential
if assetid is None:
return False
row = (EndpointCredential.query
.filter(EndpointCredential.assetid == assetid)
.first())
if row is None:
return False
if row.revokedat is None:
return True
return revoked_denies_shared_key()
def reenroll_blocked(assetid):
"""True when an admin blocked this asset from enrolling again."""
from ..models import EndpointCredential
@@ -130,8 +175,7 @@ def shared_key_refused_for(hostname):
"""True when the shared key may not report for this hostname."""
if not shared_key_allowed():
return True
assetid = resolve_hostname(hostname)
return assetid is not None and active_credential_for(assetid) is not None
return shared_key_denied_for(resolve_hostname(hostname))
def credential_invalid_response():
@@ -148,6 +192,20 @@ def shared_key_refused_response():
http_code=401)
def shared_key_revoked_response():
"""Same code and status; the operator needs to know which refusal it is.
The PC holds no credential and cannot use the shared key either, so an
admin has to let it enroll again. Nothing changes for the agent, which
keeps retrying either way.
"""
return error_response(
CODE_SHARED_KEY_REFUSED,
'This endpoint had its credential revoked, so the shared key is not '
'accepted for it; let it enroll again',
http_code=401)
def asset_mismatch_response():
return error_response(
CODE_ASSET_MISMATCH,
@@ -210,7 +268,8 @@ def bind_hostname(hostname):
ASSET_MISMATCH, a hostname on no computer record at all is refused with the
same code but its own message, and a missing hostname is fine.
Shared key: the hostname's asset (None when absent or unknown, for the
route to answer); an enrolled PC is refused.
route to answer); an enrolled PC is refused, and so is one whose credential
was revoked while revokeddeniessharedkey is on.
"""
hostname = (hostname or '').strip()
if is_credential_caller():
@@ -225,7 +284,9 @@ def bind_hostname(hostname):
if not hostname:
return None, None
assetid = resolve_hostname(hostname)
if assetid is not None and active_credential_for(assetid) is not None:
if shared_key_denied_for(assetid):
if active_credential_for(assetid) is None:
return None, shared_key_revoked_response()
return None, shared_key_refused_response()
g.agentassetid = assetid
return assetid, None

295
plugins/endpoints/api/bootstrap.py
View File

@@ -14,14 +14,25 @@ Routes
POST /siteagent/bootstrap/tokens admin, audited; secret once
GET /siteagent/bootstrap/tokens admin; never a secret
POST /siteagent/bootstrap/tokens/<id>/revoke admin, audited
GET /siteagent/bootstrap.ps1?t=... token only; not a use
GET /siteagent/bootstrap/package?t=... token only; one use
GET /siteagent/bootstrap.ps1 token only; not a use
GET /siteagent/bootstrap/package token only; one use
The two token-authenticated routes answer 404 for anything not currently
usable - unknown, expired, revoked or exhausted - so the URL confirms nothing
about which tokens exist. Both are per-IP rate limited with the same fixed
window shape the login and payload-download limiters use, and both write an
audit row carrying the caller's IP.
The two token-authenticated routes take the token in the
X-Endpoint-Bootstrap-Token header, and still accept the deprecated ?t= form so
a command already pasted on a bay keeps working. They answer 404 for anything
not currently usable - unknown, malformed, expired, revoked, exhausted, or no
token presented at all - so the URL confirms nothing about which tokens exist,
nor about whether the query form is still accepted here. The one 400 is a real
query token offered at a site that has turned the query form off.
The admin UI's Download .ps1 link has to use the query form (a browser link
cannot send a header), so its URL carries a marker that tells it apart from a
bay still running a command pasted before this change. Only the bay logs the
deprecation warning, which is what the rollout counts un-migrated bays with.
Both routes are per-IP rate limited with the same fixed window shape the login
and payload-download limiters use, and both write an audit row carrying the
caller's IP.
"""
import hashlib
@@ -60,12 +71,60 @@ LIST_LIMIT = 100
SCRIPT_PATH = '/api/endpoints/siteagent/bootstrap.ps1'
PACKAGE_PATH = '/api/endpoints/siteagent/bootstrap/package'
# Where the one-line command drops the script before running it. A literal, not
# $env:TEMP: the line has to survive being pasted into an already-running
# Where the one-line command stages the script before running it. A literal,
# not $env:TEMP: the line has to survive being pasted into an already-running
# PowerShell, which expands every $ inside the outer double quotes before the
# child process sees them. The install needs elevation anyway, so this path is
# writable by definition.
COMMAND_SCRIPT_PATH = r'C:\Windows\Temp\shopdb-bootstrap.ps1'
# child process sees them.
#
# NOT C:\Windows\Temp. That directory grants Users create-file and gives
# CREATOR OWNER full control of what they create, so any standard user on the
# bay can pre-plant this exact path and wait for a tech to paste the command,
# which then runs the planted content elevated. Program Files is admin-only on
# a stock install, the command re-asserts an admin-only DACL on the directory
# it creates, and it refuses to run a staged file owned by anyone else.
COMMAND_STAGE_DIR = r'C:\Program Files\ShopDB\bootstrap'
COMMAND_SCRIPT_PATH = COMMAND_STAGE_DIR + r'\shopdb-bootstrap.ps1'
# Well-known SIDs, not names: the bays are domain-joined and a renamed or
# localized Administrators group would quietly break both the grant and the
# ownership test.
ADMINISTRATORS_SID = 'S-1-5-32-544'
SYSTEM_SID = 'S-1-5-18'
# The token the two public routes accept. A header, not ?t=: a query string
# lands verbatim in the reverse proxy and gunicorn access logs, and a live
# install token has no business sitting in a log file.
BOOTSTRAP_TOKEN_HEADER = 'X-Endpoint-Bootstrap-Token'
QUERY_TOKEN_ARG = 't'
# Where the token on this request came from. Kept apart because the rollout
# counts un-migrated bays by the deprecation warning SOURCE_QUERY logs: an
# admin clicking Download .ps1 in the browser must not land in that count, and
# a request carrying no token at all must not answer differently from one
# carrying a token nobody issued.
SOURCE_HEADER = 'header'
SOURCE_QUERY = 'query'
SOURCE_UI_LINK = 'uilink'
SOURCE_NONE = 'none'
# The marker on the admin Download .ps1 link. A hint, NOT authentication: a
# caller can add it by hand, and nothing is granted by it. What matters is the
# other direction - a one-line command generated before this change cannot
# carry it, so a request without it really is a bay to go and re-issue.
UI_LINK_ARG = 'via'
UI_LINK_VALUE = 'download'
# DEPRECATED, kept on purpose: every command already pasted on a bay, and the
# UI's "Download .ps1" link (a browser link cannot send a header), carry the
# token in the query string. Accepting it stays the default so a mid-rollout
# bay keeps working; a site that has re-issued its live commands turns it off.
# Turning it off also disables the download link.
#
# Declared in the endpoints plugin's get_settings_defaults() (so the row seeds
# boolean, not string) and flipped from Settings > Endpoints > Build site agent
# > One-line install, "Accept the token in the URL". A switch with no seed and
# no control would be inert on an air-gapped box.
QUERY_TOKEN_SETTING = 'plugin.endpoints.bootstrapquerytoken'
# Fixed-window per-IP limiter, same shape and cache extension as the login
# limiter in shopdb.core.api.auth and the GE-Enforce payload limiter. A tech
@@ -86,6 +145,22 @@ def _naive_utcnow():
def _caller_ip():
"""The caller address for the audit rows and the deprecation warning.
Read the value before trusting it to identify a bay. client_ip() returns
remote_addr and nothing else. On the shipped Windows deploy IIS launches
waitress on loopback through HttpPlatformHandler, and deploy/windows/
web.config ships the X-Forwarded-For rewrite rule COMMENTED OUT - IIS sets
no such header on its own, so waitress's --trusted-proxy=127.0.0.1
--trusted-proxy-count=1 has nothing to rewrite REMOTE_ADDR from and leaves
it as the loopback peer. TRUSTED_PROXY_COUNT also defaults to 0, so
ProxyFix is not installed either. Every row and every warning then reads
127.0.0.1, and the un-migrated-bay list identifies no bay.
It is a real address only where the site enabled that rewrite rule (the
installer's -ClientIpSource direct), or set TRUSTED_PROXY_COUNT for a
front proxy that sets the header itself.
"""
return client_ip(request) or 'unknown'
@@ -142,11 +217,31 @@ def bootstrap_base_url():
return 'https://{0}{1}'.format(host, root)
def _base(baseurl=None):
resolved = baseurl if baseurl is not None else bootstrap_base_url()
return resolved.rstrip('/')
def bootstrap_script_url(secret, baseurl=None):
"""The token URL that serves the script: downloaded by the command, and
offered in the UI as a plain file a tech can save and run instead."""
base = (baseurl if baseurl is not None else bootstrap_base_url()).rstrip('/')
return '{0}{1}?t={2}'.format(base, SCRIPT_PATH, secret)
"""The UI's "Download .ps1" link: a plain file a tech can save and run.
Still the query-string form, because an <a download> cannot set a header.
The one-line command no longer uses this shape - see bootstrap_command.
It carries UI_LINK_ARG so the server can tell this click apart from a bay
running a pre-change command. Without that the admin's own Download link
fires the deprecation warning the rollout uses to FIND un-migrated bays,
and the count becomes noise the moment an admin mints a token.
"""
return '{0}{1}?{2}={3}&{4}={5}'.format(_base(baseurl), SCRIPT_PATH,
QUERY_TOKEN_ARG, secret,
UI_LINK_ARG, UI_LINK_VALUE)
def bootstrap_fetch_url(path, baseurl=None):
"""A token-free URL for the machine-to-machine fetches. The token rides in
BOOTSTRAP_TOKEN_HEADER so it never reaches an access log."""
return '{0}{1}'.format(_base(baseurl), path)
def bootstrap_command(secret, baseurl=None):
@@ -158,28 +253,145 @@ def bootstrap_command(secret, baseurl=None):
Single quotes inside and no $ anywhere, so the one line means the same
whether it is pasted into cmd.exe or into a PowerShell prompt. The download
and the run are wrapped in try/finally so the dropped file goes away even
and the run are wrapped in try/finally so the staged file goes away even
when the script exits non-zero or the download itself fails.
Staging is the part worth being careful about, so the line creates
COMMAND_STAGE_DIR with an admin-only DACL, clears any leftover file, and
refuses (exit 5) to run a staged script that some other principal owns -
which is what a pre-planted file looks like. Well-known SIDs throughout,
and the owner may also be the elevated caller, since a file created by an
administrator is owned by that account rather than by the group.
The hardening is also CHECKED (exit 6). icacls is a native command, so it
reports failure only in an exit code: no exception is raised, and
$ErrorActionPreference never applies to it. Unchecked, an icacls that
cannot write the DACL leaves the directory inheriting whatever the parent
grants and the line downloads into it anyway - the silent version of the
hole this staging move was made to close. The same check also covers the
New-Item before it, because icacls on a directory that was never created
exits non-zero too.
Read as `Get-Variable LASTEXITCODE -ValueOnly`, not `$LASTEXITCODE`: the
whole line must stay free of `$`, or a paste into an already-running
PowerShell expands it in the PARENT and the child is handed `-ne 0` with
the parent's last exit code baked in.
"""
return ('powershell -ExecutionPolicy Bypass -c "'
"try {{ Invoke-WebRequest -UseBasicParsing '{0}' -OutFile '{1}'; "
"& '{1}' }} finally {{ Remove-Item '{1}' -Force "
'-ErrorAction SilentlyContinue }}"'
).format(bootstrap_script_url(secret, baseurl),
COMMAND_SCRIPT_PATH)
return (
'powershell -ExecutionPolicy Bypass -c "'
"try {{ New-Item -ItemType Directory -Force -Path '{stagedir}' "
'| Out-Null; '
"icacls '{stagedir}' /inheritance:r /grant '*{admin}:(OI)(CI)F' "
"'*{system}:(OI)(CI)F' | Out-Null; "
'if ((Get-Variable LASTEXITCODE -ValueOnly) -ne 0) '
"{{ Write-Host 'Refusing to run: could not restrict "
"{stagedir} to administrators (icacls failed). Fix the directory "
"or delete it, then run the command again.' -ForegroundColor Red; "
'exit 6 }}; '
"Remove-Item '{path}' -Force -ErrorAction SilentlyContinue; "
"Invoke-WebRequest -UseBasicParsing '{url}' "
"-Headers @{{'{header}'='{secret}'}} -OutFile '{path}'; "
"if ((Get-Acl '{path}').GetOwner("
'[Security.Principal.SecurityIdentifier]).Value -notin '
"@('{admin}','{system}',"
'[Security.Principal.WindowsIdentity]::GetCurrent().User.Value)) '
"{{ Write-Host 'Refusing to run: the staged bootstrap script is not "
"owned by an administrator.' -ForegroundColor Red; exit 5 }}; "
"& '{path}' }} finally {{ Remove-Item '{path}' -Force "
'-ErrorAction SilentlyContinue }}"'
).format(stagedir=COMMAND_STAGE_DIR,
path=COMMAND_SCRIPT_PATH,
url=bootstrap_fetch_url(SCRIPT_PATH, baseurl),
header=BOOTSTRAP_TOKEN_HEADER,
secret=secret,
admin=ADMINISTRATORS_SID,
system=SYSTEM_SID)
def _hash_token(secret):
return hashlib.sha256(secret.encode('utf-8')).hexdigest()
def _resolve_token():
"""The usable token named by ?t=, else None.
def _presented_token():
"""(secret, source) for the token this request carries.
Header first, query string second. The query form is deprecated but still
the only one a browser download link can use.
SOURCE_NONE when nothing was presented, which is NOT the same as an empty
query token: reporting a bare GET as 'query' made it answer 400 naming the
header, telling an anonymous caller whether this site had turned the query
form off, and contradicting the 404-for-everything-unusable rule at the top
of this module.
"""
header = (request.headers.get(BOOTSTRAP_TOKEN_HEADER) or '').strip()
if header:
return header, SOURCE_HEADER
query = (request.args.get(QUERY_TOKEN_ARG) or '').strip()
if not query:
return '', SOURCE_NONE
if request.args.get(UI_LINK_ARG) == UI_LINK_VALUE:
return query, SOURCE_UI_LINK
return query, SOURCE_QUERY
def _is_query_source(source):
"""Both query-string forms, the bay's and the admin download link's. The
setting is about a token in an access log, and the admin link puts one
there just as surely as a pasted command does."""
return source in (SOURCE_QUERY, SOURCE_UI_LINK)
def _query_token_allowed():
"""Defaults to True: today's pasted commands and the UI download link both
use the query form, and turning that off mid-rollout strands a bay."""
return bool(Setting.get(QUERY_TOKEN_SETTING, True))
def _query_token_refused():
"""Answered before any lookup, so it confirms nothing about any token."""
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Send the bootstrap token in the {0} header.'.format(
BOOTSTRAP_TOKEN_HEADER),
http_code=400)
def _note_token_source(source):
"""Audit suffix for a token that really resolved, and, for a query-string
token, one log line naming the caller and WHICH query path it was.
Only the bay warns. That warning is the rollout's un-migrated-bay counter
(see PER-PC-CREDENTIALS-ROLLOUT.md), so the admin's own Download .ps1 link
logs at info instead - otherwise every mint-and-test by an admin adds a
bay that does not exist.
Never the token itself, in either line: keeping it out of the logs is the
whole point of the header form.
"""
if source == SOURCE_UI_LINK:
current_app.logger.info(
'bootstrap script fetched by %s from the admin Download .ps1 '
'link; query-string token, but the admin browser path, NOT an '
'un-migrated bay', _caller_ip())
return ' (query-string token, admin download link)'
if source == SOURCE_QUERY:
current_app.logger.warning(
'bootstrap token presented in the query string by %s with no '
'admin download marker, so this is an un-migrated bay running a '
'command pasted before the header form; that form is deprecated '
'and slated for removal, re-issue the one-line install command',
_caller_ip())
return ' (query-string token, deprecated)'
return ''
def _resolve_token(secret):
"""The usable token with this secret, else None.
Unknown, malformed, expired, revoked and exhausted all resolve to None so
the caller can answer 404 for every one of them.
"""
secret = (request.args.get('t') or '').strip()
if not _TOKEN_PATTERN.match(secret):
return None
row = EndpointBootstrapToken.query.filter_by(
@@ -218,6 +430,7 @@ _SCRIPT_TEMPLATE = '''# ShopDB endpoint agent bootstrap
$ErrorActionPreference = "Stop"
$BootstrapPackageUrl = {packageurl}
$BootstrapPackageHeaders = @{{ {tokenheader} = {token} }}
$BootstrapServiceName = "ShopDbEndpointAgent"
$BootstrapInstallDir = "C:\\Program Files\\ShopDB\\EndpointAgent"
$BootstrapAgentSha256 = {agentsha}
@@ -308,7 +521,7 @@ try {{
New-Item -ItemType Directory -Force -Path $bootstrapTempDir | Out-Null
Write-Host "[1/4] Downloading the agent package ..."
Invoke-WebRequest -Uri $BootstrapPackageUrl -OutFile $bootstrapZipPath -UseBasicParsing
Invoke-WebRequest -Uri $BootstrapPackageUrl -Headers $BootstrapPackageHeaders -OutFile $bootstrapZipPath -UseBasicParsing
Write-Host "[2/4] Expanding and verifying ..."
Expand-Archive -Path $bootstrapZipPath -DestinationPath $bootstrapExtractDir -Force
@@ -394,11 +607,12 @@ exit 0
def build_bootstrap_script(secret, baseurl=None):
"""The PowerShell text served for one token, with the URL and hashes baked in."""
base = (baseurl if baseurl is not None else bootstrap_base_url()).rstrip('/')
base = _base(baseurl)
hashes = siteagent.staged_binary_hashes()
return _SCRIPT_TEMPLATE.format(
packageurl=json.dumps('{0}{1}?t={2}'.format(base, PACKAGE_PATH,
secret)),
packageurl=json.dumps(bootstrap_fetch_url(PACKAGE_PATH, base)),
tokenheader=json.dumps(BOOTSTRAP_TOKEN_HEADER),
token=json.dumps(secret),
agentsha=json.dumps(hashes.get('agent', '')),
traysha=json.dumps(hashes.get('tray', '')),
forcevariable=FORCE_VARIABLE,
@@ -500,7 +714,8 @@ def revoke_bootstrap_token(tokenid):
# ---------------------------------------------------------------------------
# Token-authenticated: no login, the token in ?t= is the whole authorization
# Token-authenticated: no login, the token is the whole authorization. It
# rides in the X-Endpoint-Bootstrap-Token header; ?t= is still accepted.
# ---------------------------------------------------------------------------
@endpoints_bp.route('/siteagent/bootstrap.ps1', methods=['GET'])
@@ -516,12 +731,15 @@ def serve_bootstrap_script():
"""
if _ratelimited():
return _rate_limit_response()
row = _resolve_token()
secret, source = _presented_token()
if _is_query_source(source) and not _query_token_allowed():
return _query_token_refused()
row = _resolve_token(secret)
if row is None:
return _not_found()
secret = (request.args.get('t') or '').strip()
_audit('requested', row, 'bootstrap script served')
_audit('requested', row,
'bootstrap script served' + _note_token_source(source))
db.session.commit()
return Response(build_bootstrap_script(secret), mimetype='text/plain')
@@ -536,7 +754,10 @@ def download_bootstrap_package():
"""
if _ratelimited():
return _rate_limit_response()
row = _resolve_token()
secret, source = _presented_token()
if _is_query_source(source) and not _query_token_allowed():
return _query_token_refused()
row = _resolve_token(secret)
if row is None:
return _not_found()
@@ -544,8 +765,8 @@ def download_bootstrap_package():
row.usecount = (row.usecount or 0) + 1
row.lastusedat = now
row.lastusedip = _caller_ip()[:45]
_audit('downloaded', row, 'bootstrap package, use {0} of {1}'.format(
row.usecount, row.maxuses))
_audit('downloaded', row, 'bootstrap package, use {0} of {1}{2}'.format(
row.usecount, row.maxuses, _note_token_source(source)))
db.session.commit()
buffer, warning = siteagent.build_keyless_package(bootstrap_base_url())

119
plugins/endpoints/api/enrollment.py
View File

@@ -78,6 +78,14 @@ ENROLL_RATELIMIT_WINDOW_SECONDS = 60
PENDING_CAP_SETTING = 'plugin.endpoints.maxpendingenrollments'
DEFAULT_PENDING_CAP = 500
# How long a pending request may go untouched before the cap treats it as
# abandoned. An agent waiting for approval re-asks about once a minute, so a row
# nothing has touched for a week is a PC that was re-imaged, renamed or scrapped,
# or a junk row somebody wrote at the open door. At the cap those are dropped to
# make room before a PC nobody has seen before is refused, so a queue that filled
# with abandoned rows drains itself instead of blocking every new PC on site.
PENDING_STALE_AFTER = timedelta(days=7)
_NONCE_PATTERN = re.compile(r'^[A-Za-z0-9_-]{32,128}$')
REASON_UNKNOWN_HOSTNAME = 'unknown hostname'
@@ -265,6 +273,38 @@ def _pending_queue_is_full():
return pending >= cap
def _drop_stale_pending():
"""Delete pending requests nothing has touched in PENDING_STALE_AFTER.
Only PENDING rows: an admin decision (approved, rejected, issued) is never
aged out, so a rejection keeps standing however long the PC argues with it.
Returns how many rows went, for the caller to decide whether the cap is
still reached.
"""
cutoff = _naive_utcnow() - PENDING_STALE_AFTER
stale = (EndpointEnrollRequest.query
.filter(EndpointEnrollRequest.status
== EndpointEnrollRequest.STATUS_PENDING,
EndpointEnrollRequest.lastseenat < cutoff)
.order_by(EndpointEnrollRequest.lastseenat.asc())
.all())
if not stale:
return 0
for row in stale:
db.session.delete(row)
current_app.logger.info(
'endpoint enroll: aged out %d pending requests untouched for %d days',
len(stale), PENDING_STALE_AFTER.days)
AuditLog.log(
'deleted', 'EndpointEnrollRequest',
entityname=stale[0].hostname,
details='aged out {0} pending enroll requests untouched for {1} '
'days, to make room under the pending cap'.format(
len(stale), PENDING_STALE_AFTER.days))
db.session.commit()
return len(stale)
def _pending_queue_full_response(hostname, serialnumber):
"""Refuse a new row, say so plainly in the log and leave an audit trail."""
cap = pending_cap()
@@ -349,15 +389,42 @@ def enroll_endpoint():
if row is None:
# First contact for this hostname and serial. This is the only branch
# that grows the table, so it is the only one the cap guards.
# that grows the table, so it is the only one the cap guards. Every
# other contact reuses the one row this identity already owns, which is
# what keeps a single PC from holding a queue's worth of them.
if _pending_queue_is_full():
return _pending_queue_full_response(hostname, serialnumber)
# Abandoned rows come out before a real PC is turned away.
_drop_stale_pending()
if _pending_queue_is_full():
return _pending_queue_full_response(hostname, serialnumber)
row = EndpointEnrollRequest(
hostname=hostname, serialnumber=serialnumber, enrollnonce=nonce,
status=EndpointEnrollRequest.STATUS_PENDING, requestedat=now)
db.session.add(row)
else:
samenonce = hmac.compare_digest(row.enrollnonce, nonce)
if row.status == EndpointEnrollRequest.STATUS_REJECTED:
# A rejection binds to the IDENTITY - this hostname and serial - not
# to the nonce it was rejected with. A new installation on the same
# PC brings a new nonce, and re-arming the row on it would put the
# request back in the queue and, with the shared key and auto-enroll
# on, hand the retry the credential an admin had just refused.
# Nothing the PC sends clears a rejection; an admin approving the
# request does.
#
# The nonce is still recorded, because a rejected row hands nothing
# out and an admin who does approve it later has to bind that
# approval to the installation asking NOW, not to the one that was
# refused weeks ago.
row.enrollnonce = nonce
row.lastseenat = now
row.sourceip = sourceip
if agentversion:
row.agentversion = agentversion
db.session.commit()
return success_response(
{'status': 'pending', 'requestid': row.requestid,
'reason': REASON_REJECTED}, http_code=202)
if row.status == EndpointEnrollRequest.STATUS_APPROVED:
row.lastseenat = now
row.sourceip = sourceip
@@ -380,16 +447,10 @@ def enroll_endpoint():
row.lastseenat = now
row.sourceip = sourceip
return _pending(row, REASON_NONCE_CHANGED, row.matchedassetid)
if row.status == EndpointEnrollRequest.STATUS_REJECTED and samenonce:
row.lastseenat = now
row.sourceip = sourceip
db.session.commit()
return success_response(
{'status': 'pending', 'requestid': row.requestid,
'reason': REASON_REJECTED}, http_code=202)
if row.status != EndpointEnrollRequest.STATUS_PENDING:
# Issued earlier (credential since revoked) or rejected for another
# installation: a new enrollment cycle bound to this nonce.
# Issued earlier and the credential has since been revoked: a new
# enrollment cycle bound to this nonce. Rejected rows never reach
# here - they returned above, whatever nonce they carried.
row.enrollnonce = nonce
row.requestedat = now
row.decidedat = None
@@ -574,6 +635,7 @@ def _page(rows):
def _settings_payload():
return {'sharedkeyallowed': agentauth.shared_key_allowed(),
'autoenroll': agentauth.auto_enroll_enabled(),
'revokeddeniessharedkey': agentauth.revoked_denies_shared_key(),
'maxpendingenrollments': pending_cap()}
@@ -667,6 +729,11 @@ def approve_enroll_request(requestid):
The pending reason for a one-off PC is by definition `unknown hostname`, so
this is the normal case, not the exception.
Approving is also the only way out of a rejection: a rejected request keeps
refusing its PC whatever nonce it sends, and this is the deliberate admin
step that lifts it. The approval binds to the nonce of that PC's most recent
contact, so an agent reinstalled while it was refused still collects it.
Body: {"assetid": 123, "sethostname": false}. A computer with no hostname
is stamped with the request's. A computer carrying a DIFFERENT hostname is
refused with 409 naming both, until the caller repeats the approval with
@@ -734,6 +801,7 @@ def approve_enroll_request(requestid):
details='hostname {0} from enroll request {1}'.format(
hostnameresult, row.requestid))
wasrejected = row.status == EndpointEnrollRequest.STATUS_REJECTED
row.status = EndpointEnrollRequest.STATUS_APPROVED
row.matchedassetid = assetid
row.reason = None
@@ -741,8 +809,9 @@ def approve_enroll_request(requestid):
row.decidedby = _username()
AuditLog.log('updated', 'EndpointEnrollRequest', entityid=row.requestid,
entityname=row.hostname,
details='enroll request approved for asset {0}, hostname {1}'
.format(assetid, hostnameresult))
details='enroll request {0} for asset {1}, hostname {2}'
.format('approved out of a rejection' if wasrejected
else 'approved', assetid, hostnameresult))
db.session.commit()
approved = row.to_dict()
approved['hostnameresult'] = hostnameresult
@@ -780,6 +849,11 @@ def revoke_endpoint_credential(assetid):
before rolling a PC back to an agent that only knows the shared key: without
it the current agent re-enrolls within seconds and the older agent is then
refused. A PC whose credential is already revoked can still be blocked.
Whether a revoked PC may still report on the shared key is the site setting
plugin.endpoints.revokeddeniessharedkey. It is off here, so today it can -
which is what makes the rollback above work, and what keeps a rotation from
silencing the PC while it waits for approval.
"""
payload = request.get_json(silent=True) or {}
if not isinstance(payload, dict):
@@ -853,12 +927,24 @@ def allow_endpoint_reenroll(assetid):
@endpoints_bp.route('/enrollment/settings', methods=['PUT'])
@require_role('admin')
def update_enrollment_settings():
"""Change the enrollment settings.
revokeddeniessharedkey is off by default, which is what the fleet does
today: a PC whose credential was revoked falls back to the shared key, so a
credential rotation, and the rollback to an older agent that revoke +
blockreenroll exists for, both keep reporting. Turning it on makes a revoke
an eviction - that PC reports nothing until it is allowed to enroll again -
and is only safe once every PC here holds its own credential.
"""
from shopdb.api import Setting
payload = request.get_json(silent=True) or {}
changes = {}
for key, settingkey in (('sharedkeyallowed', agentauth.SHARED_KEY_SETTING),
('autoenroll', agentauth.AUTO_ENROLL_SETTING)):
for key, settingkey in (
('sharedkeyallowed', agentauth.SHARED_KEY_SETTING),
('autoenroll', agentauth.AUTO_ENROLL_SETTING),
('revokeddeniessharedkey',
agentauth.REVOKED_DENIES_SHARED_KEY_SETTING)):
if key not in payload:
continue
if not isinstance(payload[key], bool):
@@ -878,7 +964,8 @@ def update_enrollment_settings():
if not changes:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Send sharedkeyallowed, autoenroll and/or maxpendingenrollments')
'Send sharedkeyallowed, autoenroll, revokeddeniessharedkey '
'and/or maxpendingenrollments')
before = _settings_payload()
for key, (settingkey, value, valuetype) in changes.items():

62
plugins/endpoints/api/siteagent.py
View File

@@ -410,6 +410,57 @@ function Set-HklmValueBothViews {{
}}
}}
# Lock SOFTWARE\\GE\\ShopDB to SYSTEM and Administrators, in both views.
#
# WHY. The shared fleet key is written here and stays until every consumer is
# on a per-PC credential, and the DACL this key inherits by default grants
# BUILTIN\\Users read - so any logged-on user on any bay could read the fleet
# key, which is the exact exposure the per-PC credential migration exists to
# close. Everything that legitimately reads this key runs as SYSTEM (the agent
# service, the GE-Enforce reporters) or as an admin (the adoption tool,
# Set-ShopDBServerPins.ps1). The tray reads no registry at all - it talks to the
# service over the named pipe - so nothing loses access here.
#
# It also protects ServerPins from a standard user, which is what makes the
# registry safe to treat as authoritative for pinning.
function Protect-HklmShopdbKey {{
param([Parameter(Mandatory)] [string] $SubKey)
$rights = [System.Security.AccessControl.RegistryRights]
foreach ($view in @([Microsoft.Win32.RegistryView]::Registry64,
[Microsoft.Win32.RegistryView]::Registry32)) {{
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey(
[Microsoft.Win32.RegistryHive]::LocalMachine, $view)
try {{
$key = $base.OpenSubKey(
$SubKey,
[Microsoft.Win32.RegistryKeyPermissionCheck]::ReadWriteSubTree,
$rights::ChangePermissions -bor $rights::ReadPermissions)
if (-not $key) {{ continue }}
try {{
$acl = $key.GetAccessControl(
[System.Security.AccessControl.AccessControlSections]::Access)
# Break inheritance and DROP the inherited entries rather than
# copying them in: copying is what silently preserves the
# BUILTIN\\Users read this function exists to remove.
$acl.SetAccessRuleProtection($true, $false)
foreach ($rule in @($acl.Access)) {{ $null = $acl.RemoveAccessRule($rule) }}
foreach ($sid in @("S-1-5-18", "S-1-5-32-544")) {{
$who = New-Object System.Security.Principal.SecurityIdentifier($sid)
$acl.AddAccessRule((New-Object System.Security.AccessControl.RegistryAccessRule(
$who,
$rights::FullControl,
[System.Security.AccessControl.InheritanceFlags]::ContainerInherit,
[System.Security.AccessControl.PropagationFlags]::None,
[System.Security.AccessControl.AccessControlType]::Allow)))
}}
$key.SetAccessControl($acl)
}}
finally {{ $key.Dispose() }}
}}
finally {{ $base.Dispose() }}
}}
}}
# REG_MULTI_SZ in both views, compare-before-write so a reinstall with the same
# pins leaves the value (and its last-write time) alone.
function Set-HklmMultiStringBothViews {{
@@ -492,6 +543,17 @@ else {{
Write-Warning "no server pin in this package; ServerPins left as is."
}}
# Harden the key AFTER the writes: the DACL is replaced, not merged, so writing
# first and locking second keeps this correct whether or not the key pre-existed.
try {{
Protect-HklmShopdbKey -SubKey "SOFTWARE\\GE\\ShopDB"
Write-Host " HKLM\\SOFTWARE\\GE\\ShopDB restricted to SYSTEM and Administrators."
}}
catch {{
Write-Warning "could not restrict HKLM\\SOFTWARE\\GE\\ShopDB: $($_.Exception.Message)"
Write-Warning "the shared key on this PC stays readable by any logged-on user until that is fixed."
}}
# 1a. Trust the bundled server certificate, if one shipped. The agent's outbound
# TLS to a self-signed or internal-CA server fails closed until the machine
# trusts the cert. Importing into LocalMachine\\Root covers the SYSTEM service.

71
plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md
View File

@@ -292,17 +292,63 @@ says so when it finds an agent already installed.
a new one if it is lost.
3. On the target PC, open PowerShell **as administrator** and paste it:
**Copy the command the UI gives you.** Do not retype it from here and do not
reuse one you saved from an older round - see the warning under this block. It
looks like this:
```
powershell -ExecutionPolicy Bypass -c "try { Invoke-WebRequest -UseBasicParsing 'https://<server>/api/endpoints/siteagent/bootstrap.ps1?t=<token>' -OutFile 'C:\Windows\Temp\shopdb-bootstrap.ps1'; & 'C:\Windows\Temp\shopdb-bootstrap.ps1' } finally { Remove-Item 'C:\Windows\Temp\shopdb-bootstrap.ps1' -Force -ErrorAction SilentlyContinue }"
powershell -ExecutionPolicy Bypass -c "try { New-Item -ItemType Directory -Force -Path 'C:\Program Files\ShopDB\bootstrap' | Out-Null; icacls 'C:\Program Files\ShopDB\bootstrap' /inheritance:r /grant '*S-1-5-32-544:(OI)(CI)F' '*S-1-5-18:(OI)(CI)F' | Out-Null; if ((Get-Variable LASTEXITCODE -ValueOnly) -ne 0) { Write-Host 'Refusing to run: could not restrict C:\Program Files\ShopDB\bootstrap to administrators (icacls failed). Fix the directory or delete it, then run the command again.' -ForegroundColor Red; exit 6 }; Remove-Item 'C:\Program Files\ShopDB\bootstrap\shopdb-bootstrap.ps1' -Force -ErrorAction SilentlyContinue; Invoke-WebRequest -UseBasicParsing 'https://<server>/api/endpoints/siteagent/bootstrap.ps1' -Headers @{'X-Endpoint-Bootstrap-Token'='<token>'} -OutFile 'C:\Program Files\ShopDB\bootstrap\shopdb-bootstrap.ps1'; if ((Get-Acl 'C:\Program Files\ShopDB\bootstrap\shopdb-bootstrap.ps1').GetOwner([Security.Principal.SecurityIdentifier]).Value -notin @('S-1-5-32-544','S-1-5-18',[Security.Principal.WindowsIdentity]::GetCurrent().User.Value)) { Write-Host 'Refusing to run: the staged bootstrap script is not owned by an administrator.' -ForegroundColor Red; exit 5 }; & 'C:\Program Files\ShopDB\bootstrap\shopdb-bootstrap.ps1' } finally { Remove-Item 'C:\Program Files\ShopDB\bootstrap\shopdb-bootstrap.ps1' -Force -ErrorAction SilentlyContinue }"
```
It downloads the script, runs it and deletes it again. Do not shorten it back
to `irm ... | iex`: Defender scores that shape as `Trojan:Win32/Commando.A!ml`
and kills the process before anything runs, on a default install with no ASR
rules. Paste it as one line into an elevated **cmd** or **PowerShell** window;
it means the same in both. **Download .ps1** in the same block saves the script
instead, for a PC where pasting a long line is awkward - it carries the same
token, so treat the file the way you treat the command.
It creates an admin-only staging directory, downloads the script into it, runs
it and deletes it again. Do not shorten it back to `irm ... | iex`: Defender
scores that shape as `Trojan:Win32/Commando.A!ml` and kills the process before
anything runs, on a default install with no ASR rules. Paste it as one line
into an elevated **cmd** or **PowerShell** window; it means the same in both.
**Download .ps1** in the same block saves the script instead, for a PC where
pasting a long line is awkward - it carries the same token, so treat the file
the way you treat the command.
> **Do not use the old `?t=<token>` one-liner that staged into
> `C:\Windows\Temp`.** Any standard user on the bay can create that exact path
> first and wait for a tech to paste the command, which then runs the planted
> file elevated; the token also lands in the web server's access log. The
> command above stages under `C:\Program Files`, locks the directory to
> Administrators and SYSTEM and stops if that fails, refuses to run a staged
> file owned by anyone else, and sends the token in a header.
>
> The `?t=` form is still ACCEPTED, but only so a bay provisioned before this
> change keeps working until its command is re-issued. Every such fetch logs a
> server warning and writes an audit row against the token:
>
> ```
> bootstrap token presented in the query string by <caller> with no admin
> download marker, so this is an un-migrated bay running a command pasted
> before the header form; ... re-issue the one-line install command
> ```
>
> **`<caller>` is 127.0.0.1 on a stock Windows install.** IIS runs the app
> through HttpPlatformHandler over loopback and sets no `X-Forwarded-For` of
> its own; `deploy/windows/web.config` ships the rewrite rule that would add
> one commented out, so waitress has nothing to rewrite `REMOTE_ADDR` from.
> Check before you rely on the address: if these lines all say 127.0.0.1, the
> warning still tells you un-migrated bays EXIST and how many fetches they
> made, but not which bays. Get real addresses by enabling that rule (the
> Windows installer's `-ClientIpSource direct`, which installs URL Rewrite and
> allows `HTTP_X_FORWARDED_FOR` at server level); behind a further reverse
> proxy leave it off and set `TRUSTED_PROXY_COUNT` instead. Without that, the
> usable identifier is the token: each warning has a matching audit row on
> `EndpointBootstrapToken` carrying the 12-character `tokenprefix`, so you can
> at least tell which install round the stale command came from.
>
> Work that list down: mint a new token and hand out the new command. Clicking
> **Download .ps1** yourself does NOT add to it - that link is marked
> `&via=download` and logs at info, precisely so the warnings stay a list of
> bays. Once the list is empty, clear **Accept the token in the URL** in
> Settings -> Endpoints -> Build site agent -> One-line install
> (`plugin.endpoints.bootstrapquerytoken`) to refuse the query form outright.
> That also disables the Download .ps1 link and kills every command still in a
> technician's hands, so do it deliberately and re-issue first.
The token is short-lived and only fetches the installer. It carries no shared
key and provisions nothing by itself: the package it downloads is built with
@@ -324,6 +370,15 @@ window to override). After a successful install it waits up to two minutes for
the credential to land and prints `approved and enrolled` if an admin approves
it in that window; still pending is a normal outcome, not a failure.
If it stops before any of that with a red **Refusing to run**, nothing was
installed and nothing was downloaded. Two cases, both meaning look at the bay
rather than retry:
| Exit | Message | What to do |
|---|---|---|
| 6 | could not restrict `C:\Program Files\ShopDB\bootstrap` to administrators | the ACL could not be set - usually the directory already exists with an owner or DACL you did not put there. Inspect it (`icacls`), delete it, run the command again. |
| 5 | the staged bootstrap script is not owned by an administrator | something else wrote the staged file. Treat the bay as suspect, do not force past it. |
Revoke a token from the same block when the install round is done. Tokens list
with their prefix, expiry and uses; they are also spent automatically
(`usecount` reaching `maxuses`) and expire on their own.

82
plugins/endpoints/frontend/views/AgentKeySettings.vue
View File

@@ -482,6 +482,31 @@
Settings &gt; Endpoints &gt; Enrollment.
</p>
<!-- The deprecated query form of the bootstrap token. Rendered here
because this is the page that mints the command the setting
governs: turning it off invalidates every command already handed
out that carries ?t=. -->
<label class="checkbox-row">
<input
type="checkbox"
class="toggle-querytoken"
:checked="querytokenallowed"
:disabled="querytokenbusy"
@change="onQueryTokenToggle($event)"
/>
<span>
<strong>Accept the token in the URL.</strong> On by default:
the pasted one-line command and the Download .ps1 link above both
send the token as <code>?t=</code>, which is the only form a
browser download link can use. The token lands in the IIS and
proxy logs that way, so the header form
(<code>X-Endpoint-Bootstrap-Token</code>) replaces it. Turn this off
and every install command still in a technician's hands stops
working immediately - re-issue them first.
</span>
</label>
<p v-if="querytokenerror" class="field-error">{{ querytokenerror }}</p>
<table v-if="bootstraptokens.length" class="versions-table">
<thead>
<tr>
@@ -764,6 +789,15 @@ const bootstraptokens = ref([])
const bootstrapmessage = ref('')
const bootstraperror = ref('')
// plugin.endpoints.bootstrapquerytoken, read and written through the core
// settings API: the endpoints plugin has no settings route of its own for it,
// and the key is declared in get_settings_defaults so the upsert on PUT gets
// the boolean valuetype and the description from that declaration.
const QUERY_TOKEN_SETTING = 'plugin.endpoints.bootstrapquerytoken'
const querytokenallowed = ref(true)
const querytokenbusy = ref(false)
const querytokenerror = ref('')
function agentnextpininvalid() {
const pin = agentnextserverpin.value
return !!pin && !/^[0-9a-f]{64}$/.test(pin)
@@ -863,6 +897,7 @@ onMounted(async () => {
}
loadEnrollment()
loadBootstrapTokens()
loadQueryTokenSetting()
try {
const response = await api.get('/endpoints/siteagent')
const status = response.data.data || {}
@@ -878,6 +913,53 @@ onMounted(async () => {
}
})
async function loadQueryTokenSetting() {
try {
const response = await api.get(`/settings/${QUERY_TOKEN_SETTING}`)
// The backend reads it as bool(Setting.get(key, True)), so only an
// explicit false is off. A row seeded as boolean comes back as a real
// false; an older row that was created as a string comes back as 'false'.
const value = response.data.data?.value
querytokenallowed.value = !(value === false || value === 'false')
} catch (loadError) {
// 404 = the row was never seeded (plugin installed before the key was
// declared). The backend default is on, so show on.
if (loadError.response?.status !== 404) console.error(loadError)
querytokenallowed.value = true
}
}
async function onQueryTokenToggle(event) {
const turningOn = event.target.checked
if (!turningOn) {
const answer = prompt(
'Stop accepting the token in the URL?\n\n'
+ 'Every install command already handed out uses ?t= and stops '
+ 'working at once, including the Download .ps1 links. Re-issue them '
+ 'before turning this off.\n\n'
+ 'Type OFF to confirm.')
if (!answer || answer.trim().toUpperCase() !== 'OFF') {
event.target.checked = true
return
}
}
querytokenbusy.value = true
querytokenerror.value = ''
try {
// PUT upserts, so this works on a site whose row was never seeded.
await api.put(`/settings/${QUERY_TOKEN_SETTING}`, { value: turningOn })
querytokenallowed.value = turningOn
} catch (saveError) {
querytokenerror.value = saveError.response?.status === 403
? 'Only an administrator can change this setting'
: 'Could not save the setting'
event.target.checked = querytokenallowed.value
console.error(saveError)
} finally {
querytokenbusy.value = false
}
}
async function loadBootstrapTokens() {
try {
const response = await api.get('/endpoints/siteagent/bootstrap/tokens')

46
plugins/endpoints/frontend/views/EnrollmentSettings.vue
View File

@@ -229,6 +229,27 @@
on record matches. Anything else waits below for an admin.
</span>
</label>
<label class="checkbox-row">
<input
type="checkbox"
class="toggle-revokeddenies"
:checked="enrollmentSettings.revokeddeniessharedkey"
:disabled="!isadmin || busy"
@change="onRevokedDeniesToggle($event)"
/>
<span>
<strong>A revoked PC loses the shared key too.</strong> (setting
<code>plugin.endpoints.revokeddeniessharedkey</code>) Off by
default, which is what the fleet does today: a PC whose credential
was revoked falls back to the shared key and keeps reporting, so a
credential rotation does not strand it. Turn this on and a revoke
becomes an eviction - that PC reports nothing and receives no
commands until it enrolls again - and the documented rollback of
revoke plus block re-enrollment onto an older agent that only knows
the shared key stops working. Only safe once every PC here is
enrolled.
</span>
</label>
<label class="number-row">
<span>
<strong>Pending request limit.</strong> How many requests may wait
@@ -670,6 +691,9 @@ const DEFAULT_PENDING_CAP = 500
const enrollmentSettings = ref({
sharedkeyallowed: true,
autoenroll: true,
// Matches the seeded default (plugin.endpoints.revokeddeniessharedkey is
// false), so the box renders unchecked before the first read lands.
revokeddeniessharedkey: false,
maxpendingenrollments: DEFAULT_PENDING_CAP
})
// Bound with v-model rather than :value so the field holds what the operator
@@ -697,6 +721,8 @@ async function loadEnrollment({ quiet = false } = {}) {
enrollmentSettings.value = {
sharedkeyallowed: data.settings?.sharedkeyallowed !== false,
autoenroll: data.settings?.autoenroll !== false,
// Opt-in, so anything but an explicit true reads as off.
revokeddeniessharedkey: data.settings?.revokeddeniessharedkey === true,
maxpendingenrollments: Number.isFinite(Number(data.settings?.maxpendingenrollments))
? Number(data.settings.maxpendingenrollments)
: DEFAULT_PENDING_CAP
@@ -771,6 +797,26 @@ function onAutoEnrollToggle(event) {
})
}
function onRevokedDeniesToggle(event) {
const turningOn = event.target.checked
if (turningOn) {
// Typed confirm, like the shared-key switch: this one silences PCs that
// are reporting right now, and the revoke-plus-rollback recipe in the
// runbook stops working the moment it is on.
const answer = prompt(
'Deny the shared key to revoked PCs?\n\n'
+ 'Any PC whose credential was revoked stops reporting and stops '
+ 'receiving commands until it enrolls again. The revoke + block '
+ 're-enrollment rollback onto an older agent no longer works.\n\n'
+ 'Type DENY to confirm.')
if (!answer || answer.trim().toUpperCase() !== 'DENY') {
event.target.checked = false
return
}
}
return saveSettings({ revokeddeniessharedkey: turningOn })
}
function onPendingCapChange() {
const wanted = Math.trunc(Number(pendingCapInput.value))
if (!Number.isFinite(wanted) || wanted < 0) {

19
plugins/endpoints/models/credential.py
View File

@@ -4,16 +4,25 @@
One row per asset. Holds only the sha256 of the secret the
agent was issued, plus a short display prefix. Revoking sets
revokedat; re-issuing after a revoke UPDATES the same row, so
an asset never carries two credentials. reenrollblocked keeps
an asset never carries two credentials. The row therefore also
says that this PC has enrolled at least once, which is how
agentauth tells a revoked PC from one that never enrolled:
plugin.endpoints.revokeddeniessharedkey refuses the shared key
to the first and never to the second. reenrollblocked keeps
a revoked PC from enrolling again (rolling it back to an agent
that only knows the shared key, or a compromised PC) until an
admin allows it.
EndpointEnrollRequest
One row per (hostname, serial number) that asked to enroll. The
agent's enrollment nonce is recorded on first sight and is what
an admin approval is bound to, so a second machine claiming the
same hostname and serial cannot collect the approved credential.
One row per (hostname, serial number) that asked to enroll -
the endpoint's identity, and the reason every contact from one
PC lands on one row. The agent's enrollment nonce is recorded
on first sight and is what an admin approval is bound to, so a
second machine claiming the same hostname and serial cannot
collect the approved credential. A REJECTED row binds to the
identity instead: any nonce from that hostname and serial stays
rejected, so a reinstall cannot argue its way back into the
queue. Only an admin approving the request clears it.
EndpointMigrationWave
An admin action that pushed the agent upgrade to a set of PCs.

44
plugins/endpoints/plugin.py
View File

@@ -287,6 +287,15 @@ class EndpointsPlugin(BasePlugin):
making revocation an instant kill switch. `agentkeyprovisioned` records
that a UI-managed key has been minted so the settings page can render
the rotate/revoke state instead of the first-run generate state.
Every key an enforcement path reads belongs here, even when its default
is what the code already does with no row at all. Undeclared, the row
is only born when somebody writes it and the settings API has to guess
its type - which is how bootstrapquerytoken became a string row that
its boolean reader could never see turned off. Declared, the row is
seeded at install and enable (and by `flask plugin upgrade-all` on a
site that installed an earlier version), so the settings pages have
something to render and an operator has something to click.
"""
return [
{
@@ -318,6 +327,41 @@ class EndpointsPlugin(BasePlugin):
'approval when the hostname and serial number '
'match exactly one computer.',
},
{
'key': 'plugin.endpoints.revokeddeniessharedkey',
'value': 'false',
'valuetype': 'boolean',
'category': 'plugin.endpoints',
'description': 'Also refuse the shared fleet key to a PC whose '
'per-PC credential was revoked. Off (the '
'default) is what the fleet does today: a '
'revoked PC falls back to the shared key, so a '
'credential rotation and the documented revoke '
'plus blockreenroll rollback to an older agent '
'both keep reporting. Turning it on makes a '
'revoke an eviction - that PC reports nothing '
'and receives no commands until it enrolls '
'again - and it stops that rollback working. '
'Safe only once every PC holds its own '
'credential.',
},
{
'key': 'plugin.endpoints.bootstrapquerytoken',
'value': 'true',
'valuetype': 'boolean',
'category': 'plugin.endpoints',
'description': 'Accept a one-line install bootstrap token in '
'the ?t= query string as well as the '
'X-Endpoint-Bootstrap-Token header. On (the '
'default) '
'keeps the pasted install commands and the '
'Download .ps1 link working; the query form is '
'deprecated because the token lands in IIS and '
'proxy logs. Turn it off only after every '
'outstanding install command has been re-issued '
'- an in-flight one stops working the moment it '
'is off.',
},
{
'key': 'plugin.endpoints.agentkeyprovisioned',
'value': 'false',

42
plugins/notifications/models/notification.py
View File

@@ -1,10 +1,46 @@
"""Notifications plugin models - adapted to existing database schema."""
import json
from datetime import datetime, timezone
from zoneinfo import ZoneInfo
from sqlalchemy import Text
from sqlalchemy.types import TypeDecorator
from shopdb.api import db
class JSONText(TypeDecorator):
"""A dict on the Python side, TEXT in the database.
MySQL 5.6 has no JSON type and a site still runs it, so every JSON-bearing
column in this product is TEXT. Declared here rather than imported from the
computers plugin: notifications declares no dependency on computers, and a
cross-plugin import would be a hidden one.
"""
impl = Text
cache_ok = True
def process_bind_param(self, value, dialect):
if value is None:
return None
return json.dumps(value, separators=(',', ':'))
def process_result_value(self, value, dialect):
if value is None or value == '':
return None
try:
return json.loads(value)
except (TypeError, ValueError):
# Hand-edited or written before this column held JSON. Return the
# raw text rather than raising: a scope that cannot be parsed
# should be visible and fixable, and the endpoints matcher already
# treats a non-dict scope as matching nothing.
return value
_DEFAULT_TZ = 'America/New_York'
# Keys a fleet alert's alertscope may use. Must stay equal to the set the
@@ -155,7 +191,11 @@ class Notification(db.Model):
# Matches migration 0006: NOT NULL with server defaults.
alertfleet = db.Column(db.Boolean, nullable=False, default=False,
server_default=db.false())
alertscope = db.Column(db.JSON, nullable=True)
# JSONText, not db.JSON: migration 0006 ships this column as TEXT because a
# site still runs MySQL 5.6, which has no JSON type. db.JSON here meant
# create-all built a JSON column the migration path never produces, so the
# model and the shipped schema disagreed about the same column.
alertscope = db.Column(JSONText, nullable=True)
alerttimeout = db.Column(db.Integer, nullable=False, default=0,
server_default='0')

79
plugins/printedparts/api/routes.py
View File

@@ -80,7 +80,7 @@ import re
from flask import current_app
from werkzeug.utils import secure_filename
from shopdb.api import Setting
from shopdb.api import Setting, client_ip, ip_in_cidrs
IMAGE_EXTENSIONS = {'.png', '.jpg', '.jpeg', '.gif', '.webp'}
IMAGE_URL_PREFIX = '/api/printedparts/image/'
@@ -518,16 +518,39 @@ def adjust_item(item_id: int):
import hmac
from shopdb.api import audit_log
# Largest single take the kiosk records. A shop-floor grab is single digits; 25
# leaves real use untouched and denies draining a bin in one request.
KIOSK_MAX_TAKE_QUANTITY = 25
# Addresses that belong to the PATH, not to a caller. Subtracted from the
# allowlist before it is consulted.
#
# WHY, and it is the half that was missing here: a reverse proxy normally sits
# ON the network a site enters in printedparts_kiosk_allowed_cidrs. A request
# that arrives wearing the proxy's own address - because the proxy forwarded
# nothing, or because nothing forwarded at all - therefore MATCHED the allowlist
# and was admitted while carrying no evidence about who actually sent it. Being
# inside the allowlist is not enough; the address also has to be one a client
# could plausibly have had.
#
# Loopback is never a client at any site, so it needs no configuration. The USB
# kiosk closed this on 2026-09-18 and this one did not, which is the only reason
# the two read differently.
PARTS_KIOSK_TRUSTED_PROXIES = 'printedparts_kiosk_trusted_proxies'
PARTS_KIOSK_NEVER_A_CLIENT = '127.0.0.0/8, ::1/128'
def _kiosk_open():
"""True when this site has the kiosk on AND the caller carries its token."""
enabled = Setting.get('printedparts_kiosk_enabled', default=False)
if not (enabled is True or str(enabled).strip().lower() in ('true', '1', 'yes', 'on')):
return False
def _kiosk_address_is_infrastructure(address):
"""True when this address belongs to the path rather than to a caller."""
if ip_in_cidrs(address, PARTS_KIOSK_NEVER_A_CLIENT):
return True
return ip_in_cidrs(address, Setting.get(PARTS_KIOSK_TRUSTED_PROXIES))
def _kiosk_token_presented():
"""True when the caller carries this kiosk's device token."""
expected = (Setting.get('printedparts_kiosk_token') or '').strip()
presented = (request.headers.get('X-Kiosk-Token') or '').strip()
if not expected or not presented:
@@ -537,6 +560,50 @@ def _kiosk_open():
return hmac.compare_digest(presented, expected)
def _kiosk_open():
"""True when this site has the kiosk on AND the caller is admitted.
Two ways in, either sufficient:
1. the per-device token, for the unattended screen by the printers, and
2. a source address inside printedparts_kiosk_allowed_cidrs, for people
who open the page on their own PC and never log in.
Its own setting, not the USB kiosk's: the two are separate screens in
separate rooms, and one allowlist granting the other is exactly the sharing
the separate tokens exist to prevent.
"""
enabled = Setting.get('printedparts_kiosk_enabled', default=False)
if not (enabled is True or str(enabled).strip().lower() in ('true', '1', 'yes', 'on')):
return False
if _kiosk_token_presented():
return True
address = client_ip(request)
if _kiosk_address_is_infrastructure(address):
return False
return ip_in_cidrs(address, Setting.get('printedparts_kiosk_allowed_cidrs'))
@printedparts_bp.route('/kiosk/provisioning-link', methods=['GET'])
@require_permission('settings.edit')
def kiosk_provisioning_link():
"""The URL that provisions a screen, token included, for an administrator.
Same reason as the USB kiosk: the settings API masks any key named
'*token*', so the value is visible for one moment after Generate and an
administrator who copies the field later provisions a screen with
'********' and gets a silent 404. This hands the real value to somebody
already allowed to set it, so the mask can stay.
"""
token = (Setting.get('printedparts_kiosk_token') or '').strip()
if not token:
return error_response(ErrorCodes.VALIDATION_ERROR,
'No kiosk token set. Generate one first.')
audit_log('viewed', 'Setting', entityname='printedparts_kiosk_token',
details='kiosk provisioning link')
db.session.commit()
return success_response({'url': '/parts-kiosk?token=' + token})
def _kiosk_shut():
"""404, not 403 - a caller without the token does not learn the door exists."""
return error_response(ErrorCodes.NOT_FOUND, 'Not found', http_code=404)

62
plugins/printedparts/frontend/views/PrintedPartsSettings.vue
View File

@@ -55,7 +55,13 @@
<button type="button" class="btn btn-secondary" @click="generateToken">
Generate
</button>
<button type="button" class="btn btn-secondary" @click="copyLink">
Copy setup link
</button>
</div>
<p v-if="linkMessage" class="field-hint">{{ linkMessage }}</p>
<input v-if="link" :value="link" type="text" class="form-control"
readonly @focus="$event.target.select()" />
<p class="field-hint">
Each kiosk screen is provisioned once by opening
<code>/parts-kiosk?token=&lt;this value&gt;</code> on it. The browser
@@ -68,6 +74,30 @@
</p>
</div>
<div class="form-group">
<label>Networks allowed without the token</label>
<NetworkAllowlistPicker v-model="values.printedparts_kiosk_allowed_cidrs" />
<p class="field-hint">
For people who open the kiosk on their own PC rather than at the
screen. A browser on one of these networks is admitted with no token
and no login, so keep this to the shop-floor ranges. Leave empty to
require the token from every caller. Separate from the USB kiosk
allowlist: one does not grant the other.
</p>
</div>
<div class="form-group">
<label>Reverse proxies in front of this server</label>
<NetworkAllowlistPicker v-model="values.printedparts_kiosk_trusted_proxies" />
<p class="field-hint">
Only needed when a proxy sits between the kiosk and this server. A
request arriving FROM one of these addresses is never treated as a
kiosk, so a proxy inside an allowed network above cannot admit itself.
Loopback is always excluded and needs no entry. Leave empty when
browsers reach this server directly.
</p>
</div>
<div class="form-group">
<label>Unknown badge at the kiosk</label>
<select v-model="values.printedparts_unknown_badge" class="form-control">
@@ -143,6 +173,34 @@
<script setup>
import { ref, computed, onMounted } from 'vue'
import { settingsApi, usersApi, supportteamsApi } from '@/api'
import NetworkAllowlistPicker from '@/components/NetworkAllowlistPicker.vue'
import api from '@/api'
import { withBase } from '@/utils/basePath'
// The settings API masks the token, so the field shows '********'. Copying that
// provisions a screen with the mask and every kiosk call answers 404 with
// nothing to say why. Ask the server for the real link instead.
const link = ref('')
const linkMessage = ref('')
async function copyLink () {
linkMessage.value = ''
link.value = ''
try {
const response = await api.get('/printedparts/kiosk/provisioning-link')
const url = window.location.origin + withBase(response.data.data.url)
link.value = url
try {
await navigator.clipboard.writeText(url)
linkMessage.value = 'Copied. Open this once on the kiosk screen.'
} catch (clipboardError) {
linkMessage.value = 'Copy this and open it once on the kiosk screen.'
}
} catch (linkError) {
linkMessage.value = linkError.response?.data?.data?.error?.message
|| 'Could not build the setup link.'
}
}
const KEYS = [
'printedparts_code_prefix',
@@ -150,6 +208,8 @@ const KEYS = [
'printedparts_default_threshold',
'printedparts_kiosk_enabled',
'printedparts_kiosk_token',
'printedparts_kiosk_allowed_cidrs',
'printedparts_kiosk_trusted_proxies',
'printedparts_unknown_badge',
'printedparts_alert_email',
'printedparts_alert_userids',
@@ -163,6 +223,8 @@ const values = ref({
printedparts_default_threshold: 5,
printedparts_kiosk_enabled: 'false',
printedparts_kiosk_token: '',
printedparts_kiosk_allowed_cidrs: '',
printedparts_kiosk_trusted_proxies: '',
printedparts_unknown_badge: 'deny',
printedparts_alert_email: '',
printedparts_alert_userids: '',

2
plugins/printedparts/manifest.json
View File

@@ -5,7 +5,7 @@
"display_name": "3D Printed Parts",
"author": "",
"dependencies": ["employees"],
"core_version": ">=0.16.0,<1.0.0",
"core_version": ">=0.35.0,<1.0.0",
"api_prefix": "/api/printedparts",
"default_enabled": false
}

30
plugins/printedparts/plugin.py
View File

@@ -172,6 +172,36 @@ class PrintedpartsPlugin(BasePlugin):
'Generate one, then open the kiosk once as '
'/parts-kiosk?token=<value> on each screen',
},
{
'key': 'printedparts_kiosk_trusted_proxies',
'value': '',
'valuetype': 'string',
'category': 'printedparts',
'description': 'Addresses or CIDRs of the reverse proxies and '
'load balancers in front of this server, comma '
'separated. SUBTRACTED from the kiosk allowlist '
'below: a request wearing one of these addresses '
'is the proxy talking about itself, so it is '
'refused rather than admitted by a range that '
'happens to contain the proxy. Loopback is '
'always subtracted and needs no entry. Leave '
'empty when browsers reach this server directly.',
},
{
'key': 'printedparts_kiosk_allowed_cidrs',
'value': '',
'valuetype': 'string',
'category': 'printedparts',
'description': 'Networks allowed to use the kiosk WITHOUT the '
'device token, as comma-separated CIDRs (for '
'example 192.0.2.0/24). For people who open '
'the kiosk on their own PC and never log in. '
'Empty = token only. Separate from the USB '
'kiosk allowlist on purpose. Requires the IIS '
'X-Forwarded-For rewrite rule to be live, or the '
'app sees the proxy address and this matches '
'nothing.',
},
{
'key': 'printedparts_unknown_badge',
'value': 'deny',

560
plugins/usb/api/routes.py
View File

@@ -20,13 +20,17 @@ cmmc_usb schema used here:
'YYYY-MM-DD HH:MM:SS' string; scanned_viruses/sanitized are '0'|'1'|null.
"""
import hashlib
import ipaddress
import logging
import re
import secrets
import time
from datetime import datetime, timedelta
import hmac
from flask import Blueprint, request, Response
from flask import Blueprint, current_app, redirect, request, Response
from flask_jwt_extended import jwt_required
from shopdb.api import (
@@ -39,7 +43,7 @@ from shopdb.api import (
get_pagination_params,
require_permission,
)
from shopdb.api import Setting
from shopdb.api import Setting, audit_log, cache, client_ip, db, ip_in_cidrs
from . import selfhosted
@@ -743,8 +747,132 @@ def _kiosk_enabled():
return _kiosk_setting('usb_kiosk_enabled', False)
# The device token also travels as a cookie, set by the setup exchange below,
# so it never has to sit in a URL to reach a screen.
KIOSK_TOKEN_COOKIE = 'usbkiosktoken'
# Where this site's reverse proxies are, as CIDRs. NOT a source of addresses -
# see the block below - but a subtraction from the allowlist: a request wearing
# one of these addresses is infrastructure talking, never a caller.
KIOSK_TRUSTED_PROXIES = 'usb_kiosk_trusted_proxies'
# Addresses that are never a kiosk client at any site, whatever was typed into
# the allowlist. Loopback is the shipped Windows stack with no forwarding header
# at all: HttpPlatformHandler connects from 127.0.0.1, waitress has nothing to
# promote, and every caller then arrives wearing the app server's own address.
KIOSK_NEVER_A_CLIENT = '127.0.0.0/8, ::1/128'
def _kiosk_token_presented():
"""True when the caller presents the configured device token.
The token is per-DEVICE. A kiosk is a browser on a screen by the cabinet, so
a token served inside the page would be readable by anyone who could load
it.
Two carriers, one value. A screen provisioned the old way keeps it in
localStorage and sends X-Kiosk-Token; a screen provisioned through
kiosk_setup holds it in an HttpOnly cookie that the browser sends on its
own. Accepting both is what lets the URL-borne token go away without
reprovisioning every screen already on the floor.
"""
expected = (Setting.get('usb_kiosk_token') or '').strip()
if not expected:
return False
presented = ((request.headers.get('X-Kiosk-Token') or '').strip()
or (request.cookies.get(KIOSK_TOKEN_COOKIE) or '').strip())
if not presented:
return False
# compare_digest, not ==: == leaks the token's length and matching prefix
# through timing.
return hmac.compare_digest(presented, expected)
# ---------------------------------------------------------------------------
# The address a caller is admitted on, traced through the stack that ships
# ---------------------------------------------------------------------------
# IIS -> HttpPlatformHandler -> waitress on 127.0.0.1 -> this app, and waitress
# is launched with --trusted-proxy=127.0.0.1
# --trusted-proxy-headers=x-forwarded-for --trusted-proxy-count=1
# (deploy/windows/web.config). HttpPlatformHandler ALWAYS connects over
# loopback, so waitress always sees its configured trusted proxy as the peer: it
# takes the LAST X-Forwarded-For hop, writes it into REMOTE_ADDR, and truncates
# the environ header to that one hop, all before Flask is entered.
#
# So request.remote_addr IS the forwarded address by the time anything here
# runs, and X-Forwarded-For carries a single entry equal to it. That is why the
# version of this code that walked the header could not work on the Windows
# deployment: it looked for the peer inside usb_kiosk_trusted_proxies before it
# would read the header, and the peer is never the proxy any more - waitress
# already replaced it with whatever the proxy forwarded. The branch could not
# fire, and the parsing under it was inert.
#
# THIS APP TAKES THE WSGI LAYER'S WORD, and the app-level parsing is gone. Not
# because that word is trustworthy in itself, but because Python is the wrong
# place to adjudicate it: the proxy's own address is destroyed before Flask is
# reached, so nothing here can tell a hop IIS wrote from one a caller sent.
# Whether the address is real is settled in front of waitress, and only there:
# -ClientIpSource direct: the IIS rewrite rule OVERWRITES X-Forwarded-For with
# IIS's own REMOTE_ADDR, so a caller-sent header is destroyed on arrival.
# -ClientIpSource proxy: the reverse proxy APPENDS the caller, so the last hop
# is the address that proxy itself observed.
# Under either, a client-sent header cannot choose the value waitress promotes.
# The installer checks this in both modes and, when it does not hold, says to
# leave the kiosk allowlist empty (deploy/windows/installer/shopdb-install.ps1).
#
# Reading request.headers['X-Forwarded-For'] here would undo all of it: on a box
# whose front end does not overwrite, that header is the caller's to invent, and
# the kiosk would admit on a value the caller's network position does not
# support. So it is not read. Anywhere in this file.
def _kiosk_client_ip():
"""The caller address, as the layer in front of Flask resolved it."""
return client_ip(request)
def _kiosk_address_is_infrastructure(address):
"""True when this address belongs to the path, not to a caller.
The fail-closed half of network admission, and the half that was missing. A
reverse proxy normally sits ON the plant network a site enters in
usb_kiosk_allowed_cidrs. A request that arrives wearing the proxy's own
address - because the proxy forwarded nothing, or because nothing forwarded
at all - therefore MATCHED the allowlist and was admitted, while carrying no
evidence whatever about who sent it. That is the one case that has to
refuse.
Being inside the allowlist is not enough on its own: the address also has to
be one a client could plausibly have had. The previous version only looked
closed because the proxy in its test happened to sit outside the allowlist.
"""
if ip_in_cidrs(address, KIOSK_NEVER_A_CLIENT):
return True
return ip_in_cidrs(address, Setting.get(KIOSK_TRUSTED_PROXIES))
def _kiosk_ip_allowlisted():
"""True when the caller sits in the configured kiosk network allowlist.
Why this exists alongside the token: the token model assumes a handful of
fixed screens provisioned once. Most people reach this kiosk from their own
PC instead, and there is no login - so per-browser provisioning would mean
provisioning every PC on the floor, and losing it on every reimage or
cleared profile. Network position is what those callers actually have in
common.
Fail-closed behaviour is documented on ip_in_cidrs; where the caller address
comes from is on _kiosk_client_ip, and which addresses can never be a caller
at all is on _kiosk_address_is_infrastructure.
"""
address = _kiosk_client_ip()
if _kiosk_address_is_infrastructure(address):
return False
return ip_in_cidrs(address, Setting.get('usb_kiosk_allowed_cidrs'))
def _kiosk_open():
"""True when the kiosk is on for this site AND the caller has its token.
"""True when the kiosk is on for this site AND the caller is admitted.
The flag alone was the whole gate, and a flag is a decision, not a
credential: with the kiosk on, anyone who could reach the app could move a
@@ -752,20 +880,18 @@ def _kiosk_open():
badge they guessed. A badge is a short numeric SSO, so it is attribution,
never authorization - the same reasoning that closed the parts kiosk.
The token is per-DEVICE. A kiosk is a browser on a screen by the cabinet, so
a token served inside the page would be readable by anyone who could load
it. An admin opens the screen once as /usb-kiosk?token=..., the page keeps
it in that browser and sends it as X-Kiosk-Token.
Two ways in, either sufficient, mirroring what the GE-Enforce client
endpoints already do:
1. the per-device token, for unattended screens by the cabinet, and
2. a source address inside usb_kiosk_allowed_cidrs, for the people who
open this page on their own PC and never log in.
Both are still gated on usb_kiosk_enabled, so the door stays shut until a
site opens it deliberately.
"""
if not _kiosk_enabled():
return False
expected = (Setting.get('usb_kiosk_token') or '').strip()
presented = (request.headers.get('X-Kiosk-Token') or '').strip()
if not expected or not presented:
return False
# compare_digest, not ==: == leaks the token's length and matching prefix
# through timing.
return hmac.compare_digest(presented, expected)
return _kiosk_token_presented() or _kiosk_ip_allowlisted()
def _kiosk_disabled_response():
@@ -783,6 +909,387 @@ def _kiosk_badge(data):
return badge, None
# ---------------------------------------------------------------------------
# What a site may store in a kiosk CIDR list
# ---------------------------------------------------------------------------
# These two settings decide who may write without logging in, and the settings
# API stored whatever was typed: it checks valuetype and nothing else, and has
# no per-key hook a plugin can declare. So '0.0.0.0/0' saved cleanly and turned
# the kiosk into an open write for anything that could reach the server.
#
# Checked at the SAVE (see validate_kiosk_settings_write, registered by the
# plugin), because that is the only moment an administrator finds out. Refusing
# a range at admission time instead would look exactly like the setting quietly
# not working.
KIOSK_CIDR_KEYS = ('usb_kiosk_allowed_cidrs', 'usb_kiosk_trusted_proxies')
KIOSK_WIDE_OVERRIDE = 'usb_kiosk_allow_wide_cidrs'
# A /16 is already 65,536 addresses. Anything wider is a claim about the site
# that a site should have to make on purpose, not a typo that widens a door.
KIOSK_MIN_PREFIX = {4: 16, 6: 48}
def kiosk_cidr_problem(raw, allowwide=False):
"""Why this CIDR list may not be stored, or None when it may.
The message names the entry and says what it would mean, because the person
reading it is fixing a range on a settings page, not debugging a validator.
"""
for part in (raw or '').split(','):
part = part.strip()
if not part:
continue
try:
network = ipaddress.ip_network(part, strict=False)
except ValueError:
return ('"{0}" is not a network. Use address/prefix, for example '
'192.0.2.0/24.'.format(part))
if network.prefixlen == 0:
return ('"{0}" is every address there is. The kiosk writes without '
'a login, so this would let anything that can reach the '
'server move a drive against a badge it guessed. Name the '
'shop-floor ranges instead.'.format(part))
floor = KIOSK_MIN_PREFIX[network.version]
if network.prefixlen < floor and not allowwide:
return ('"{0}" covers {1} addresses, which is wider than a site '
'range (/{2} or narrower). Everyone inside it is admitted '
'with no token and no login. Narrow it, or turn on "{3}" '
'if this site really is one flat network that '
'wide.'.format(part, network.num_addresses, floor,
KIOSK_WIDE_OVERRIDE))
return None
# Both settings routes are gated on this one permission. Named here because the
# guard below has to apply the SAME gate the view would, and a guard that
# guessed a weaker one would be the hole it is meant to close.
SETTINGS_WRITE_PERMISSION = 'settings.edit'
def _settings_write_refusal():
"""The view's own 401/403 for this caller, or None when it may write.
A before_request runs ahead of the view, so it runs ahead of
@require_permission too. That ordering leaked the validation rules: an
anonymous PUT of '0.0.0.0/0' was answered 400 with the reason, while an
anonymous PUT of a sane range fell through to the view and was answered 401.
The difference is an oracle - anyone who could reach the app could walk the
allowlist rules without ever holding an account.
So authentication decides first, by running the same decorator the view
carries rather than a local imitation of it. Guest grants, scoped personal
tokens and the admin bypass therefore behave identically here and there, and
the response an unauthorized caller gets is the one it would have got if
this guard did not exist.
"""
@require_permission(SETTINGS_WRITE_PERMISSION)
def permitted():
return None
return permitted()
def validate_kiosk_settings_write():
"""before_request guard on the core settings save, for the kiosk CIDR keys.
Registered app-wide by the plugin (init_app): the keys belong to this
plugin, so this plugin is what knows a bad value, and returning a response
from before_request answers the save without performing it.
"""
if request.endpoint not in ('settings.update_setting',
'settings.create_setting'):
return None
data = request.get_json(silent=True) or {}
if request.endpoint == 'settings.update_setting':
key = (request.view_args or {}).get('key')
else:
key = data.get('key')
if key not in KIOSK_CIDR_KEYS or 'value' not in data:
return None
refusal = _settings_write_refusal()
if refusal is not None:
return refusal
# _kiosk_setting, not bool(Setting.get(...)): a row stored as a string holds
# the word 'false', and bool('false') is True - which would turn the
# override on for a site that had just turned it off.
problem = kiosk_cidr_problem(str(data.get('value') or ''),
_kiosk_setting(KIOSK_WIDE_OVERRIDE, False))
if problem:
return error_response(ErrorCodes.VALIDATION_ERROR, problem)
return None
# ---------------------------------------------------------------------------
# Badge lookups: a bounded number of misses per caller
# ---------------------------------------------------------------------------
# Every kiosk call that takes a badge answers with the employee's NAME, which is
# what makes the kiosk usable and also makes it a lookup table: a badge is a
# short number, so a caller who is admitted can walk it. Admission used to mean
# one screen by the cabinet; with the network allowlist it can mean a whole
# VLAN, and that changes what the lookup is worth to somebody with a loop.
#
# Only MISSES are counted. An ordinary day at the cabinet produces none - a miss
# is a scan of something this site has never heard of - so the budget can sit
# well above a bad scanning day and still stop enumeration inside a minute.
# Same fixed-window cache shape as the login limiter in shopdb.core.api.auth:
# under SimpleCache the counter is per-process, which is defence in depth rather
# than a hard quota.
BADGE_MISS_MAX = 15
BADGE_MISS_WINDOW_SECONDS = 600
# The two of those an operator would ever reach for, as settings. They were
# app-config only, which on an air-gapped server means hand-editing
# instance/config.py and restarting IIS - so in practice the limiter had no off
# switch and no dial at all, and the one lever anybody had was to turn the whole
# kiosk off. Both default to exactly what the constants above did.
#
# The WINDOW deliberately stays app-config. It is the shape of the limiter, not
# a number a site tunes; a budget expressed per ten minutes is the thing being
# decided, and offering two interacting dials invites a combination nobody
# meant.
KIOSK_BADGE_LIMIT_ENABLED = 'usb_kiosk_badge_limit_enabled'
KIOSK_BADGE_MISS_MAX = 'usb_kiosk_badge_miss_max'
def _kiosk_int_setting(key, default):
"""A positive whole number from a setting, else default.
A settings page stores text, so a blank, a word or a pasted range can all
arrive here. Any of those has to land on the shipped budget rather than on
zero, because zero is not "no limit" in this counter - it refuses every
badge lookup the site has from the first scan of the window.
"""
row = Setting.query.filter_by(key=key).first()
raw = row.value if row is not None and row.value is not None else ''
try:
value = int(str(raw).strip())
except (TypeError, ValueError):
return default
return value if value > 0 else default
def _badge_miss_budget():
# Config first so instance/config.py and the test fixtures keep overriding,
# then the setting, then the shipped number.
maxmisses = current_app.config.get('USB_KIOSK_BADGE_MISS_MAX')
if maxmisses is None:
maxmisses = _kiosk_int_setting(KIOSK_BADGE_MISS_MAX, BADGE_MISS_MAX)
return (current_app.config.get('USB_KIOSK_BADGE_WINDOW_SECONDS',
BADGE_MISS_WINDOW_SECONDS),
maxmisses)
def _badge_miss_key(window):
bucket = int(time.time() // window) if window > 0 else 0
return 'usbkioskbadgemiss:{0}:{1}'.format(
_kiosk_client_ip() or 'unknown', bucket)
def _badge_limit_enabled():
"""On everywhere but a test run, where the shared cache would bleed.
The counter lives in the process-wide cache, which outlives a test and is
shared by whatever else lands on the same xdist worker - so misses spent in
one module surface as an unrelated 429 in another, and which modules share a
worker moves around. AUTH_RATELIMIT_ENABLED and the endpoints enrollment
limiter are switched off in TestingConfig for exactly that reason; this one
is younger than that file and defaults itself the same way.
Order: an explicit app config value (instance/config.py, and the fixtures in
the test suite) wins, then TESTING, then the site setting, which ships on.
A site turning this off is a real decision - the kiosk answers with an
employee NAME, so an unbounded badge lookup is a directory anyone admitted
can walk - which is why it is reachable rather than buried in a config file
nobody on an air-gapped box can edit without a maintenance window.
"""
override = current_app.config.get('USB_KIOSK_BADGE_LIMIT_ENABLED')
if override is not None:
return bool(override)
if current_app.config.get('TESTING', False):
return False
return _kiosk_setting(KIOSK_BADGE_LIMIT_ENABLED, True)
def badge_lookups_exhausted():
"""True when this caller has spent its budget of unrecognised badges."""
if not _badge_limit_enabled():
return False
window, maxmisses = _badge_miss_budget()
return (cache.get(_badge_miss_key(window)) or 0) >= maxmisses
def record_badge_miss():
"""Count one unrecognised scan against this caller.
Called from both directory modes - the raw-SQL path below and the
self-hosted one in selfhosted.py - at the point where a scanned badge
resolves to nobody. Holder-name lookups do not come through here: those are
the kiosk naming a drive's holder, not a caller guessing.
"""
if not _badge_limit_enabled():
return
window, maxmisses = _badge_miss_budget()
key = _badge_miss_key(window)
count = (cache.get(key) or 0) + 1
cache.set(key, count, timeout=window)
if count == maxmisses:
logger.warning('usb kiosk: %s unrecognised badges from %s within %ss; '
'refusing badge lookups for the rest of the window',
count, _kiosk_client_ip(), window)
def _badge_lookups_refused():
return error_response(
'RATE_LIMITED',
'Too many badges this screen does not recognise. Wait a few minutes '
'and try again, or see IT.', http_code=429)
# ---------------------------------------------------------------------------
# Provisioning a screen without putting the token in a URL
# ---------------------------------------------------------------------------
# A query string is the one place a credential is guaranteed to be copied:
# browser history, the Referer on the next request, the IIS access log, whatever
# the proxy keeps. So the link an administrator carries to a screen holds a code
# that is good once and briefly, and the token itself comes back in a Set-Cookie
# that no log line carries.
#
# Only the code's DIGEST is stored, so the pending row is not itself a
# credential; the expiry rides with it.
KIOSK_SETUP_CODE = 'usb_kiosk_setup_code'
# Long enough to walk from a desk to the cabinet, short enough that a link left
# in a history or a log is already dead when somebody reads it back.
KIOSK_SETUP_TTL_SECONDS = 30 * 60
# The screen is provisioned once and must not quietly fall out of the kiosk
# months later, which is the failure this cookie has to avoid above all.
KIOSK_COOKIE_MAX_AGE = 5 * 365 * 24 * 60 * 60
def _issue_kiosk_setup_code():
"""Mint a single-use code, store its digest and expiry, return the code."""
code = secrets.token_urlsafe(24)
digest = hashlib.sha256(code.encode()).hexdigest()
expiry = int(time.time()) + KIOSK_SETUP_TTL_SECONDS
Setting.set(KIOSK_SETUP_CODE, '{0}|{1}'.format(digest, expiry),
valuetype='string', category='usb',
description='Digest and expiry of the pending kiosk setup '
'code. Written and cleared by the setup link; not '
'edited by hand.')
return code
def _redeem_kiosk_setup_code(code):
"""True exactly once per issued code, and never once it has expired.
Cleared on ANY attempt. A code is worth one screen; burning it on a wrong
guess costs an administrator one click and leaves a prober nothing to
retry against.
"""
stored = (Setting.get(KIOSK_SETUP_CODE) or '').strip()
if not stored or '|' not in stored:
return False
Setting.set(KIOSK_SETUP_CODE, '', valuetype='string', category='usb')
digest, _, expiry = stored.partition('|')
try:
if int(expiry) < int(time.time()):
return False
except ValueError:
return False
return hmac.compare_digest(
digest, hashlib.sha256((code or '').encode()).hexdigest())
@usb_bp.route('/kiosk/provisioning-link', methods=['GET'])
@require_permission('settings.edit')
def kiosk_provisioning_link():
"""The URL that provisions a screen, token included, for an administrator.
WHY THIS EXISTS. The settings API masks every key whose name contains
'token', so the settings page shows the kiosk token as '********'. The real
value is visible for exactly one moment - after Generate, before Save - and
an administrator who copies the field at any other time provisions a screen
with the literal string '********'. Nothing says so: the kiosk answers 404
the same way it answers a caller it has never heard of, because the route
deliberately does not advertise itself. That has cost real time.
So the copy step stops depending on catching that moment. The mask stays -
it is right that a token does not travel in an ordinary settings list - and
this named, permission-gated route hands over a way in to somebody who is
already allowed to set it.
WHAT IT HANDS OVER IS NOT THE TOKEN. The link used to be
'/usb-kiosk?token=<value>', which put the credential in a query string: the
address bar, the browser history, a Referer, the IIS log and anything in
between. It is now a single-use code that kiosk_setup trades for the token
over a Set-Cookie. One click here, one open on the screen - the operator
flow is the same.
Returns a relative URL. The caller's own origin is the right base for a
screen being provisioned from that same browser, and the server has no
reliable idea what hostname the operator reached it by.
"""
token = (Setting.get('usb_kiosk_token') or '').strip()
if not token:
return error_response(ErrorCodes.VALIDATION_ERROR,
'No kiosk token set. Generate one first.')
code = _issue_kiosk_setup_code()
# Issuing a way to claim the token is worth a line in the audit log: the
# trail should say who asked, and when the clock on it started.
audit_log('viewed', 'Setting', entityname='usb_kiosk_token',
details='kiosk setup link issued')
db.session.commit()
return success_response({
'url': '/api/usb/kiosk/setup?code=' + code,
'expiresinminutes': KIOSK_SETUP_TTL_SECONDS // 60,
})
@usb_bp.route('/kiosk/setup', methods=['GET'])
def kiosk_setup():
"""Trade a setup code for this browser's kiosk cookie, then open the kiosk.
Opened once on the screen being provisioned. The code is spent here, the
token travels back in a Set-Cookie rather than a URL, and the browser is
sent on to the kiosk page with nothing left in the address bar.
HttpOnly, so the page's own JavaScript cannot read the token back out and
put it somewhere that does get logged. SameSite=Lax, so another site cannot
make this browser POST at the kiosk routes with it. Secure only over https:
setting it unconditionally would drop the cookie on the plain-http sites
this is deployed on and provision nothing.
Not gated on usb_kiosk_enabled: a site sets its screens up before opening
the door, and a kiosk that is off refuses every call anyway.
"""
token = (Setting.get('usb_kiosk_token') or '').strip()
if not token:
return error_response(ErrorCodes.VALIDATION_ERROR,
'No kiosk token set. Generate one first.')
if not _redeem_kiosk_setup_code(request.args.get('code')):
# 410, not 404: this one is read by an operator standing at a screen,
# and "gone" plus what to do next is the difference between a working
# kiosk and a support call.
return error_response(
ErrorCodes.NOT_FOUND,
'This setup link has already been used or has expired. Generate a '
'new one under Settings > USB.', http_code=410)
audit_log('viewed', 'Setting', entityname='usb_kiosk_token',
details='kiosk setup link redeemed')
db.session.commit()
mount = request.script_root or ''
response = redirect(mount + '/usb-kiosk')
response.set_cookie(KIOSK_TOKEN_COOKIE, token,
max_age=KIOSK_COOKIE_MAX_AGE, httponly=True,
samesite='Lax', secure=request.is_secure,
path=mount + '/')
# This response carries the token. Nothing anywhere may keep a copy.
response.headers['Cache-Control'] = 'no-store'
return response
@usb_bp.route('/kiosk/device/<device_id>', methods=['GET'])
def kiosk_device(device_id):
"""What the kiosk shows after a device is scanned.
@@ -854,6 +1361,8 @@ def kiosk_session():
"""
if not _kiosk_open():
return _kiosk_disabled_response()
if badge_lookups_exhausted():
return _badge_lookups_refused()
data = request.get_json() or {}
if _usb_selfhosted():
return selfhosted.kiosk_session(data)
@@ -867,6 +1376,7 @@ def kiosk_session():
with conn.cursor() as cur:
name = _resolve_badge_name(cur, badge)
if not name:
record_badge_miss()
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Badge not recognized - see IT',
@@ -912,6 +1422,8 @@ def kiosk_others():
return _kiosk_disabled_response()
if not _kiosk_setting('usb_kiosk_allow_return_for_others', True):
return _kiosk_disabled_response()
if badge_lookups_exhausted():
return _badge_lookups_refused()
data = request.get_json() or {}
if _usb_selfhosted():
return selfhosted.kiosk_others(data)
@@ -924,6 +1436,7 @@ def kiosk_others():
try:
with conn.cursor() as cur:
if not _resolve_badge_name(cur, badge):
record_badge_miss()
return error_response(ErrorCodes.VALIDATION_ERROR,
'Badge not recognized - see IT',
http_code=422)
@@ -954,6 +1467,8 @@ def kiosk_checkout():
"""Check a device out at the kiosk. Body: {device_id, badge}."""
if not _kiosk_open():
return _kiosk_disabled_response()
if badge_lookups_exhausted():
return _badge_lookups_refused()
data = request.get_json() or {}
if _usb_selfhosted():
return selfhosted.kiosk_checkout(data)
@@ -988,6 +1503,7 @@ def kiosk_checkout():
# kiosk there is no signed-in operator to vouch for it.
name = _resolve_badge_name(cur, badge)
if not name:
record_badge_miss()
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Badge not recognized - see IT to check this device out',
@@ -1041,6 +1557,8 @@ def kiosk_checkin():
"""
if not _kiosk_open():
return _kiosk_disabled_response()
if badge_lookups_exhausted():
return _badge_lookups_refused()
data = request.get_json() or {}
if _usb_selfhosted():
return selfhosted.kiosk_checkin(data)
@@ -1065,6 +1583,7 @@ def kiosk_checkin():
name = _resolve_badge_name(cur, badge)
if not name:
record_badge_miss()
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Badge not recognized - see IT to return this device',
@@ -1143,7 +1662,8 @@ def _auditcsv(rows, columns, filename):
AUDITCOLUMNS = ['timestamp', 'action', 'device_id', 'device_desc', 'badge',
'name', 'locker_location']
'name', 'locker_location', 'sanitized', 'scanned_viruses']
@usb_bp.route('/reports/audit', methods=['GET'])
@@ -1169,7 +1689,8 @@ def report_audit():
try:
with conn.cursor() as cur:
sql = ('SELECT l.timestamp, l.action, l.device_id, l.badge_number, '
'l.locker_location, d.device_desc, '
'l.locker_location, l.sanitized, l.scanned_viruses, '
'd.device_desc, '
'u.first_name, u.last_name '
'FROM checkinoutlog l '
'LEFT JOIN devices d ON d.device_id = l.device_id '
@@ -1197,6 +1718,13 @@ def report_audit():
'badge': row.get('badge_number') or '',
'name': name,
'locker_location': row.get('locker_location') or '',
# Only a return can carry these: the wipe and the scan
# happen when the drive comes back, so a check-out row
# showing them would invent a fact.
'sanitized': (selfhosted.auditflag(row.get('sanitized'))
if row.get('action') == 'check-in' else ''),
'scanned_viruses': (selfhosted.auditflag(row.get('scanned_viruses'))
if row.get('action') == 'check-in' else ''),
})
finally:
conn.close()

62
plugins/usb/api/selfhosted.py
View File

@@ -343,6 +343,27 @@ def kiosk_device(device_id):
})
def _badge_not_recognised(suffix=''):
"""One unrecognised scan, counted against the caller, then the refusal.
WHY COUNTED. The answer to a recognised badge is an employee's NAME, and a
badge is a short number, so this is a lookup somebody admitted to the kiosk
can walk. Network-position admission can now mean a whole VLAN rather than
one screen by the cabinet, so the number of badges a caller may guess wrong
has to be bounded. The budget lives in routes.py alongside the admission
checks; the import is deferred because routes imports this module.
The wording stays per-action ("to check this device out"), because the
person at the screen is being told what did not happen.
"""
from .routes import record_badge_miss
record_badge_miss()
return error_response(ErrorCodes.VALIDATION_ERROR,
'Badge not recognized - see IT' + suffix,
http_code=422)
def kiosk_session(data):
"""See routes.kiosk_session for why the list costs a valid badge."""
badge, badgeerror = _kiosk_badge_digits(data.get('badge'))
@@ -350,8 +371,7 @@ def kiosk_session(data):
return badgeerror
name = _resolve_name(badge)
if not name:
return error_response(ErrorCodes.VALIDATION_ERROR,
'Badge not recognized - see IT', http_code=422)
return _badge_not_recognised()
yours, available, others = [], [], 0
for device in USBDevice.query.filter_by(isactive=True).order_by(
@@ -377,8 +397,7 @@ def kiosk_others(data):
if badgeerror:
return badgeerror
if not _resolve_name(badge):
return error_response(ErrorCodes.VALIDATION_ERROR,
'Badge not recognized - see IT', http_code=422)
return _badge_not_recognised()
out = []
for device in USBDevice.query.filter_by(isactive=True, ischeckedout=True
@@ -419,10 +438,7 @@ def kiosk_checkout(data):
# path. A checkout nobody can trace to a person defeats the tracking.
name = _resolve_name(badge)
if not name:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Badge not recognized - see IT to check this device out',
http_code=422)
return _badge_not_recognised(' to check this device out')
now = datetime.now(timezone.utc).replace(tzinfo=None)
# The transition IS the guard. The ischeckedout test above reads, decides
@@ -477,10 +493,7 @@ def kiosk_checkin(data):
name = _resolve_name(badge)
if not name:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Badge not recognized - see IT to return this device',
http_code=422)
return _badge_not_recognised(' to return this device')
holderbadge = (device.currentuserid or '').strip()
if holderbadge and holderbadge != badge:
@@ -544,6 +557,22 @@ def _kiosk_flag(key, default):
return str(row.value).strip().lower() in ('true', '1', 'yes', 'on')
def auditflag(value):
"""Render a stored tri-state as 'yes', 'no', or '' for the report.
THREE states, not two. Both stores keep these nullable on purpose: a null
means nobody recorded it, which is not the same claim as 'no'. Rendering an
unrecorded return as 'No' would put a assertion in an audit record that
nobody ever made - the opposite of what the record is for. So unknown stays
blank and reads as the gap it is.
"""
if value is None or value == '':
return ''
if isinstance(value, str):
return 'yes' if value.strip() in ('1', 'true', 'True') else 'no'
return 'yes' if value else 'no'
def audit_rows(days):
"""Check-out / check-in events over a window (self-hosted mode).
@@ -576,11 +605,16 @@ def audit_rows(days):
'locker_location': device.storagelocation or '',
}
if checkout.checkouttime and (cutoff is None or checkout.checkouttime >= cutoff):
# No wipe or scan state on a take: both are recorded on the return,
# so putting them here would invent a fact.
rows.append(dict(base, timestamp=checkout.checkouttime.isoformat(),
action='check-out'))
action='check-out',
sanitized='', scanned_viruses=''))
if checkout.checkintime and (cutoff is None or checkout.checkintime >= cutoff):
rows.append(dict(base, timestamp=checkout.checkintime.isoformat(),
action='check-in'))
action='check-in',
sanitized=auditflag(checkout.waswiped),
scanned_viruses=auditflag(checkout.scannedviruses)))
# Newest first, and a total order so two events in the same second do not
# swap places between identical runs.

47
plugins/usb/frontend/views/USBKiosk.vue
View File

@@ -12,6 +12,16 @@
<p class="kiosk-hint">An administrator enables it in Settings under USB.</p>
</div>
<div v-else-if="refused" class="kiosk-step">
<p class="kiosk-prompt">This screen is not set up for the kiosk.</p>
<p class="kiosk-hint">
The kiosk is on, but this browser was not admitted. A wall screen is
provisioned once by opening it with its device token; a PC is admitted
by being on an allowed network. An administrator sets both in Settings
under USB.
</p>
</div>
<template v-else>
<div v-if="error" class="kiosk-error">{{ error }}</div>
@@ -145,6 +155,12 @@ const error = ref('')
const doneMessage = ref('')
const busy = ref(false)
const disabled = ref(false)
// The kiosk is on, but the server refused THIS caller: no device token in this
// browser and no allowed network. The server answers 404 rather than 403 so the
// route does not advertise itself, which means the client cannot know until it
// asks. Without this the page rendered as a working kiosk whose every call
// failed - the parts kiosk already says so, and this one did not.
const refused = ref(false)
const manualBadge = ref('')
const entryInput = ref(null)
let resetTimer = null
@@ -161,8 +177,33 @@ onMounted(async () => {
// An unreachable setting is not proof it is off; let the API decide.
disabled.value = false
}
if (!disabled.value) await probeAdmission()
focusEntry()
})
// ASK, rather than guess, whether this screen may use the kiosk.
//
// The enabled flag is public and says only that the site HAS a kiosk; whether
// THIS caller is admitted is decided per request, and the server answers 404
// rather than 403 so the route does not advertise itself. Without asking, the
// page rendered a keypad and only failed once somebody had typed their SSO -
// which reads as broken rather than as not set up.
//
// The parts kiosk answers the same question by checking whether its browser
// holds a token. That test is wrong here: a PC admitted by network carries no
// token and is perfectly entitled, so it would warn exactly the callers the
// allowlist exists to serve. Hence a real request.
//
// An empty badge is deliberate. The gate runs before any badge handling, so a
// refused caller gets 404 and an admitted one gets the badge validation error -
// no lookup, no write, nothing recorded either way.
async function probeAdmission () {
try {
await api.post('/usb/kiosk/session', { badge: '' })
} catch (probeError) {
if (probeError.response?.status === 404) refused.value = true
}
}
onBeforeUnmount(() => clearTimeout(resetTimer))
function focusEntry (event) {
@@ -214,7 +255,11 @@ async function startSession (value) {
step.value = 'pick'
focusFilter()
} catch (sessionError) {
error.value = messageFrom(sessionError, 'Could not look that badge up')
if (sessionError.response?.status === 404) {
refused.value = true
} else {
error.value = messageFrom(sessionError, 'Could not look that badge up')
}
} finally {
busy.value = false
focusEntry()

132
plugins/usb/frontend/views/USBSettings.vue
View File

@@ -41,11 +41,19 @@
<button type="button" class="btn btn-secondary" @click="generateToken">
Generate
</button>
<button type="button" class="btn btn-secondary" @click="copyLink">
Copy setup link
</button>
</div>
<p v-if="linkMessage" class="field-hint">{{ linkMessage }}</p>
<input v-if="link" :value="link" type="text" class="form-control"
readonly @focus="$event.target.select()" />
<p class="field-hint">
Provision each screen once by opening
<code>/usb-kiosk?token=&lt;this value&gt;</code> on it. That browser
keeps it; the address bar does not. Turning the kiosk on is a
Provision each screen once with <strong>Copy setup link</strong>
above and open that link on the screen. The link carries a single-use
setup code, not the token, so the token never reaches an address bar,
a browser history or a proxy log. That browser keeps the token
afterwards. Turning the kiosk on is a
decision, not a credential - without this token the routes answer 404,
so nobody who merely finds the URL can move a drive and sign it with a
badge they guessed.
@@ -55,6 +63,63 @@
</p>
</div>
<div class="form-group">
<label>Reverse proxies in front of this server</label>
<NetworkAllowlistPicker v-model="values.usb_kiosk_trusted_proxies" />
<p class="field-hint">
Only fill this in when a proxy sits between the kiosk screens and this
server. A request arriving FROM one of these addresses is never
treated as a kiosk itself, so a proxy that sits inside an allowed
network below cannot admit itself. Leave empty when browsers reach
the server directly.
</p>
</div>
<div class="form-group">
<label>Networks allowed without the token</label>
<NetworkAllowlistPicker v-model="values.usb_kiosk_allowed_cidrs" />
<p class="field-hint">
For people who open the kiosk on their own PC. A browser on one of
these networks is admitted with no token and no login, so keep this to
the shop-floor ranges rather than everything. Leave empty to require
the token from every caller. Wall screens keep using the token either
way.
</p>
<p class="field-hint token-warn">
Anyone on an allowed network can check a drive in or out against a
badge number they guess, and the record carries that person's name.
Badges are attribution, not authorization.
</p>
</div>
<div class="form-group">
<label class="switch-label">
<input type="checkbox" v-model="values.usb_kiosk_allow_wide_cidrs" />
Allow an unusually wide allowed network
</label>
<p class="field-hint">
Off, a range wider than a site normally uses is refused when you save,
and 0.0.0.0/0 is refused outright. Turn this on only to enter a wide
range deliberately: every address in it can move drives with no token
and no login.
</p>
</div>
<div class="form-group">
<label class="switch-label">
<input type="checkbox" v-model="values.usb_kiosk_badge_limit_enabled" />
Limit repeated unknown badge lookups
</label>
<p class="field-hint">
The kiosk turns a badge number into an employee name without a login,
so an allowed network can otherwise be walked to enumerate staff. On,
a source that misses too many times in a row is refused for a while.
</p>
<label>Unknown badges allowed before refusing</label>
<input v-model="values.usb_kiosk_badge_miss_max" type="text"
class="form-control" placeholder="15" />
</div>
<div class="form-group">
<label class="check-row">
<input type="checkbox" v-model="values.usb_kiosk_allow_return_for_others"
@@ -80,16 +145,30 @@
<script setup>
import { ref, onMounted } from 'vue'
import { settingsApi } from '@/api'
import api, { settingsApi } from '@/api'
import { withBase } from '@/utils/basePath'
import NetworkAllowlistPicker from '@/components/NetworkAllowlistPicker.vue'
const KEYS = ['usb_kiosk_enabled', 'usb_kiosk_allow_return_for_others',
'usb_kiosk_token']
'usb_kiosk_token', 'usb_kiosk_allowed_cidrs',
'usb_kiosk_trusted_proxies', 'usb_kiosk_allow_wide_cidrs',
'usb_kiosk_badge_limit_enabled', 'usb_kiosk_badge_miss_max']
// Keys whose value is text. Everything else is a switch stored as true/false.
// Saving a text setting through the boolean path is what replaced the kiosk
// token with the word 'true' and 404'd every provisioned screen.
const TEXTKEYS = ['usb_kiosk_token', 'usb_kiosk_allowed_cidrs',
'usb_kiosk_trusted_proxies', 'usb_kiosk_badge_miss_max']
const values = ref({
usb_kiosk_enabled: false,
usb_kiosk_allowed_cidrs: '',
usb_kiosk_allow_return_for_others: true,
usb_kiosk_token: '',
usb_kiosk_trusted_proxies: '',
usb_kiosk_allow_wide_cidrs: false,
usb_kiosk_badge_limit_enabled: true,
usb_kiosk_badge_miss_max: '15',
})
const tokenJustGenerated = ref(false)
@@ -106,6 +185,35 @@ const saving = ref(false)
const message = ref('')
const error = ref('')
// The settings API masks the token, so the field above shows '********' rather
// than the value. Copying that provisions a screen with the mask and every
// kiosk call answers 404 with nothing to say why. This asks the server for the
// real link instead, so the copy step no longer depends on catching the one
// moment after Generate when the value is on screen.
const link = ref('')
const linkMessage = ref('')
async function copyLink () {
linkMessage.value = ''
link.value = ''
try {
const response = await api.get('/usb/kiosk/provisioning-link')
const url = window.location.origin + withBase(response.data.data.url)
link.value = url
try {
await navigator.clipboard.writeText(url)
linkMessage.value = 'Copied. Open this once on the kiosk screen.'
} catch (clipboardError) {
// Clipboard access is refused over plain http and in some locked-down
// browsers. The link is shown below either way, so this is not a failure.
linkMessage.value = 'Copy this and open it once on the kiosk screen.'
}
} catch (linkError) {
linkMessage.value = linkError.response?.data?.data?.error?.message
|| 'Could not build the setup link.'
}
}
function asBool (value) {
return String(value).toLowerCase() === 'true' || value === true || value === 1
}
@@ -115,9 +223,9 @@ async function load () {
const response = await settingsApi.list({ category: 'usb' })
for (const row of (response.data.data || [])) {
if (!KEYS.includes(row.key)) continue
// The token is text (the server sends a mask once set); only the two
// switches are booleans.
values.value[row.key] = row.key === 'usb_kiosk_token'
// Text settings stay text (the server sends a mask for the token once it
// is set); only the switches are booleans.
values.value[row.key] = TEXTKEYS.includes(row.key)
? (row.value || '')
: asBool(row.value)
}
@@ -132,10 +240,10 @@ async function save () {
error.value = ''
try {
for (const key of KEYS) {
// Saving the token as true/false replaced the real token with the word
// 'true', and every provisioned kiosk then got 404. The server keeps the
// stored token when it receives the mask back unchanged.
const value = key === 'usb_kiosk_token'
// Saving a text setting through the boolean path replaced the real token
// with the word 'true', and every provisioned kiosk then got 404. The
// server keeps the stored token when it receives the mask back unchanged.
const value = TEXTKEYS.includes(key)
? String(values.value[key] ?? '')
: (values.value[key] ? 'true' : 'false')
await settingsApi.update(key, value)

14
plugins/usb/frontend/views/usbSettings.spec.js
View File

@@ -49,9 +49,23 @@ describe('USBSettings save', () => {
usb_kiosk_enabled: 'true',
usb_kiosk_allow_return_for_others: 'false',
usb_kiosk_token: MASK,
usb_kiosk_allowed_cidrs: '',
})
})
it('saves the network allowlist as text, not as true/false', async () => {
// Same trap the token fell into: a text setting sent through the boolean
// path becomes the word 'true', and the gate then admits nobody while the
// settings page still shows something plausible.
const wrapper = await render([
{ key: 'usb_kiosk_enabled', value: 'true' },
{ key: 'usb_kiosk_allowed_cidrs', value: '192.0.2.0/24, 198.51.100.0/24' },
])
await wrapper.find('button.btn-primary').trigger('click')
await flushPromises()
expect(saved().usb_kiosk_allowed_cidrs).toBe('192.0.2.0/24, 198.51.100.0/24')
})
it('saves a generated token as the token itself', async () => {
const wrapper = await render([
{ key: 'usb_kiosk_enabled', value: 'true' },

2
plugins/usb/manifest.json
View File

@@ -5,7 +5,7 @@
"description": "USB device checkout management",
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.1.0,<1.0.0",
"core_version": ">=0.35.0,<1.0.0",
"api_prefix": "/api/usb",
"default_enabled": false
}

130
plugins/usb/plugin.py
View File

@@ -121,6 +121,18 @@ class USBPlugin(BasePlugin):
The kiosk is an UNAUTHENTICATED write, so it ships off and a site turns
it on deliberately. Opening that door is a decision about the door.
DECLARING A KEY HERE IS HALF THE JOB. A site on an air-gapped server
changes a setting through the USB settings page or not at all, and that
page renders a hardcoded list - frontend/views/USBSettings.vue, its KEYS
and TEXTKEYS arrays plus a field for each. A key declared here and absent
there is a constant with extra steps.
tests/test_plugin_settings_reachable.py is the gate for it, and it is
currently RED for four of the keys below: usb_kiosk_trusted_proxies,
usb_kiosk_allow_wide_cidrs, usb_kiosk_badge_limit_enabled and
usb_kiosk_badge_miss_max. All four need a field on that page. The last
two are a checkbox and a number; the first two belong beside the existing
allowlist picker, since all three describe the same decision.
"""
return [
{
@@ -145,10 +157,101 @@ class USBPlugin(BasePlugin):
# Masking keys on the NAME containing 'token'.
'valuetype': 'string',
'category': 'usb',
'description': 'Device token the USB kiosk sends as '
'X-Kiosk-Token. Generate one, then open the '
'kiosk once as /usb-kiosk?token=<value> on each '
'screen',
'description': 'Device token the USB kiosk sends on every '
'call. Generate one, then use "Copy setup link" '
'and open that link once on each screen: the '
'link carries a single-use code, not the token, '
'so the token never sits in a URL or a proxy '
'log. Screens provisioned by an older '
'/usb-kiosk?token=<value> link keep working',
},
{
'key': 'usb_kiosk_allowed_cidrs',
'value': '',
'valuetype': 'string',
'category': 'usb',
'description': 'Networks allowed to use the kiosk WITHOUT the '
'device token, as comma-separated CIDRs (for '
'example 192.0.2.0/24). For people who open '
'the kiosk on their own PC and never log in - '
'provisioning a token into every browser is not '
'workable. Empty = token only. Wall screens keep '
'using the token either way. A range wider than '
'/16 is refused unless usb_kiosk_allow_wide_cidrs '
'is on, and 0.0.0.0/0 always is. This is only an '
'admission check if the caller cannot choose the '
'address the app sees: on Windows that means the '
'IIS rewrite rule is live (-ClientIpSource '
'direct), or a reverse proxy in front appends the '
'caller to X-Forwarded-For. The installer checks '
'both and says so. Until one holds, leave this '
'empty and provision each screen with the token.',
},
{
'key': 'usb_kiosk_trusted_proxies',
'value': '',
'valuetype': 'string',
'category': 'usb',
'description': 'Addresses or CIDRs of the reverse proxies and '
'load balancers that sit in front of this '
'server, comma separated. These are SUBTRACTED '
'from the kiosk allowlist: a request that arrives '
'wearing one of these addresses is the proxy '
'talking about itself - it forwarded nothing, or '
'nothing forwarded at all - so it says nothing '
'about who the caller is and is refused rather '
'than admitted. Set this whenever a proxy is in '
'front, because a proxy usually sits on the very '
'plant network the allowlist names, and would '
'otherwise match it. The app never reads '
'X-Forwarded-For itself; the layer in front of '
'it already resolved the caller address.',
},
{
'key': 'usb_kiosk_allow_wide_cidrs',
'value': 'false',
'valuetype': 'boolean',
'category': 'usb',
'description': 'Allow a kiosk network range wider than /16 '
'(/48 for IPv6) to be saved. Off by default '
'because the usual reason a wide range appears '
'is a typo, and everyone inside one is admitted '
'to an unauthenticated write with no token and '
'no login. Turn it on only for a site that '
'really is one flat network that wide. '
'0.0.0.0/0 and ::/0 stay refused either way.',
},
{
'key': 'usb_kiosk_badge_limit_enabled',
'value': 'true',
'valuetype': 'boolean',
'category': 'usb',
'description': 'Stop answering badge lookups for a caller that '
'has scanned too many badges this site does not '
'recognise. On by default: every kiosk call '
'answers with the employee NAME, so an unbounded '
'lookup is a directory anyone admitted can walk, '
'and admission by network can mean a whole VLAN '
'rather than one screen by the cabinet. An '
'ordinary day at the cabinet produces no misses '
'at all. Turn it off only for a site whose '
'scanners misread often enough that people are '
'being locked out mid-shift.',
},
{
'key': 'usb_kiosk_badge_miss_max',
'value': '15',
'valuetype': 'integer',
'category': 'usb',
'description': 'How many unrecognised badges one caller may '
'scan within ten minutes before the kiosk stops '
'answering for the rest of that window. Only '
'misses count, so this can sit well above a bad '
'scanning day and still stop somebody walking '
'the badge range with a loop. Blank, zero or '
'anything that is not a number falls back to 15 '
'- zero here would refuse the first scan of '
'every window, not lift the limit.',
},
{
'key': 'usb_kiosk_allow_return_for_others',
@@ -185,6 +288,25 @@ class USBPlugin(BasePlugin):
def init_app(self, app: Flask, db_instance) -> None:
"""Initialize plugin with Flask app."""
# The kiosk CIDR settings decide who may write to this site without
# logging in, and the core settings API stores what it is given: it
# checks valuetype and has no per-key hook a plugin can declare. These
# are this plugin's keys, so this plugin validates them, at the save,
# where the administrator who typed the range is still there to read
# why it was refused.
#
# Guarded because init_app also runs when an operator enables this
# plugin on a RUNNING server, and Flask refuses a setup method after
# the first request. Symmetric with the blueprint, which the plugin
# manager already declines to register at that point: the guard starts
# working at the next restart, and a plugin being switched on is not a
# moment to fail the whole load over.
from .api.routes import validate_kiosk_settings_write
try:
app.before_request(validate_kiosk_settings_write)
except Exception:
logger.warning('USB kiosk CIDR validation is not active until this '
'app restarts (plugin enabled after first request)')
logger.info(f"USB plugin initialized (v{self.meta.version})")
def on_install(self, app: Flask) -> None:

65
scripts/check-gates-ran.sh
View File

@@ -6,7 +6,7 @@
#
# tests/test_api_inventory_parity.py skipif docs/api-inventory.json missing
# tests/test_openapi_spec.py skipif docs/openapi.json missing
# tests/test_docs_publishable.py skipif docs/ missing
# tests/test_docs_publishable.py skipif docs/*.md missing
#
# That is correct in a published checkout, where docs/ becomes the wiki and
# there is genuinely nothing to check. It was silently wrong on GitHub, which
@@ -30,21 +30,33 @@ cd "$(dirname "$0")/.."
# always a fault.
GATES_EVERYWHERE=(
"tests/test_openapi_spec.py"
"tests/test_api_inventory_parity.py"
"tests/test_core/test_public_read_surface.py"
"tests/test_core/test_authz.py"
"tests/test_plugin_migrations.py"
)
# Gates that need files the publication export deliberately leaves behind -
# docs/ becomes the wiki, so a published checkout genuinely has nothing to check
# and skipping is correct there. In THIS repo, skipping means something broke.
# docs/*.md becomes the wiki, so a published checkout genuinely has nothing to
# check and skipping is correct there. In THIS repo, skipping means something
# broke.
#
# The inventory parity gate used to be here, and that was the hole it exists to
# close. docs/api-inventory.json is not wiki prose: it is a generated artifact,
# tracked exactly like docs/openapi.json beside it, because the app publishes the
# spec built from it. Keying the gate to prose meant a checkout with no docs/*.md
# - which is every fresh clone and every worktree - called itself published and
# ran the route check nowhere. That is where 24 undocumented routes came in.
GATES_INTERNAL_ONLY=(
"tests/test_api_inventory_parity.py"
"tests/test_docs_publishable.py"
)
# A published checkout is one with no docs/ prose. openapi.json alone does not
# make it internal: that file ships because the app serves it at runtime.
#
# NOTE: docs/wiki-drafts/*.md does NOT count here, and that is a standing
# question rather than an oversight - see the comment on the skipif in
# tests/test_docs_publishable.py. Keep the two definitions identical.
if compgen -G "docs/*.md" > /dev/null; then
GATES=("${GATES_EVERYWHERE[@]}" "${GATES_INTERNAL_ONLY[@]}")
echo "==> Internal checkout: every gate must run"
@@ -56,11 +68,26 @@ fi
PYTHON="${PYTHON:-python}"
[ -x venv/bin/python ] && PYTHON=venv/bin/python
# REFUSE TO RUN WITHOUT A REAL INTERPRETER. In a git worktree there is no venv,
# `python` is often not on PATH at all, and the pytest line below ends in
# `|| true` - so the run died 127, the report was a single "command not found"
# line, every gate matched nothing, and the script still printed "All gates ran."
# and exited 0. A gate-checker that passes when it ran nothing is worse than no
# gate-checker, because its exit code is what everything else trusts.
if [ -z "${1:-}" ] && ! "$PYTHON" -c 'import pytest' >/dev/null 2>&1; then
echo "FAIL: '$PYTHON' cannot import pytest, so no gate can run here." >&2
echo " Set PYTHON=/path/to/python (a venv that has pytest) and re-run." >&2
exit 1
fi
report="${1:-}"
if [ -z "$report" ]; then
report="$(mktemp)"
# -rs lists every skip with its reason, which is the thing being checked.
"$PYTHON" -m pytest "${GATES[@]}" -q -rs > "$report" 2>&1 || true
# -rf and -rE add the "FAILED <nodeid>" and "ERROR <nodeid>" lines, which -rs
# alone does NOT emit: this script grepped for them anyway, so on a real
# failure it printed its header and then nothing at all.
"$PYTHON" -m pytest "${GATES[@]}" -q -rsfE > "$report" 2>&1 || true
fi
echo "==> Checking that every gate ran"
@@ -82,10 +109,34 @@ for gate in "${GATES[@]}"; do
echo " ran $gate"
done
if grep -qE "^[0-9]+ failed|error" "$report"; then
# Name what failed, from whatever the report actually contains. This script can
# be handed a report somebody else produced, and that one may have been run
# without -rf, so the "FAILED <nodeid>" lines are not guaranteed to be there.
# The FAILURES section underlines are: pytest prints "____ test_name ____" above
# every traceback whether the short summary was asked for or not.
list_failures() {
if grep -qE '^(FAILED|ERROR) ' "$1"; then
grep -E '^(FAILED|ERROR) ' "$1"
return
fi
if grep -qE '^_{3,} .+ _{3,}$' "$1"; then
sed -n 's/^_\{3,\} \(.*[^_]\) _\{3,\}$/\1/p' "$1"
return
fi
# Nothing named. Hand over the count line rather than an empty list, so the
# reader knows to open the report instead of believing there is nothing here.
grep -E '[0-9]+ (failed|error)' "$1" | tail -1
echo "(no test names in this report; run without an argument to get them)"
}
# The old condition was "^[0-9]+ failed" or the bare word "error" anywhere,
# which matches a test name, a warning or a docstring. Anchor on the shapes
# pytest actually prints: a short-summary line, a count in the tally, or an
# interrupted collection.
if grep -qE '^(FAILED|ERROR) |(^|, )[0-9]+ (failed|error)|^!+ Interrupted' "$report"; then
echo
echo "Gate tests FAILED (not merely skipped):"
grep -E "^FAILED|^ERROR" "$report" | sed 's/^/ /'
list_failures "$report" | sed 's/^/ /'
failed=1
fi

161
scripts/check-naming-and-style.sh
View File

@@ -78,10 +78,18 @@ fi
# DO NOT ADD --exclude=<file> TO THE ARRAY ABOVE.
#
# In this grep build a single file-level --exclude silently CANCELS EVERY
# --include filter on the same command. Bisected: with only --exclude-dir flags
# a shorthand scan restricted to --include='*.py' returns 0 markdown hits; add
# any one --exclude and the same scan returns 1142. Directory excludes are
# unaffected - it is specifically the file form.
# --include filter that comes AFTER it on the same command. Bisected: with only
# --exclude-dir flags a shorthand scan restricted to --include='*.py' returns 0
# markdown hits; add any one --exclude ahead of the includes and the same scan
# returns 1142. Re-proved on GNU grep 3.11 with six files of one literal:
# --include='*.py' --include='*.cs' -> 2 hits, as written
# --exclude=README.md --include='*.py' --include='*.cs' -> all 6 hits
# --include='*.py' --include='*.cs' --exclude=README.md -> 2 hits
# Directory excludes are unaffected in any position - it is specifically the
# file form. DO NOT rely on writing the exclude last: the position rule is not
# documented behaviour, the array above is expanded ahead of every include list
# in this file, and a later edit that reorders the flags would silently widen
# the scan again with nothing failing. Post-filter instead.
#
# This is not theoretical, it was live: --exclude=findings.sarif and
# --exclude=report.md sat in this array and had been quietly disabling the
@@ -246,6 +254,107 @@ if [ -n "$SNAKE_COLS" ]; then
VIOLATIONS=$((VIOLATIONS + 1))
fi
# The check above reads MODEL attributes, and a model attribute is not where a
# column is born - a migration is. migrations/versions/ is filtered out of
# GIT_FILES_FILTER and out of EXCLUDES for every other check in this file, so
# from the day this gate was written not one created column name was ever
# looked at. A model whose attribute reads fine still ships whatever the
# migration wrote, because the migration is what the database got.
#
# CREATION SITES ONLY: sa.Column(<name>) and an alter_column rename. drop_column
# and an ordinary alter_column name a column that already exists, so a bad name
# there was already wrong where it was created and flagging it twice only makes
# the same rename look like two findings.
#
# awk, not grep, because sa.Column( is routinely followed by a newline before
# the name (op.add_column('applications', sa.Column(\n 'isrequired', ...)) and a
# line-at-a-time grep sees neither half: the first line has no name and the
# second has no sa.Column. One line of lookahead joins them - that shape is 9
# of the ~1180 call sites here, so dropping it is not acceptable.
#
# KNOWN LIMIT, shared with every text scan in this file: a name held in a module
# constant or walked out of a list (_SNAPSHOT_COLUMNS and the COLUMN_NAME
# constants in the endpoints and notifications chains) is not a literal and is
# not seen. Those are all clean today; a text scan cannot promise they stay so.
echo "==> Checking for snake_case DB column names in migrations..."
# NUL-delimited end to end, and the list is PIPED rather than held in a
# variable: command substitution drops null bytes, which would silently rejoin
# a path containing a space into two half-paths.
SNAKE_MIG_COLS=$(
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
git ls-files -z -- '*migrations/versions/*.py' 2>/dev/null
else
find . -path '*/migrations/versions/*.py' -not -path '*/venv/*' \
-not -path '*/node_modules/*' -not -path '*/.plugins-staged/*' -print0 2>/dev/null
fi | xargs -0 -r awk -v Q="'" '
# THE RULE IS LOWERCASE CONCATENATED, so the test is on the NAME, not on
# the shape of the name. The first version of this matched
# [a-z0-9]*_[a-z0-9_]* inside the quotes, which asks "does a lowercase
# name contain an underscore" - so sa.Column("Monitor_Name", ...) passed
# because of the leading capital, and so did every CamelCase name with no
# underscore at all. Extract whatever literal the call names and test it
# against ^[a-z][a-z0-9]*$ instead: anything else is a finding.
#
# SHAPES a migration writes a column name in, all of which land here:
# sa.Column("x", ...) positional
# sa.Column(name="x", type_=sa.String()) keyword, first or later
# op.add_column("t", sa.Column("x", ...)) the sa.Column( inside it
# with op.batch_alter_table(...) as batch_op:
# batch_op.add_column(sa.Column("x", ...)) likewise
# op.alter_column("t", "old", new_column_name="x") the rename target
# drop_column and a plain alter_column name a column that already exists,
# so they stay out: a bad name there was already wrong where it was
# created, and reporting it twice makes one rename look like two.
function nameat(s, re, quote, rest, stop) {
# the literal that follows re, where re ENDS on the opening quote
if (!match(s, re)) return ""
quote = substr(s, RSTART + RLENGTH - 1, 1)
rest = substr(s, RSTART + RLENGTH)
stop = index(rest, quote)
if (stop == 0) return ""
return substr(rest, 1, stop - 1)
}
BEGIN {
QQ = Q "\""
callre = "sa\\.Column\\("
posre = "^[ \t]*[" QQ "]"
kwre = "(^|[^A-Za-z0-9_])name[ \t]*=[ \t]*[" QQ "]"
renre = "new_column_name[ \t]*=[ \t]*[" QQ "]"
contre = "sa\\.Column\\([ \t]*$"
okre = "^[a-z][a-z0-9]*$"
}
FNR == 1 { held = ""; heldline = 0 }
{
line = $0; lineno = FNR
if (held != "") { line = held line; lineno = heldline; held = "" }
if (line ~ contre) { held = line; heldline = lineno; next }
# walk every sa.Column( on the line: op.create_table() writes several
# on one line and each names a column of its own.
rest = line
while (match(rest, callre)) {
rest = substr(rest, RSTART + RLENGTH)
# bound this call at the next sa.Column( so a later call cannot
# lend this one its name= argument
if (match(rest, callre)) chunk = substr(rest, 1, RSTART - 1)
else chunk = rest
colname = nameat(chunk, posre)
if (colname == "") colname = nameat(chunk, kwre)
if (colname != "" && colname !~ okre)
printf "%s:%d: %s\n", FILENAME, lineno, colname
}
colname = nameat(line, renre)
if (colname != "" && colname !~ okre)
printf "%s:%d: %s\n", FILENAME, lineno, colname
}' || true)
if [ -n "$SNAKE_MIG_COLS" ]; then
echo "FAIL: column name created in a migration is not lowercase concatenated"
echo " (see CONTRIBUTING.md):"
echo "$SNAKE_MIG_COLS"
echo
VIOLATIONS=$((VIOLATIONS + 1))
fi
echo "==> Checking for snake_case ForeignKey targets..."
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
SNAKE_FK=$(git ls-files -z '*.py' | eval "$GIT_FILES_FILTER" \
@@ -402,6 +511,43 @@ echo "==> Checking for site-specific literals in product code (ADR-015)..."
# only worth having where
# the leaks are.
#
# agent/ arrived late and is the reason this note now names it. The C# endpoint
# agent is compiled once and installed on every bay at every site, and it holds
# exactly the values this rule is about - a default base URL, a server hostname,
# an SPKI pin. A site literal baked into a .cs file there ships to the next site
# INSIDE A BINARY, where no reviewer reads it and no grep over the deployed tree
# finds it. agent/appsettings.json and the agent's .ps1 utilities come in free,
# since *.json and *.ps1 were already in the include list.
#
# --exclude=README.md IS GONE, and nothing replaces it. That flag used to sit on
# this command ABOVE the --include list, and on GNU grep 3.11 a file-level
# --exclude placed before the includes cancels all of them: proven
# here by planting the same literal in x.py, x.cs, x.txt, x.md, x.csproj and
# x.gitignore, then running the scan with and without the flag. Without it, two
# files match (.py and .cs, as the flags say). With it, all six match. So this
# check was reading every file under every directory listed below, and
# --include='*.cs' was decorative - adding agent/ to the scan did NOT mean the
# C# agent was being scanned for the reason the include list claims, and any
# future narrowing of the include list would have been silently inert.
#
# No post-filter took its place because it never had anything to skip: *.md is
# not in the include list below, so README.md cannot match this scan at all. The
# flag's only effect was the one above.
#
# The .csproj post-filter is kept even though *.csproj is no longer in scope. The
# only site names under agent/ sit in the version-history comment of one project
# file, recording which sites a past build went to; that is a record of what
# happened, not a value a bay receives, and a .csproj is a build input that is
# never deployed. Keeping the filter means adding *.csproj to the include list
# later cannot quietly turn that comment into a gate failure.
#
# bin/, obj/, publish/ and .vs/ are excluded ON THIS COMMAND rather than left to
# EXCLUDES, which is empty whenever USE_GIT_LS_FILES is 1. A developer machine
# with Visual Studio has all four under agent/, holding vendored runtime DLLs
# and per-edit source snapshots, and walking them is what would make this scan
# slow enough to stop being run. Directory excludes are safe here; a file-level
# --exclude is not (see the warning above the EXCLUDES array).
#
# Fleet-wide vocabulary (gea-shopfloor-*) is NOT matched: it is overridable
# through the pctypemap settings and is not one site's data.
#
@@ -432,11 +578,11 @@ SITE_TENANTS='geaero''spaceqa'
SITE_PATTERNS="${SITE_HOST}|${SITE_DOMAIN}|\bwjs\b|${SITE_NAME}|${SITE_NETS}|${SITE_TENANTS}"
SITE_HITS=$(grep -rPni --binary-files=without-match \
"$SITE_PATTERNS" "${EXCLUDES[@]}" \
--exclude=README.md \
--include='*.py' --include='*.vue' --include='*.js' \
--include='*.ps1' --include='*.psm1' --include='*.sh' --include='*.iss' \
--include='*.json' --include='*.html' \
plugins/ shopdb/ scripts/ deploy/ frontend/src/ tools/ 2>/dev/null \
--include='*.json' --include='*.html' --include='*.cs' \
--exclude-dir=bin --exclude-dir=obj --exclude-dir=publish --exclude-dir=.vs \
plugins/ shopdb/ scripts/ deploy/ frontend/src/ tools/ agent/ 2>/dev/null \
| grep -v '/tests\?/' \
| grep -v '\.spec\.js:' \
| grep -v 'site_imports/' \
@@ -445,6 +591,7 @@ SITE_HITS=$(grep -rPni --binary-files=without-match \
| grep -v '\.plugins-staged/' \
| grep -v 'check-naming-and-style\.sh' \
| grep -v 'export-github\.sh' \
| grep -v '\.csproj:' \
| grep -v 'ADR-015-OK' || true)
if [ -n "$SITE_HITS" ]; then
COUNT=$(echo "$SITE_HITS" | wc -l)

585
scripts/gen_openapi.py
View File

@@ -1,21 +1,54 @@
"""Generate docs/openapi.json from docs/api-inventory.json.
"""Generate docs/openapi.json from docs/api-inventory.json, and refresh that
inventory from the routes the app actually serves.
The inventory is a list of {surface, endpoints:[{method,path,auth,params,purpose,
example}]} objects (one per API surface). It is HAND-MAINTAINED - nothing
generates it, and the docstring here used to imply otherwise. That is deliberate:
its value is the prose, and what an endpoint is FOR cannot be derived from the
code. Coverage is enforced instead, by tests/test_api_inventory_parity.py, which
fails when a served route has no entry.
example}]} objects (one per API surface). Two halves, maintained differently:
So: describe the endpoint in api-inventory.json, then re-run this. Served
interactively at /api/docs (see shopdb/core/api/docs.py).
STRUCTURE (surface, method, path, auth) is DERIVED. It is walked out of the
live url_map and the decorators on each view, so it cannot drift from the
code. It had: this branch carried no inventory at all, so the generator had no
input, the parity test skipped, and 24 routes served traffic documented
nowhere.
venv/bin/python scripts/gen_openapi.py
PROSE (purpose, params, example) is HUMAN. What an endpoint is FOR cannot be
derived, so a refresh never overwrites text that is already there - it fills
in a new route from its docstring and leaves every existing entry's words
alone.
So: add a route, run --refresh, and replace the seeded prose with the reason the
route exists. Coverage is enforced on top of that by
tests/test_api_inventory_parity.py, which fails when a served route has no entry
AND when a committed auth string disagrees with what this file re-derives.
Served interactively at /api/docs (see shopdb/core/api/docs.py).
venv/bin/python scripts/gen_openapi.py # spec from the inventory
venv/bin/python scripts/gen_openapi.py --refresh # inventory first, then spec
THE AUTH STRING IS A GRAMMAR, not free text. Its first word names the mechanism
and everything after it is prose; '-optional' on that word means the gate can be
satisfied without a credential. See AUTH_SCHEME and _security. Derived strings
come from DECORATOR_AUTH, HANDLER_AUTH and INLINE_AUTH below, so a route whose
gate this cannot see says so rather than guessing.
KNOWN LIMITATION: QUERY PARAMETERS ARE NOT DERIVED. Path arguments come out of
the rule, and a body is declared for the verbs that take one, but nothing here
reads request.args out of a handler - so every route whose real interface is a
query string ('?q=', '?page=', '?status=') publishes params as 'not documented'
and an operation with no `parameters` beyond its path. That is a gap in the
generated spec, not in the route: a machine client built from this document can
call those endpoints but cannot filter, search or page them. Filling it means
either a handler sweep for request.args reads - which cannot see a key built at
runtime, so it would be right most of the time and silently wrong the rest - or
writing the query contract into the inventory's PROSE half by hand, where a
human owns it. The prose half is the honest option and is where it should land.
The one query credential that IS expressed is the kiosk setup code, because it
is auth rather than a filter and rides in as a securityScheme.
"""
import json
import os
import re
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
REPO = os.path.dirname(HERE)
@@ -68,23 +101,62 @@ def _norm_path(p):
return normalised
# The machine-readable half of an auth string is its FIRST word. Everything
# after it is prose for a human reader. An optional gate is spelled by suffixing
# that word with '-optional'.
#
# Substring matching is what this replaces, and it is why nine operations
# published `security: []` - "no credentials needed" - while their handler
# required one. 'none (single-use bootstrap token...)' starts with 'none', so
# the old sweep published the two bootstrap routes as open, and one of them
# hands out the installable agent package and spends a use. A leading token
# cannot be fooled by the prose that explains it.
AUTH_SCHEME = {
'jwt': 'bearerAuth',
'permission': 'bearerAuth',
'api-key': 'apiKeyAuth',
'setup-code': 'setupCodeAuth',
}
OPTIONAL_SUFFIX = '-optional'
def _auth_head(auth):
"""The mechanism word at the front of an auth string, lowercased."""
match = re.match(r'[a-z-]+', (auth or '').strip().lower())
return match.group(0) if match else ''
def _security(auth):
"""OpenAPI security for one endpoint's documented auth string.
`jwt-optional` is the interesting case, and the common one - 121 endpoints.
It means "works logged out, returns more when logged in", which OpenAPI
expresses as a list containing BOTH the empty requirement and the scheme.
Publishing those as bearer-required told every reader, human and machine,
that a public endpoint needs a token.
Optional auth is the interesting case. It means "works without a
credential, does more with one", which OpenAPI expresses as a list carrying
BOTH the empty requirement and the scheme. Two shapes reach it:
jwt-optional works signed out; the handler decides what shows.
api-key-optional token OR network. A kiosk admits either the per-device
token or a source address inside the site's CIDR
allowlist, so an allowlisted caller sends no
credential. Publishing that as `[]` is not a lie, but
it is not the truth either: it tells a reader there is
no token to send, and the allowlist is invisible.
Optional says both halves exist.
An unrecognised word publishes as bearer-required, which is the safe
direction to be wrong in: a reader who brings a token they did not need
loses nothing, a reader told none was needed is locked out on the day.
"""
a = (auth or '').lower()
if 'optional' in a:
return [{}, {'bearerAuth': []}]
if a in ('none', 'public', '') or a.startswith('none') or a.startswith('public'):
head = _auth_head(auth)
optional = head.endswith(OPTIONAL_SUFFIX)
if optional:
head = head[:-len(OPTIONAL_SUFFIX)]
if not head or head in ('none', 'public'):
return []
if 'api-key' in a or 'api_key' in a or 'x-api-key' in a:
return [{'apiKeyAuth': []}]
return [{'bearerAuth': []}]
scheme = AUTH_SCHEME.get(head, 'bearerAuth')
if optional:
return [{scheme: []}, {}]
return [{scheme: []}]
def _path_parameters(rawpath):
@@ -237,7 +309,10 @@ def build(surfaces):
'`success_response` envelope: `{status, data, meta}`. Auth: Bearer '
'JWT (login or a managed PAT) for `jwt`/`admin`/`permission:*`; '
'`X-API-Key` for collector/managed-token endpoints; public endpoints '
'need neither.'),
'need neither. A permission-gated route is published as '
'bearer-required, which is what a default install does - a site '
'that grants the permission to the `guest` role makes that same '
'route answer a caller with no token.'),
},
# One site's production hostname was hardcoded here, so every generated
# spec published it to the public wiki and offered a second site a
@@ -302,18 +377,478 @@ def build(surfaces):
'securitySchemes': {
'bearerAuth': {'type': 'http', 'scheme': 'bearer', 'bearerFormat': 'JWT',
'description': 'Login token or a managed Personal Access Token (scoped).'},
# One scheme covers every non-JWT header credential in this API,
# because the header NAME varies per surface and each operation's
# **Auth:** line says which one it wants. Publishing a scheme per
# header would be tidier and no more informative.
'apiKeyAuth': {'type': 'apiKey', 'in': 'header', 'name': 'X-API-Key',
'description': 'Managed service token (e.g. collector.ingest, geenforce.fetch).'},
'description': 'A managed service or device token in a header. '
'X-API-Key for collector and managed tokens '
'(collector.ingest, geenforce.fetch); '
'X-Endpoint-Bootstrap-Token for agent bootstrap; '
'X-Kiosk-Token for a kiosk screen. The operation '
'description names the header and, where the gate '
'is token-OR-network, the allowlist that is the '
'other half.'},
'setupCodeAuth': {'type': 'apiKey', 'in': 'query', 'name': 'code',
'description': 'A single-use provisioning code, redeemed in the '
'query string because the caller is a browser being '
'pointed at a URL. Spent on any attempt, right or '
'wrong, and short-lived.'},
}},
'tags': tags,
'paths': paths,
}
# ---------------------------------------------------------------------------
# The derived half: refresh the inventory from the live url_map (--refresh)
# ---------------------------------------------------------------------------
# Only the API. The frontend shell and its assets are served by this app but are
# not the contract this document describes, and tests/test_api_inventory_parity
# walks the same /api-only surface - an entry for anything else reports there as
# a documented route that is not served.
API_PREFIX = '/api'
# The auth string for a route whose gate this cannot see. Deliberately not
# 'none': an unclassified route is the one case where a guess writes a lie into
# the published contract, and a site reads that contract to decide what its
# guest role and its firewall may reach. This greps, and it renders at /api/docs.
# Hand-editing the inventory does NOT clear this: auth is derived, so the next
# --refresh writes it straight back. Teaching the sweep below is the fix, which
# is also the only way the classification survives the route moving.
NEEDS_REVIEW = ('review: no gate found on the view - teach the sweep in '
'scripts/gen_openapi.py, or gate the view')
# The jwt-optional wording, in one place because three branches reach it.
JWT_OPTIONAL = 'jwt-optional (works signed out; the handler decides what shows)'
# Decorators that ARE the gate. require_permission and require_role are not
# here: they stamp _requires_permission onto the wrapper, which is read first
# and carries the permission name itself.
DECORATOR_AUTH = {
'require_api_key': 'api-key (per-plugin collector token)',
'require_api_key_legacy': 'api-key (collector token, legacy shared key)',
'require_fetch_token': 'api-key (geenforce fetch token, or the IP allowlist)',
'require_report_token': 'api-key (geenforce report token)',
}
# Gates that live INSIDE the handler, found by the call the handler makes. An
# agent, a kiosk screen and a bootstrapping PC each authenticate on something
# that is not a JWT, so none of them can be gated by a decorator - and from
# outside the view, an inline gate and no gate at all look exactly alike. That
# is how 165 GET routes came to answer anybody.
HANDLER_AUTH = (
('_agent_auth_or_error', 'api-key (endpoint agent credential, or the fleet '
'key while sharedkeyallowed is on)'),
('authenticate_agent', 'api-key (endpoint agent credential, or the fleet '
'key while sharedkeyallowed is on)'),
('collector_ingest', 'api-key (endpoints collector token)'),
('_check_collector_auth', 'api-key (per-plugin collector token)'),
('collector_authorized', 'api-key (per-plugin collector token)'),
# Token OR network, either sufficient, so neither 'none' nor 'api-key' is
# the truth on its own: an allowlisted PC sends no credential at all, and a
# screen by the cabinet sends the device token from anywhere. 'none' was
# published here, which told a reader there is no token to send and left
# the allowlist invisible. Optional says both halves exist.
('_kiosk_open', 'api-key-optional (kiosk: the per-device token in the '
'X-Kiosk-Token header, OR a caller address inside the '
"site's kiosk CIDR allowlist - an allowlisted caller sends "
"no credential; 404 while the site's kiosk flag is off)"),
# REQUIRED, not 'none'. The handler reads the token and 404s without it,
# and /bootstrap/package hands out the installable agent package and spends
# a use doing it.
('_presented_token', 'api-key (single-use bootstrap token in the '
'X-Endpoint-Bootstrap-Token header, or ?t= while that '
'is enabled; per-IP rate limited, 404 when not usable)'),
# The code IS the credential: unguessable, spent on any attempt, and gone in
# minutes. No header and no session, because the caller is a browser being
# pointed at a URL by an administrator.
('_redeem_kiosk_setup_code', 'setup-code (single-use kiosk setup code in '
'?code=, issued by the permission-gated '
'provisioning link; spent on any attempt, 410 '
'once used or expired)'),
('_bootstrap_allowed', 'none (first run only: refused once an admin exists)'),
('verify_jwt_in_request', 'jwt'),
)
# Routes whose gate is inline AND conditional, where the call sweep above would
# name the wrong thing. One line each, and the line is the point: these are the
# handlers where "what authenticates this?" has an answer no sweep can derive.
INLINE_AUTH = {
# The key may be absent, or invalid, and the call still proceeds: a PC
# installed without the shared key must still be able to reach an admin. A
# valid key only decides whether auto-enroll may issue on a clean match.
'endpoints.enroll_endpoint': 'none (per-IP rate limited and capped; a valid '
'fleet key only decides auto-enroll)',
# The unguessable token in the path IS the authorisation - the recipient of
# a secure send has no account here.
'tools.peek_secret': 'none (the unguessable token in the path is the '
'authorisation)',
'tools.redeem_secret': 'none (the unguessable token in the path is the '
'authorisation; redeeming spends the secret)',
# This IS the auth surface, so a token cannot gate it. The bounds it carries
# instead are a per-IP rate limit and the 5-strike account lockout.
'auth.login': 'none (the login surface itself; rate limited, 5-strike lockout)',
}
# Verbs that carry a body, for the seeded params line.
BODY_VERBS = ('POST', 'PUT', 'PATCH')
def _literal_set(filename, name):
"""One module-level set literal, read without importing the module.
Imported, tests/test_core/test_authz.py builds a second app at import time,
which rebinds the SQLAlchemy extension out from under the one this script
is walking.
"""
import ast
try:
with open(filename, 'r', encoding='utf-8') as handle:
tree = ast.parse(handle.read())
except (OSError, SyntaxError):
return set()
for node in tree.body:
if not isinstance(node, ast.Assign):
continue
for target in node.targets:
if isinstance(target, ast.Name) and target.id == name:
try:
return set(ast.literal_eval(node.value))
except ValueError:
return set()
return set()
def _reviewed_exceptions():
"""(endpoints, blueprints) already reviewed as needing no permission.
Read from the two sweeps in tests/ rather than restated here, so the
published contract cannot disagree with the gate that enforces it. A route
on one of those lists was argued about in front of somebody, in a file whose
whole job is to make that argument visible; a route on neither, with no
gate, is marked for review instead. That is the difference between a
decision and an oversight, which is what those lists exist to keep.
"""
tests = os.path.join(REPO, 'tests', 'test_core')
reads = os.path.join(tests, 'test_public_read_surface.py')
writes = os.path.join(tests, 'test_authz.py')
endpoints = (_literal_set(reads, 'PUBLIC_ENDPOINTS')
| _literal_set(writes, 'EXEMPT_ENDPOINTS'))
return endpoints, _literal_set(writes, 'EXEMPT_BLUEPRINTS')
def _source_index(filename, _cache={}):
"""{function name: (decorator sources, call sources)} for one file.
Read from the source, not from the live object: functools.wraps copies the
name and the docstring onto the wrapper and nothing else, so by the time a
view is in url_map its decorators are invisible. _requires_permission
survives only because require_permission deliberately stamps it on.
"""
import ast
if filename in _cache:
return _cache[filename]
index = {}
try:
with open(filename, 'r', encoding='utf-8') as handle:
tree = ast.parse(handle.read())
except (OSError, SyntaxError):
_cache[filename] = index
return index
for node in ast.walk(tree):
if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
index[node.name] = (
[ast.unparse(item) for item in node.decorator_list],
[ast.unparse(item) for item in ast.walk(node)
if isinstance(item, ast.Call)],
)
_cache[filename] = index
return index
def _view_source(view):
"""(decorators, calls) for the undecorated view behind a url_map entry."""
import inspect
original = inspect.unwrap(view)
filename = inspect.getsourcefile(original) or ''
return _source_index(filename).get(original.__name__, ([], []))
def _auth_for(view, endpoint, reviewed, blueprints):
"""The honest auth string for one route, derived in gate order.
Nothing here guesses. Every branch names a mechanism that is in the code - a
marker the decorator stamped on, a decorator in the source, a call the
handler makes, or a name on a reviewed exemption list - and a route that
matches none of them says exactly that.
"""
marker = getattr(view, '_requires_permission', None)
if marker:
roles = [name.split(':', 1)[1] for name in marker
if name.startswith('role:')]
if roles:
return 'jwt + role:' + ' or '.join(sorted(roles))
return 'permission:' + ' or '.join(marker)
decorators, calls = _view_source(view)
for text in decorators:
name = text.split('(', 1)[0].strip().split('.')[-1]
if name in DECORATOR_AUTH:
return DECORATOR_AUTH[name]
if name == 'jwt_required':
if 'refresh=True' in text:
return 'jwt (refresh token)'
return JWT_OPTIONAL if 'optional=True' in text else 'jwt'
if endpoint in INLINE_AUTH:
return INLINE_AUTH[endpoint]
for call in calls:
tail = call.split('(', 1)[0].split('.')[-1]
for needle, auth in HANDLER_AUTH:
if tail != needle:
continue
# Optional verification is a read that works signed out, not a gate.
# Only the required form authenticates anybody.
if needle == 'verify_jwt_in_request' and 'optional' in call:
return JWT_OPTIONAL
return auth
blueprint, _, shortname = endpoint.partition('.')
if endpoint in reviewed or shortname in reviewed or blueprint in blueprints:
return 'none (reviewed: named in the public-read or mutation sweep)'
return NEEDS_REVIEW
def _surface_for(view, endpoint):
"""The tag this route is grouped under: one surface per plugin, and one per
core blueprint.
Derived, not preserved: a route that moves into a plugin moves tag with it,
which is the whole reason the reader is looking at the tag.
"""
import inspect
module = getattr(inspect.unwrap(view), '__module__', '') or ''
if module.startswith('plugins.'):
return 'plugin-' + module.split('.')[1]
return 'core-' + endpoint.partition('.')[0]
def _seed_purpose(view, endpoint):
"""First paragraph of the view's docstring, as a starting point.
A docstring says what the handler does; the inventory wants what the
endpoint is FOR, which is usually more. This seeds the entry so a new route
is never published blank, and the prose is preserved from then on.
"""
import inspect
doc = inspect.getdoc(inspect.unwrap(view)) or ''
first = ' '.join(doc.split('\n\n')[0].split())
return first or 'UNDOCUMENTED: %s has no docstring.' % endpoint
def _path_arguments(rule):
"""[(name, converter)] for one rule, in the order they appear."""
return [(name, converter or 'string') for converter, name
in re.findall(r'<(?:([^:>]+):)?([^>]+)>', rule)]
def _seed_params(rule, methods):
parts = []
arguments = _path_arguments(rule)
if arguments:
parts.append('path: ' + ', '.join(
'%s (%s)' % (name, converter) for name, converter in arguments))
if any(method in BODY_VERBS for method in methods):
parts.append('body: JSON (fields not yet documented)')
# Path arguments are in the rule; query arguments are read out of the
# handler and cannot be derived, so an entry that has neither says it does
# not know rather than claiming the route takes nothing.
return '; '.join(parts) if parts else 'not documented'
def _seed_example(rule, methods, auth):
"""A curl that is the right SHAPE for this route: its verb, a filled path
and the header its auth actually wants.
Placeholder ids, because the generator has no database - but a reader who
substitutes one gets a call that works, which is more than a blank field
gives them.
"""
def fill(match):
converter = (match.group(1) or '').split('(')[0]
if converter.startswith(('int', 'float')):
return '1'
return 'example.png' if converter == 'path' else 'x'
url = 'http://localhost:5001' + re.sub(r'<(?:([^:>]+):)?[^>]+>', fill, rule)
method = sorted(methods)[0] if methods else 'GET'
command = 'curl' if method == 'GET' else 'curl -X ' + method
header = ''
if auth.startswith(('permission:', 'jwt')):
header = ' -H "Authorization: Bearer $TOK"'
elif auth.startswith('api-key'):
# The header name varies per surface (X-API-Key, X-Kiosk-Token,
# X-Endpoint-Bootstrap-Token), and the auth string names the one this
# route wants. A pasted example with the wrong header is worse than
# none: it looks like it should work.
named = re.search(r'\bX-[A-Za-z0-9-]+\b', auth)
header = ' -H "%s: $KEY"' % (named.group(0) if named else 'X-API-Key')
elif auth.startswith('setup-code'):
url += '?code=$CODE'
body = ''
if any(verb in methods for verb in BODY_VERBS):
body = " -H 'Content-Type: application/json' -d '{}'"
return '%s %s%s%s' % (command, url, header, body)
def _served_routes(app):
"""Every /api rule, as (path, sorted methods, endpoint, view).
HEAD and OPTIONS are Werkzeug's, not the product's. A rule with several
verbs stays one entry, written 'PUT|PATCH', which is how the inventory has
always spelled it and what the parity test splits on.
"""
routes = []
for rule in app.url_map.iter_rules():
if rule.endpoint == 'static' or not rule.rule.startswith(API_PREFIX):
continue
methods = sorted((rule.methods or set()) - {'HEAD', 'OPTIONS'})
if not methods:
continue
routes.append((rule.rule, methods, rule.endpoint,
app.view_functions[rule.endpoint]))
return sorted(routes, key=lambda row: (row[0], row[1]))
def _existing_prose():
"""{(path, METHOD): entry} for whatever the inventory already says.
Keyed per method rather than per entry so prose survives a route that gains
or loses a verb: 'PUT' still finds the words written when the entry said
'PUT|PATCH'.
"""
prose = {}
if not os.path.isfile(INVENTORY):
return prose
with open(INVENTORY, 'r', encoding='utf-8') as handle:
surfaces = json.load(handle)
for surface in surfaces:
for entry in surface.get('endpoints', []):
path = entry.get('path') or ''
for method in re.split(r'[/,|]', entry.get('method') or 'GET'):
method = method.strip().upper()
if method:
prose.setdefault((path, method), entry)
return prose
def _inventory_app():
"""The app, with every bundled plugin enabled.
The inventory describes the PRODUCT, not one site's switched-on subset: a
plugin disabled on the machine that happens to run this would drop its whole
surface out of the published contract without a word, and the next refresh
somewhere else would put it back. So a disabled plugin is refused here
rather than quietly omitted.
"""
import sys
# .env BEFORE create_app, not after: config is read while the app is built,
# so a load underneath it changes nothing. A plain `python scripts/...` run
# gets neither wsgi.py's load nor the flask CLI's, and falls back to the
# hardcoded root@localhost URL - which surfaces as a bare 1045 access-denied
# that says nothing about the real cause.
from dotenv import load_dotenv
load_dotenv(os.path.join(REPO, '.env'))
os.environ.setdefault('FLASK_ENV', 'testing')
if REPO not in sys.path:
sys.path.insert(0, REPO)
from shopdb import create_app
app = create_app('testing')
plugins = os.path.join(REPO, 'plugins')
bundled = sorted(
name for name in os.listdir(plugins)
if os.path.isfile(os.path.join(plugins, name, 'manifest.json')))
manager = app.extensions.get('plugin_manager')
off = [name for name in bundled
if not (manager and manager.registry.is_enabled(name))]
if off:
raise SystemExit(
'these bundled plugins are disabled, so their routes would be '
'missing\nfrom the inventory: %s\nenable them and re-run:\n'
' FLASK_APP=shopdb venv/bin/flask plugins enable %s'
% (', '.join(off), off[0]))
return app
def refresh_inventory():
"""Rewrite the inventory's derived half from the routes the app serves.
Structure (surface, method, path, auth) is replaced outright. Prose
(purpose, params, example) is carried over untouched wherever an entry for
that route already exists, and seeded from the view's docstring where one
does not.
"""
app = _inventory_app()
reviewed, blueprints = _reviewed_exceptions()
prose = _existing_prose()
surfaces = {}
seeded = []
unclassified = []
for path, methods, endpoint, view in _served_routes(app):
auth = _auth_for(view, endpoint, reviewed, blueprints)
if auth == NEEDS_REVIEW:
unclassified.append('%s %s (%s)' % ('|'.join(methods), path, endpoint))
known = prose.get((path, methods[0]))
if known is None:
seeded.append('%s %s' % ('|'.join(methods), path))
entry = dict(known or {})
entry.update({
'method': '|'.join(methods),
'path': path,
'auth': auth,
'params': entry.get('params') or _seed_params(path, methods),
'purpose': entry.get('purpose') or _seed_purpose(view, endpoint),
'example': entry.get('example') or _seed_example(path, methods, auth),
})
# Fixed key order, so a refresh that changes nothing changes no bytes.
ordered = {key: entry[key] for key in
('method', 'path', 'auth', 'params', 'purpose', 'example')}
ordered.update({key: value for key, value in entry.items()
if key not in ordered})
surfaces.setdefault(_surface_for(view, endpoint), []).append(ordered)
document = [{'surface': name, 'endpoints': entries}
for name, entries in sorted(surfaces.items())]
with open(INVENTORY, 'w', encoding='utf-8') as handle:
json.dump(document, handle, indent=2)
handle.write('\n')
total = sum(len(surface['endpoints']) for surface in document)
print('refreshed %s: %d entries in %d surfaces (%d new)'
% (INVENTORY, total, len(document), len(seeded)))
for line in seeded:
print(' seeded %s' % line)
if unclassified:
print('%d route(s) have no gate this can see. Classify each one by hand '
'in the inventory, or gate it:' % len(unclassified))
for line in unclassified:
print(' review %s' % line)
def main():
surfaces = json.load(open(INVENTORY))
if '--refresh' in sys.argv[1:]:
refresh_inventory()
with open(INVENTORY, 'r', encoding='utf-8') as handle:
surfaces = json.load(handle)
spec = build(surfaces)
json.dump(spec, open(OUT, 'w'), indent=1)
with open(OUT, 'w', encoding='utf-8') as handle:
json.dump(spec, handle, indent=1)
ops = sum(len(v) for v in spec['paths'].values())
print('wrote %s: %d paths, %d operations from %d surfaces'
% (OUT, len(spec['paths']), ops, len(surfaces)))

12
shopdb/__init__.py
View File

@@ -139,7 +139,17 @@ from .plugins import plugin_manager
# bump can catch, and it is the kind of thing that gets copied. Exposing the
# model closes the gap instead of leaving the workaround as the worked example.
# Additive name on the surface, minor bump.
__contract_version__ = '0.34.0'
# 0.35.0 adds ip_in_cidrs and request_ip_allowlisted to shopdb.api: network
# position as an admission check for routes that run without a login. The two
# kiosks needed the identical gate - a device token suits an unattended screen
# provisioned once, and suits nothing about somebody opening the same page on
# their own PC - and a CIDR parser copied into two plugins is exactly where a
# fail-open creeps in later. Both fail closed on an empty setting, an
# unparseable caller address and a malformed entry. Caller trustworthiness is a
# deployment property: they read remote_addr, never raw X-Forwarded-For, so the
# IIS rewrite rule is what makes them meaningful. Additive names on the surface,
# minor bump.
__contract_version__ = '0.35.0'
# Product release version (see ADR-007). The product version and the
# plugin-contract version above are distinct series with independent

7
shopdb/api/__init__.py
View File

@@ -187,6 +187,11 @@ from shopdb.utils.uploads import send_upload, harden_upload_response
# nothing. See shopdb/utils/clientip.py.
from shopdb.utils.clientip import client_ip, strip_port
# Network-position allowlists for the unauthenticated kiosk routes. On the
# contract surface because two plugins need the identical gate and a
# copy-pasted CIDR parser is exactly where a fail-open creeps in.
from shopdb.utils.ipallowlist import ip_in_cidrs, request_ip_allowlisted
# Strict date parsing. On the contract surface because every plugin that stores
# a date needs it and the two that rolled their own both swallowed the error.
from shopdb.utils.queryargs import QueryArgError, intarg, intargs
@@ -477,6 +482,8 @@ __all__ = [
'harden_upload_response',
'client_ip',
'strip_port',
'ip_in_cidrs',
'request_ip_allowlisted',
# Import-mode helpers
'apply_import_timestamps',
'import_mode_active',

98
shopdb/core/api/collector.py
View File

@@ -27,6 +27,44 @@ COLLECTOR_SCOPE = 'collector.ingest'
# declaring acceptsagentcredential.
AGENT_CREDENTIAL_HEADER = 'X-Endpoint-Credential'
# How a collector request authorized, recorded on g so a route can tell a
# managed service token from the shared env key after the check has passed.
AUTH_MANAGED_TOKEN = 'token'
AUTH_SHARED_KEY = 'sharedkey'
# The four legacy routes below (/pc, /apps, /heartbeat, /bulk) predate both the
# generic collector contract and per-PC credentials. They take a hostname from
# the body and rewrite that PC's logged-in user, boot time, serial number and
# installed-app inventory, so the shared key is an any-host write key on them.
# plugin.endpoints.sharedkeyallowed does not reach here - it gates the endpoints
# collector, not core - so turning it off leaves these open. This setting is the
# core counterpart. Default false = today's behaviour, unchanged.
LEGACY_SHARED_KEY_SETTING = 'collector.legacysharedkeyrefused'
# Declared here so the wording stays with the code that enforces it.
# build_default_settings (shopdb/core/api/settings.py) has to include this dict
# for the row to be seeded; until it does, the key is created on demand through
# the settings API and the absent-row default below is what applies.
LEGACY_SHARED_KEY_SETTING_DEFAULT = {
'key': LEGACY_SHARED_KEY_SETTING,
'value': 'false',
'valuetype': 'boolean',
'category': 'collector',
'description': 'Refuse the shared collector key on the legacy core routes '
'POST /api/collector/pc, /apps, /heartbeat and /bulk. Those '
'routes name their target PC in the body, so any holder of '
'the shared key can rewrite logged-in user, last boot time, '
'serial number and installed apps for ANY hostname. Turning '
'this on closes them to the shared key; a collector-scoped '
'managed token still works. Turn it on only after checking '
'the "Legacy collector route used with the shared key" '
'warnings in the log for a while: whatever still calls them '
'stops working the moment it is on. The two GE-Enforce '
'reporters post to /api/collector/computers and '
'/api/collector/printers and are NOT affected. This should '
'follow the move to per-PC credentials, not precede it.',
}
def _computer_models():
"""Lazily import the computers plugin models.
@@ -131,13 +169,17 @@ def _check_collector_auth(expected_key, pluginname=None):
api_key = request.headers.get('X-API-Key')
g.collectorauth = None
token = _collector_managed_token(api_key)
if token is not None and _token_is_collector(token, pluginname):
touch_apitoken_lastused(token)
g.collectorauth = AUTH_MANAGED_TOKEN
return None
if expected_key and api_key == expected_key \
and not _env_fallback_disabled(pluginname):
g.collectorauth = AUTH_SHARED_KEY
return None
# Fail-closed only on pure server misconfiguration: no env key AND the
@@ -164,6 +206,54 @@ def require_api_key(f):
return decorated
def _legacy_shared_key_refused():
"""True when the site has closed the legacy routes to the shared key.
Absent setting = false = the routes stay open, which is what every site
does today. Read tolerantly because the row can be created by hand through
the settings API, where it may land with valuetype string.
"""
from shopdb.core.models.setting import Setting
value = Setting.get(LEGACY_SHARED_KEY_SETTING, default=False)
if isinstance(value, str):
return value.strip().lower() in ('true', '1', 'yes')
return bool(value)
def require_api_key_legacy(f):
"""require_api_key plus the legacy-route shared-key gate.
These routes take their target hostname from the body, so the shared key is
an any-host write key on them and no per-PC credential narrows it. A site
that has finished the move to per-PC credentials can shut it off here
without waiting for the routes to be retired.
Every shared-key use is logged whether the gate is on or off, so a site can
read the warnings and learn what still depends on the key BEFORE flipping
it - otherwise turning it on is a guess.
"""
from shopdb.utils.clientip import client_ip
@wraps(f)
def decorated(*args, **kwargs):
error = _check_collector_auth(current_app.config.get('COLLECTOR_API_KEY'))
if error is not None:
return error
if getattr(g, 'collectorauth', None) == AUTH_SHARED_KEY:
if _legacy_shared_key_refused():
return error_response(
'SHARED_KEY_REFUSED',
'The shared key is not accepted on the legacy collector '
'routes; use a collector-scoped token',
http_code=401)
current_app.logger.warning(
'Legacy collector route used with the shared key: %s from %s',
request.path, client_ip(request))
return f(*args, **kwargs)
return decorated
def _find_pc(hostname):
"""Find a computer by hostname, falling back to its asset number.
@@ -347,7 +437,7 @@ def collector_schemas():
@collector_bp.route('/pc', methods=['POST'])
@require_api_key
@require_api_key_legacy
def update_pc_info():
"""Update one PC from the collection script (matched by hostname)."""
data = request.get_json()
@@ -385,7 +475,7 @@ def update_pc_info():
@collector_bp.route('/apps', methods=['POST'])
@require_api_key
@require_api_key_legacy
def update_installed_apps():
"""Update installed applications for a PC (matched by hostname)."""
data = request.get_json()
@@ -457,7 +547,7 @@ def update_installed_apps():
@collector_bp.route('/heartbeat', methods=['POST'])
@require_api_key
@require_api_key_legacy
def pc_heartbeat():
"""Record PC online status / heartbeat (single or batch)."""
data = request.get_json()
@@ -492,7 +582,7 @@ def pc_heartbeat():
@collector_bp.route('/bulk', methods=['POST'])
@require_api_key
@require_api_key_legacy
def bulk_update():
"""Bulk update multiple PCs at once."""
data = request.get_json()

69
shopdb/core/api/settings.py
View File

@@ -161,14 +161,50 @@ def _declared_default(key: str) -> dict:
return _plugin_declared_settings().get(key, {})
def _is_secret(key: str) -> bool:
return 'password' in key or 'token' in key or 'secret' in key
def _is_secret(key: str, valuetype=None) -> bool:
"""True when a key's value must never leave the API in plaintext.
The test is the key NAME, which is a heuristic, so it is narrowed by the
stored type when the caller knows it. A boolean, integer or float setting
cannot hold a secret, and masking one breaks the control that renders it:
plugin.endpoints.bootstrapquerytoken is a true/false switch whose name
contains 'token', and the settings page read back '********' instead of
its state, so the checkbox could not show whether it was on.
valuetype omitted means "judge by name alone", which is the safe direction:
an unknown type is treated as a string.
"""
named = 'password' in key or 'token' in key or 'secret' in key
if not named:
return False
return valuetype is None or str(valuetype).lower() == 'string'
def _inferred_valuetype(value) -> str:
"""Valuetype for a key nobody declares, from the JSON type sent.
Only reached on the upsert path in update_setting. JSON already carries the
distinction the row has to keep, so honour it: true/false is a boolean row,
a number is integer or float, everything else is a string. Guessing from
the TEXT (e.g. the string 'true') is deliberately not done - a genuine
string setting whose value happens to read 'true' would then be typed
boolean and coerced out from under its reader.
bool is checked before int because bool subclasses int in Python.
"""
if isinstance(value, bool):
return 'boolean'
if isinstance(value, int):
return 'integer'
if isinstance(value, float):
return 'float'
return 'string'
def _serialize_setting(setting):
"""Serialize a setting, masking secret values so they never leave the API."""
data = setting.to_dict()
if _is_secret(setting.key):
if _is_secret(setting.key, setting.valuetype):
data['value'] = SECRET_MASK if setting.value else ''
return data
@@ -795,7 +831,14 @@ def update_setting(key: str):
setting = Setting(
key=key,
value='',
valuetype=declared.get('valuetype', 'string'),
# An undeclared key used to be born valuetype='string'
# unconditionally, and that was a trap: a caller PUTting JSON true
# got a string row, get_typed_value then handed every reader the
# text 'true', and a reader doing bool(...) or `is True` silently
# disagreed with the UI. Infer from the JSON type the client sent
# instead, so a boolean key is born boolean.
valuetype=declared.get('valuetype') or _inferred_valuetype(
data.get('value')),
category=declared.get('category', 'plugin'),
description=declared.get('description'),
)
@@ -808,7 +851,7 @@ def update_setting(key: str):
# A secret submitted as the mask placeholder means "leave unchanged" - the
# client only ever received the mask, so don't overwrite the real secret.
if _is_secret(key) and value == SECRET_MASK:
if _is_secret(key, setting.valuetype) and value == SECRET_MASK:
return success_response(_serialize_setting(setting), message='Setting unchanged')
# The declared valuetype is a promise about what every reader will get, and
@@ -846,7 +889,7 @@ def update_setting(key: str):
setting.value = str(value) if value is not None else None
# Audit log (mask sensitive values)
is_sensitive = _is_secret(key)
is_sensitive = _is_secret(key, setting.valuetype)
AuditLog.log('updated', 'Setting', entityname=key, changes={
'value': {
'old': '***' if is_sensitive else old_value,
@@ -915,10 +958,13 @@ def create_setting():
else:
value_str = str(value) if value is not None else None
# Same reasoning as the upsert in update_setting: a caller that omits
# valuetype and sends JSON true must not get a string row that every
# boolean reader then misreads.
setting = Setting(
key=data['key'],
value=value_str,
valuetype=data.get('valuetype', 'string'),
valuetype=data.get('valuetype') or _inferred_valuetype(value),
category=data.get('category', 'general'),
description=data.get('description')
)
@@ -1268,7 +1314,16 @@ def build_default_settings():
# Collector pc-type -> ComputerType mapping is computers-plugin domain;
# the plugin seeds pctypemap_<pxetype> settings on install.
# Collector authentication. The wording lives with the code that enforces
# it (shopdb/core/api/collector.py); seeding it here is what makes the row
# exist, and the row is what the settings UI can render a control for.
# Imported inside the function because core.api.__init__ imports collector
# and settings in the same package - a module-level import would couple
# their import order for no gain.
from shopdb.core.api.collector import LEGACY_SHARED_KEY_SETTING_DEFAULT
defaults = sitedefaults + brandingdefaults + printingdefaults + identifierdefaults + searchdefaults + mapdefaults + [
LEGACY_SHARED_KEY_SETTING_DEFAULT,
# ServiceNow ticket links. Each instance points these at its own
# ServiceNow tenant; blank/disabled renders tickets as plain text.
#

64
shopdb/utils/ipallowlist.py Normal file
View File

@@ -0,0 +1,64 @@
"""Network-position allowlists for unauthenticated kiosk routes.
A kiosk route writes without a login. The device token (see the kiosk views)
covers unattended screens an administrator provisions once, but it does not fit
the other caller: somebody opening the kiosk page on their own PC, who never
logs in and whose browser nobody provisioned. Per-browser provisioning would
mean touching every PC on the floor and losing it again on the next reimage.
What those callers have in common is where they are on the network.
Fails closed, in every direction that matters:
- an empty or absent setting matches nothing (the feature is off),
- an unparseable caller address matches nothing,
- a malformed entry in the list is skipped rather than widening the match.
SPOOF RESISTANCE IS A DEPLOYMENT PROPERTY, NOT A PROPERTY OF THIS CODE.
The caller address comes from client_ip(), which reads remote_addr and never
the raw X-Forwarded-For header - reading that header directly would let any
caller send 'X-Forwarded-For: <allowlisted-ip>' and walk straight through.
remote_addr is only the real client because of what sits in front of the app:
behind IIS the URL-Rewrite rule overwrites X-Forwarded-For with the real TCP
peer and ProxyFix resolves it into remote_addr (TRUSTED_PROXY_COUNT). Remove
that rule and IIS forwards whatever the caller sent; waitress trusts it because
it arrives from 127.0.0.1, remote_addr becomes attacker-controlled, and every
allowlist built on this becomes bypassable from anywhere on the network. It
does NOT fail closed in that case. The rewrite rule is the control. The
GE-Enforce client allowlist carries the same dependency.
Callers store plain CIDR strings rather than references to subnet records: a
site that does not run the network plugin still has to be able to configure
this, and a stored allowlist must not break when an inventory row is deleted.
"""
import ipaddress
from shopdb.utils.clientip import client_ip
def ip_in_cidrs(address, rawcidrs):
"""True when `address` falls inside any CIDR in the comma-separated list.
Pure and side-effect free so it can be tested without a request context.
"""
raw = (rawcidrs or '').strip()
if not raw:
return False
try:
caller = ipaddress.ip_address((address or '').strip())
except ValueError:
return False
for part in raw.split(','):
part = part.strip()
if not part:
continue
try:
if caller in ipaddress.ip_network(part, strict=False):
return True
except ValueError:
continue
return False
def request_ip_allowlisted(request_obj, rawcidrs):
"""True when the request's client address is inside the allowlist."""
return ip_in_cidrs(client_ip(request_obj), rawcidrs)

112
tests/test_api_inventory_parity.py
View File

@@ -1,20 +1,26 @@
"""Every route the app serves is described in docs/api-inventory.json.
The inventory is hand-maintained, and deliberately so: its value is the prose -
what each endpoint is FOR, what it accepts, a worked example - and none of that
can be derived from the code. An audit of all 372 documented operations found
zero phantom routes and zero wrong parameter names, so the maintenance itself is
in good order.
The inventory has two halves. Its STRUCTURE - surface, method, path and the auth
each route actually enforces - is derived from the live url_map and the
decorators on each view by `gen_openapi.py --refresh`. Its PROSE - what an
endpoint is FOR, what it accepts, a worked example - is human, and a refresh
never overwrites words that are already there.
What it cannot do is notice a route somebody added. Twenty-two were serving
traffic undocumented, including an entire plugin's surface and every one of the
dashboard card endpoints - each of them added after its surface was written.
This module is the coverage half of that, and it is the half a generator cannot
do for you: a refresh describes whatever is served at the moment it runs, so
without a gate the file rots the moment the next route lands.
So the prose stays hand-written and the COVERAGE is enforced here. Adding an
endpoint now means describing it, in the same commit, which is when the author
still knows what it is for.
It had. On this branch the inventory was not tracked at all, so the skipif below
fired, this module reported green while checking nothing, and 24 routes went to
production documented nowhere - including the whole endpoint-agent delivery
surface and both kiosks. scripts/check-gates-ran.sh now asserts this module RAN,
in every checkout rather than only in one with wiki prose in it.
So: add a route, run `gen_openapi.py --refresh`, and replace the seeded purpose
with the reason the route exists - which is while the author still knows it.
"""
import importlib.util
import json
import re
from pathlib import Path
@@ -93,3 +99,87 @@ def test_nothing_documented_has_been_removed():
assert not stale, (
'%d documented route(s) are no longer served:\n %s'
% (len(stale), '\n '.join('%s %s' % row for row in stale)))
# ---------------------------------------------------------------------------
# The other half: the auth string, which nothing gated until now
# ---------------------------------------------------------------------------
#
# The two tests above check that a route EXISTS in both directions. Nothing
# checked what the entry SAYS about it, so a wrong or stale auth string passed
# every gate in the repo - and auth is the field a site reads to decide what its
# guest role and its firewall may reach, and the field the published spec turns
# into a machine-readable security requirement.
#
# It had rotted exactly that way: nine operations documented as needing no
# credential while the handler required one, and a kiosk route documented as
# ungated when its gate is a single-use setup code. Both survived a refresh,
# because a refresh only rewrites; nothing compared.
#
# So this re-derives auth from the live app with the generator's own classifier
# and fails on any disagreement with the committed file. The derivation is
# authoritative: hand-editing an auth string in the inventory does not survive
# the next --refresh, so the fix for a wrong one is always to teach the sweep in
# scripts/gen_openapi.py, or to gate the view.
def _generator():
"""scripts/gen_openapi.py, loaded by path.
It is a script, not a package module, so there is nothing to import. Loading
it runs its imports and nothing else - it builds no app until it is asked
to.
"""
path = REPO / 'scripts' / 'gen_openapi.py'
spec = importlib.util.spec_from_file_location('gen_openapi', path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def derived_auth():
"""{(METHOD, path): auth} as the classifier reads the code right now."""
generator = _generator()
app = create_app('testing')
reviewed, blueprints = generator._reviewed_exceptions()
derived = {}
for rule, methods, endpoint, view in generator._served_routes(app):
auth = generator._auth_for(view, endpoint, reviewed, blueprints)
for method in methods:
derived[(method, normalise(rule))] = auth
return derived
def documented_auth():
"""{(METHOD, path): auth} as docs/api-inventory.json has it committed."""
documented = {}
for surface in json.loads(INVENTORY.read_text()):
for entry in surface.get('endpoints', []):
path = normalise(entry.get('path') or '')
for method in re.split(r'[/,|]', entry.get('method') or 'GET'):
method = method.strip().upper()
if method:
documented[(method, path)] = entry.get('auth') or ''
return documented
def test_documented_auth_is_what_the_code_enforces():
"""A stale auth string is a lie about who may call a route.
Only routes present on both sides are compared. A route missing from either
is a coverage fault, and the two tests above already name it; reporting it
twice here would bury this one.
"""
derived = derived_auth()
documented = documented_auth()
disagreements = []
for key in sorted(set(derived) & set(documented)):
if derived[key] != documented[key]:
disagreements.append(
'%s %s\n documented: %s\n enforced: %s'
% (key[0], key[1], documented[key] or '(blank)', derived[key]))
assert not disagreements, (
'%d route(s) document an auth that is not what the code enforces. The '
'inventory is derived, so fix the code or teach the classifier in '
'scripts/gen_openapi.py, then re-run it with --refresh:\n %s'
% (len(disagreements), '\n '.join(disagreements[:15])))

143
tests/test_core/test_collector_legacy_sharedkey.py Normal file
View File

@@ -0,0 +1,143 @@
"""The legacy core collector routes and the shared key.
/pc, /apps, /heartbeat and /bulk name their target PC in the request body, so
the shared collector key is an any-host write key on them and no per-PC
credential narrows it. collector.legacysharedkeyrefused closes them to the
shared key. It defaults OFF so nothing on an un-migrated fleet changes, and the
shared key is logged either way so a site can see what depends on it first.
"""
import pytest
from shopdb.core.api.collector import (
LEGACY_SHARED_KEY_SETTING,
LEGACY_SHARED_KEY_SETTING_DEFAULT,
)
KEY = 'testlegacykey'
LEGACY_POSTS = [
('/api/collector/pc', {'hostname': 'WJLEG01'}),
('/api/collector/apps', {'hostname': 'WJLEG01',
'apps': [{'appname': 'Nothing'}]}),
('/api/collector/heartbeat', {'hostname': 'WJLEG01'}),
('/api/collector/bulk', {'pcs': [{'hostname': 'WJLEG01'}]}),
]
@pytest.fixture
def collector_key(app):
old = app.config.get('COLLECTOR_API_KEY')
app.config['COLLECTOR_API_KEY'] = KEY
yield KEY
app.config['COLLECTOR_API_KEY'] = old
@pytest.fixture
def legacy_closed(db):
"""Turn the closure on for the duration of a test."""
from shopdb.core.models.setting import Setting
Setting.set(LEGACY_SHARED_KEY_SETTING, True, valuetype='boolean',
category='collector')
yield
Setting.query.filter_by(key=LEGACY_SHARED_KEY_SETTING).delete()
db.session.commit()
def _mint_collector_token(client, auth_headers):
resp = client.post('/api/apitokens',
json={'name': 'collector svc',
'scopes': ['collector.ingest']},
headers=auth_headers)
assert resp.status_code == 201, resp.get_json()
return resp.get_json()['data']['secret']
@pytest.mark.parametrize('path,payload', LEGACY_POSTS)
def test_shared_key_still_works_by_default(client, db, collector_key,
path, payload):
"""No setting row: today's behaviour. The routes answer the shared key.
404/400 is fine - the point is that authorization passed, not that the
hostname exists.
"""
resp = client.post(path, json=payload, headers={'X-API-Key': KEY})
assert resp.status_code != 401, resp.get_json()
@pytest.mark.parametrize('path,payload', LEGACY_POSTS)
def test_shared_key_refused_when_closed(client, db, collector_key,
legacy_closed, path, payload):
"""Setting on: the shared key is refused the way the endpoints plugin
refuses it, before any write happens."""
resp = client.post(path, json=payload, headers={'X-API-Key': KEY})
assert resp.status_code == 401
assert resp.get_json()['data']['error']['code'] == 'SHARED_KEY_REFUSED'
def test_managed_token_unaffected_by_closure(client, db, auth_headers,
legacy_closed):
"""The closure targets the shared key only. A collector-scoped managed
token keeps working on a legacy route."""
secret = _mint_collector_token(client, auth_headers)
resp = client.post('/api/collector/heartbeat',
json={'hostname': 'WJLEG01'},
headers={'X-API-Key': secret})
assert resp.status_code == 200, resp.get_json()
def test_status_stays_open_when_closed(client, db, collector_key,
legacy_closed):
"""/status writes nothing, so the closure does not touch it - it stays a
usable connectivity check for a fleet still on the shared key."""
resp = client.get('/api/collector/status', headers={'X-API-Key': KEY})
assert resp.status_code == 200
def test_generic_collector_unaffected_by_closure(client, db, auth_headers,
collector_key, legacy_closed):
"""The generic per-plugin route has its own shared-key rules (per-PC
credentials). This setting must not reach it."""
from shopdb.core.models import AssetType
db.session.add(AssetType(assettype='computer', pluginname='computers',
tablename='computers', description='PCs'))
db.session.commit()
resp = client.post('/api/collector/computers',
json={'hostname': 'WJLEG02'},
headers={'X-API-Key': KEY})
assert resp.status_code == 200, resp.get_json()
def test_shared_key_use_is_logged(client, db, collector_key, caplog):
"""The warning is what a site reads before flipping the setting, so it has
to be emitted while the routes are still open."""
with caplog.at_level('WARNING'):
client.post('/api/collector/heartbeat', json={'hostname': 'WJLEG01'},
headers={'X-API-Key': KEY})
assert any('Legacy collector route used with the shared key' in r.message
for r in caplog.records)
def test_managed_token_use_is_not_logged(client, db, auth_headers, caplog):
"""A managed token is the supported credential; logging it would bury the
shared-key warnings the site is looking for."""
secret = _mint_collector_token(client, auth_headers)
with caplog.at_level('WARNING'):
client.post('/api/collector/heartbeat', json={'hostname': 'WJLEG01'},
headers={'X-API-Key': secret})
assert not any('used with the shared key' in r.message
for r in caplog.records)
def test_setting_default_is_todays_behaviour():
"""The declared default must leave the routes open: a site mid-migration
has un-enrolled PCs and the shared key is all they have."""
assert LEGACY_SHARED_KEY_SETTING_DEFAULT['key'] == LEGACY_SHARED_KEY_SETTING
assert LEGACY_SHARED_KEY_SETTING_DEFAULT['value'] == 'false'
assert LEGACY_SHARED_KEY_SETTING_DEFAULT['valuetype'] == 'boolean'
description = LEGACY_SHARED_KEY_SETTING_DEFAULT['description']
for route in ('/pc', '/apps', '/heartbeat', '/bulk'):
assert route in description

7
tests/test_core/test_public_read_surface.py
View File

@@ -86,6 +86,13 @@ PUBLIC_ENDPOINTS = {
# USB kiosk.
'usb.kiosk_device',
# The screen's one-time provisioning hop. It carries no JWT by design: it is
# opened on a wall screen that has never signed in, and it exists precisely
# so the kiosk token stops travelling in a URL. What it accepts is a
# single-use setup code that is redeemed and burned on first use, so it is
# open to anybody the way a password-reset link is open to anybody - the
# secret is the code, not the route.
'usb.kiosk_setup',
# Client scripts that run as SYSTEM before any user session exists.
'printers_asset.printers_for_host',

228
tests/test_core/test_settings_switches_reachable.py Normal file
View File

@@ -0,0 +1,228 @@
"""A switch nobody can flip is not a feature.
Three security switches shipped readable by their enforcement code but with no
declared default and no control, so on an air-gapped box they could only be
changed by editing the database by hand. These tests pin the two halves that
make one real: the default is SEEDED (so the row exists and the settings API
can render it), and a PUT through the public API actually changes behaviour.
collector.legacysharedkeyrefused is the one proven end to end here, because it
is core and its enforcement is a plain route refusal.
Also pinned: the upsert in update_setting must not give an UNDECLARED key
valuetype 'string' when the caller sent JSON true. That was how
plugin.endpoints.bootstrapquerytoken became a string row whose readers, doing
bool(Setting.get(key, True)), then saw the non-empty text 'false' as True.
"""
import pytest
from shopdb.core.api.collector import (
LEGACY_SHARED_KEY_SETTING,
LEGACY_SHARED_KEY_SETTING_DEFAULT,
)
from shopdb.core.api.settings import build_default_settings
from shopdb.core.models import Setting
KEY = 'testlegacykey'
LEGACY_URL = '/api/collector/heartbeat'
LEGACY_BODY = {'hostname': 'WJSW01'}
@pytest.fixture
def collector_key(app):
old = app.config.get('COLLECTOR_API_KEY')
app.config['COLLECTOR_API_KEY'] = KEY
yield KEY
app.config['COLLECTOR_API_KEY'] = old
def _defaults():
return {entry['key']: entry for entry in build_default_settings()}
def test_collector_switch_is_declared_in_the_seed_list():
"""Seeding is what creates the row; without it the toggle has nothing to
read and the UI has no control to render."""
entry = _defaults().get(LEGACY_SHARED_KEY_SETTING)
assert entry is not None, 'collector.legacysharedkeyrefused is not seeded'
assert entry is LEGACY_SHARED_KEY_SETTING_DEFAULT, \
'the seed list must reuse the declaration next to the enforcing code'
# Today's behaviour: the legacy routes stay open to the shared key.
assert entry['value'] == 'false'
assert entry['valuetype'] == 'boolean'
assert entry['category'] == 'collector'
def test_seeding_creates_the_row_as_a_real_boolean(client, db, auth_headers):
resp = client.post('/api/settings/seed', headers=auth_headers)
assert resp.status_code == 200, resp.get_json()
row = Setting.query.filter_by(key=LEGACY_SHARED_KEY_SETTING).one()
assert row.valuetype == 'boolean'
assert row.get_typed_value() is False
def test_the_seeded_switch_is_listed_by_the_settings_api(client, db,
auth_headers):
"""The settings hub control reads this list, so the key has to be in it."""
client.post('/api/settings/seed', headers=auth_headers)
resp = client.get('/api/settings', headers=auth_headers)
assert resp.status_code == 200
items = {item['key']: item for item in resp.get_json()['data']}
assert LEGACY_SHARED_KEY_SETTING in items
assert items[LEGACY_SHARED_KEY_SETTING]['value'] is False
assert items[LEGACY_SHARED_KEY_SETTING]['valuetype'] == 'boolean'
def test_put_flips_the_switch_and_the_legacy_route_closes(client, db,
auth_headers,
collector_key):
"""End to end through the public API: default on, PUT, behaviour changes."""
client.post('/api/settings/seed', headers=auth_headers)
# Default: the shared key still works on a legacy route.
assert client.post(LEGACY_URL, json=LEGACY_BODY,
headers={'X-API-Key': KEY}).status_code == 200
flip = client.put('/api/settings/' + LEGACY_SHARED_KEY_SETTING,
json={'value': True}, headers=auth_headers)
assert flip.status_code == 200, flip.get_json()
assert flip.get_json()['data']['value'] is True
refused = client.post(LEGACY_URL, json=LEGACY_BODY,
headers={'X-API-Key': KEY})
assert refused.status_code == 401, refused.get_json()
assert refused.get_json()['data']['error']['code'] == 'SHARED_KEY_REFUSED'
# And back off again, or an operator could not undo a mistake.
client.put('/api/settings/' + LEGACY_SHARED_KEY_SETTING,
json={'value': False}, headers=auth_headers)
assert client.post(LEGACY_URL, json=LEGACY_BODY,
headers={'X-API-Key': KEY}).status_code == 200
def test_an_undeclared_boolean_key_is_not_born_a_string(client, db,
auth_headers):
"""The trap that made bootstrapquerytoken misbehave.
JSON true on a key nobody declares used to create valuetype 'string', and
every boolean reader then got the text 'true'/'false' - both truthy.
"""
key = 'testundeclaredswitch'
resp = client.put('/api/settings/' + key, json={'value': False},
headers=auth_headers)
assert resp.status_code == 200, resp.get_json()
row = Setting.query.filter_by(key=key).one()
assert row.valuetype == 'boolean'
assert row.get_typed_value() is False
assert bool(Setting.get(key, True)) is False
def test_an_undeclared_number_key_is_born_a_number(client, db, auth_headers):
key = 'testundeclaredcount'
client.put('/api/settings/' + key, json={'value': 25},
headers=auth_headers)
row = Setting.query.filter_by(key=key).one()
assert row.valuetype == 'integer'
assert row.get_typed_value() == 25
# And the type is now enforced on the next write, which is the point of
# recording it: a typo cannot silently become 0 for every reader.
bad = client.put('/api/settings/' + key, json={'value': 'twenty-five'},
headers=auth_headers)
assert bad.status_code == 400
def test_an_undeclared_string_that_reads_true_stays_a_string(client, db,
auth_headers):
"""Inference reads the JSON type, never the text. A string setting whose
value happens to be 'true' must not be coerced into a boolean row."""
key = 'testundeclaredlabel'
client.put('/api/settings/' + key, json={'value': 'true'},
headers=auth_headers)
row = Setting.query.filter_by(key=key).one()
assert row.valuetype == 'string'
assert row.get_typed_value() == 'true'
def test_a_declared_key_keeps_its_declared_type(client, db, auth_headers):
"""Declaration wins over inference: the declaration is the contract every
reader was written against."""
client.put('/api/settings/' + LEGACY_SHARED_KEY_SETTING,
json={'value': 'true'}, headers=auth_headers)
row = Setting.query.filter_by(key=LEGACY_SHARED_KEY_SETTING).one()
assert row.valuetype == 'boolean'
assert row.category == 'collector'
assert row.get_typed_value() is True
# --- masking is keyed on the NAME, so it has to know the type ----------------
#
# A settings page cannot render a checkbox it is not allowed to read. The
# secret mask is a name heuristic ('password'/'token'/'secret' anywhere in the
# key), and plugin.endpoints.bootstrapquerytoken is a true/false switch whose
# name contains 'token': the API answered '********' and the control could not
# tell on from off.
def test_a_boolean_named_like_a_secret_is_not_masked(client, db, auth_headers):
key = 'plugin.endpoints.bootstrapquerytoken'
client.put('/api/settings/' + key, json={'value': False},
headers=auth_headers)
resp = client.get('/api/settings/' + key, headers=auth_headers)
assert resp.get_json()['data']['value'] is False
def test_a_string_secret_is_still_masked(client, db, auth_headers):
"""The narrowing must not open the real secrets: they are string rows."""
client.put('/api/settings/zabbix_token', json={'value': 'supersecret'},
headers=auth_headers)
resp = client.get('/api/settings/zabbix_token', headers=auth_headers)
assert resp.get_json()['data']['value'] == '********'
listed = client.get('/api/settings', headers=auth_headers).get_json()['data']
body = client.get('/api/settings', headers=auth_headers).get_data(as_text=True)
assert any(item['key'] == 'zabbix_token' for item in listed)
assert 'supersecret' not in body
def test_the_mask_still_means_unchanged_for_a_string_secret(client, db,
auth_headers):
client.put('/api/settings/zabbix_token', json={'value': 'supersecret'},
headers=auth_headers)
client.put('/api/settings/zabbix_token', json={'value': '********'},
headers=auth_headers)
assert Setting.get('zabbix_token') == 'supersecret'
# --- the control itself ------------------------------------------------------
#
# Seeding alone renders nothing: this UI has no generic key/value settings
# editor, so a core key with no page is still SQL-only on an air-gapped box.
def test_the_security_page_carries_the_collector_toggle():
from pathlib import Path
repo = Path(__file__).resolve().parents[2]
page = (repo / 'frontend' / 'src' / 'views' / 'settings'
/ 'AuthenticationSettings.vue').read_text(encoding='utf-8')
assert LEGACY_SHARED_KEY_SETTING in page, 'no control for the switch'
assert 'toggleLegacySharedKey' in page
# It has to name what stops reporting, or turning it on is a guess.
assert 'Report-AssetToShopDB.ps1' in page
assert 'Report-PrintersToShopDB.ps1' in page

8
tests/test_docs_publishable.py
View File

@@ -34,6 +34,14 @@ DOCS = REPO / 'docs'
# published mirror - keying off the directory made this module run there against
# a prose-less tree and fail. A published checkout is one with no authored
# prose, matching the canonical definition in scripts/check-gates-ran.sh.
# KNOWN GAP, left deliberately for a decision rather than closed here: this
# keys on docs/*.md, so prose that lives only in docs/wiki-drafts/ does NOT
# turn the module on, and on gh-wj that is every page there is. Pointed at the
# drafts on 2026-09-18 the scrub fails on 27 lines - site name, internal git
# server, developer home paths - which is a question about what those drafts
# are for (internal wiki, or publication) and not a bug to fix by rewriting
# them. scripts/check-gates-ran.sh matches this definition exactly; change both
# together or the gate runner reports a skip it was told to expect.
pytestmark = pytest.mark.skipif(
not any(DOCS.glob('*.md')),
reason='no docs/ prose in this checkout - it is excluded from publication and lives in the wiki')

295
tests/test_naming_gate_migrations.py Normal file
View File

@@ -0,0 +1,295 @@
"""Holes in `scripts/check-naming-and-style.sh`, each held open by a test.
The naming gate must read the column names migrations CREATE, not just models.
`scripts/check-naming-and-style.sh` checked `db.Column` attribute names on
models and nothing else, while `migrations/versions/` is filtered out of the
file list every other check in that script uses. A column is born in a
migration, so from the day the gate was written no created column name was ever
looked at: a model attribute reading fine says nothing about what the database
actually got.
Two tests, and the second is the one that matters. A gate that prints nothing
looks identical whether it is clean or broken, and this one was silently
passing everything for its whole life, so the test plants a violation and
requires the gate to fail on it. Without that, a future edit that breaks the
regex would leave every test here green.
"""
import re
import shutil
import subprocess
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parent.parent
GATE = REPO_ROOT / 'scripts' / 'check-naming-and-style.sh'
# Same shapes the gate matches: the name literal of a created column, positional
# or keyword, on the sa.Column( line or on the line after it, plus the target of
# a rename.
COLUMN_LITERAL = re.compile(r"""sa\.Column\(\s*['"]([^'"]+)['"]""", re.S)
COLUMN_KEYWORD = re.compile(
r"""sa\.Column\(\s*name\s*=\s*['"]([^'"]+)['"]""", re.S)
RENAME_LITERAL = re.compile(r"""new_column_name\s*=\s*['"]([^'"]+)['"]""")
# THE RULE, not a proxy for it. The first version of this file asked whether the
# name contained an underscore, which is the same mistake the gate's own regex
# made: 'Monitor_Name' has one but was missed for its leading capital, and a
# CamelCase name with no underscore at all is just as wrong and has nothing to
# find. Lowercase concatenated means lowercase concatenated.
LOWERCASE_CONCATENATED = re.compile(r'[a-z][a-z0-9]*\Z')
PLANTED = '''"""Planted by the test suite. Never applied."""
import sqlalchemy as sa
from alembic import op
revision = 'plantedprobe'
down_revision = None
def upgrade():
op.add_column('widgets', sa.Column('planted_column', sa.Integer(),
nullable=True))
def downgrade():
op.drop_column('widgets', 'planted_column')
'''
# ASSEMBLED FROM FRAGMENTS, like the gate's own patterns and
# tests/test_docs_publishable.py: written literally, a test for the site-literal
# rule is itself a site literal, and both the gate and the publication scrub
# would find it here.
PLANTED_SITE_HOST = 'tsg' 'wp00525' '.geaero' 'space.net'
PLANTED_AGENT = (
'// Planted by the test suite.\n'
'internal static class Probe\n'
'{\n'
' public const string DefaultBaseUrl = "https://'
+ PLANTED_SITE_HOST + '/shopdb";\n'
'}\n'
)
def _migration_files():
found = list((REPO_ROOT / 'migrations' / 'versions').glob('*.py'))
found += list(REPO_ROOT.glob('plugins/*/migrations/versions/*.py'))
return sorted(found)
def test_there_are_migrations_to_check():
"""Guard the guard: a glob that matches nothing passes everything below."""
assert _migration_files(), 'no migrations found - has the layout moved?'
def test_no_migration_creates_a_badly_named_column():
offenders = []
for path in _migration_files():
text = path.read_text(encoding='utf-8')
for pattern in (COLUMN_LITERAL, COLUMN_KEYWORD, RENAME_LITERAL):
for name in pattern.findall(text):
if not LOWERCASE_CONCATENATED.match(name):
offenders.append('{0}: {1}'.format(
path.relative_to(REPO_ROOT), name))
assert not offenders, (
'column names are lowercase concatenated (CONTRIBUTING.md):\n '
+ '\n '.join(offenders))
@pytest.mark.skipif(shutil.which('bash') is None, reason='needs bash')
def test_the_gate_fails_on_a_planted_migration_column(tmp_path):
"""Plant one, and require the gate to say so.
Run against a throwaway tree rather than the repository, so the planted
file cannot be left behind by a failing assert. tmp_path is outside any
checkout, so the script's git file list is unavailable and it takes its
own find fallback - which is the branch a developer without git metadata
gets, and so is worth exercising.
"""
versions = tmp_path / 'migrations' / 'versions'
versions.mkdir(parents=True)
gate = tmp_path / 'scripts' / 'check-naming-and-style.sh'
gate.parent.mkdir(parents=True)
shutil.copy(GATE, gate)
planted = versions / 'zz01_planted.py'
planted.write_text(PLANTED, encoding='utf-8')
dirty = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert dirty.returncode == 1, (
'the gate passed a snake_case migration column:\n' + dirty.stdout)
assert 'planted_column' in dirty.stdout
planted.unlink()
clean = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert clean.returncode == 0, (
'the gate fails with nothing planted:\n' + clean.stdout)
@pytest.mark.skipif(shutil.which('bash') is None, reason='needs bash')
def test_the_gate_fails_on_a_site_literal_in_the_agent(tmp_path):
"""agent/ is in the ADR-015 scan, and has to stay in it.
The C# agent is compiled once and installed on every bay at every site. A
site hostname, base URL or pin baked into a .cs file there ships to the next
site inside a binary, where no reviewer reads it - and agent/ was not one of
the directories the site-literal scan looked at.
"""
gate = tmp_path / 'scripts' / 'check-naming-and-style.sh'
gate.parent.mkdir(parents=True)
shutil.copy(GATE, gate)
agent = tmp_path / 'agent' / 'Probe'
agent.mkdir(parents=True)
planted = agent / 'Probe.cs'
planted.write_text(PLANTED_AGENT, encoding='utf-8')
dirty = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert dirty.returncode == 1, (
'the gate passed a site hostname in the agent:\n' + dirty.stdout)
assert 'Probe.cs' in dirty.stdout
planted.unlink()
clean = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert clean.returncode == 0, (
'the gate fails with nothing planted:\n' + clean.stdout)
# Every shape a migration writes a column name in. The gate's first regex asked
# for ['"][a-z0-9]*_ inside the quotes, so only the first of these was ever
# caught: a leading capital, a keyword argument or a CamelCase name with no
# underscore all walked past it. One case per shape, named so a failure says
# which shape regressed.
PLANTED_COLUMN_SHAPES = {
'positional_snake': (
" op.add_column('widgets', sa.Column('planted_column',\n"
" sa.Integer()))\n", 'planted_column'),
'positional_leading_capital': (
" op.add_column('widgets', sa.Column('Monitor_Name',\n"
" sa.String(50)))\n", 'Monitor_Name'),
'positional_camelcase_no_underscore': (
" op.add_column('widgets', sa.Column('MonitorName', sa.String(50)))\n",
'MonitorName'),
'keyword_name': (
" op.add_column('widgets', sa.Column(name='kw_form',\n"
" type_=sa.String()))\n", 'kw_form'),
'wrapped_after_open_paren': (
" op.add_column('widgets', sa.Column(\n"
" 'wrapped_name', sa.Integer()))\n", 'wrapped_name'),
'alter_column_rename': (
" op.alter_column('widgets', 'oldname',\n"
" new_column_name='New_Name')\n", 'New_Name'),
'create_table_second_column': (
" op.create_table('t2', sa.Column('goodone', sa.Integer()),\n"
" sa.Column('Bad_Two', sa.Integer()))\n", 'Bad_Two'),
'batch_alter_table_add': (
" with op.batch_alter_table('widgets') as batch_op:\n"
" batch_op.add_column(sa.Column('batchCol', sa.Integer()))\n",
'batchCol'),
'batch_alter_table_rename': (
" with op.batch_alter_table('widgets') as batch_op:\n"
" batch_op.alter_column('a', new_column_name='b_c')\n", 'b_c'),
}
PLANTED_CLEAN_BODY = (
" op.add_column('widgets', sa.Column('goodname', sa.Integer()))\n"
" op.add_column('widgets', sa.Column(name='alsogood', type_=sa.String()))\n"
" op.alter_column('widgets', 'oldname', new_column_name='newname')\n"
# drop_column names a column that already exists, so a bad name here is
# deliberately NOT a finding: it was wrong where it was created, and
# reporting it again makes one rename look like two.
" op.drop_column('widgets', 'legacy_old')\n"
)
def _migration_module(body):
return ('"""Planted by the test suite. Never applied."""\n'
'import sqlalchemy as sa\n'
'from alembic import op\n\n'
"revision = 'plantedprobe'\n"
'down_revision = None\n\n\n'
'def upgrade():\n' + body + '\n\n'
'def downgrade():\n'
' pass\n')
def _gate_tree(tmp_path):
gate = tmp_path / 'scripts' / 'check-naming-and-style.sh'
gate.parent.mkdir(parents=True, exist_ok=True)
shutil.copy(GATE, gate)
return gate
@pytest.mark.skipif(shutil.which('bash') is None, reason='needs bash')
@pytest.mark.parametrize('shape', sorted(PLANTED_COLUMN_SHAPES))
def test_the_gate_catches_every_column_shape(tmp_path, shape):
"""Plant one shape at a time and require the gate to name it.
Parametrised rather than one file holding all nine, so a regression says
WHICH shape stopped being seen. A single file would fail on the first one
and hide the rest.
"""
body, expected = PLANTED_COLUMN_SHAPES[shape]
versions = tmp_path / 'migrations' / 'versions'
versions.mkdir(parents=True)
gate = _gate_tree(tmp_path)
planted = versions / 'zz01_planted.py'
planted.write_text(_migration_module(body), encoding='utf-8')
dirty = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert dirty.returncode == 1, (
'the gate passed a {0} column:\n{1}'.format(shape, dirty.stdout))
assert expected in dirty.stdout, (
'the gate failed but never named {0}:\n{1}'.format(
expected, dirty.stdout))
@pytest.mark.skipif(shutil.which('bash') is None, reason='needs bash')
def test_the_gate_passes_well_named_columns(tmp_path):
"""The other half: a gate that fails on everything is no better.
Includes a drop_column with a bad name, which is out of scope on purpose.
"""
versions = tmp_path / 'migrations' / 'versions'
versions.mkdir(parents=True)
gate = _gate_tree(tmp_path)
(versions / 'zz01_clean.py').write_text(
_migration_module(PLANTED_CLEAN_BODY), encoding='utf-8')
clean = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert clean.returncode == 0, (
'the gate failed on lowercase concatenated names:\n' + clean.stdout)
@pytest.mark.skipif(shutil.which('bash') is None, reason='needs bash')
def test_the_site_literal_scan_honours_its_include_list(tmp_path):
"""The ADR-015 scan must read the file types it declares, and no others.
The scan carried --exclude=README.md AHEAD of its --include list, and on GNU
grep a file-level exclude in that position cancels every include on the
command. So the scan was reading EVERY file under agent/ and the other
scanned trees, and --include='*.cs' was decorative - adding agent/ did not
mean the C# agent was covered for the reason the flags claimed, and any
later narrowing of the include list would have been silently inert.
The .cs half is covered by the test above. This is the other half: a file
type the scan does not declare must not fail it, which is only true when the
include list is actually in force. Before the fix this run came back red on
notes.txt.
"""
gate = _gate_tree(tmp_path)
agent = tmp_path / 'agent' / 'Probe'
agent.mkdir(parents=True)
for name in ('notes.txt', 'Probe.csproj', 'README.md'):
(agent / name).write_text(
'shipped to ' + PLANTED_SITE_HOST + '\n', encoding='utf-8')
result = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert result.returncode == 0, (
'the scan read a file type it does not declare:\n' + result.stdout)

84
tests/test_openapi_spec.py
View File

@@ -72,15 +72,83 @@ def test_write_verbs_can_carry_a_body(spec):
'from this spec cannot send one:\n ' + '\n '.join(offenders[:15]))
def documented_auth(operation):
"""The auth string the generator wrote into this operation's description.
The spec is the only thing this module reads, and the description is where
the human-readable auth lands ('**Auth:** jwt-optional (...)'). Comparing
the published `security` against it is how a lie gets caught: the two come
from the same inventory entry and must agree.
"""
match = re.search(r'\*\*Auth:\*\*\s*(.+)', operation.get('description') or '')
return match.group(1).strip().lower() if match else ''
def test_optional_auth_is_expressed_as_optional(spec):
"""An endpoint that works logged out must not be published as requiring a
token - that is a lie to every reader, and it is the majority case here."""
optional = [operation for _, _, operation in operations(spec)
if any(item == {} for item in operation.get('security') or [])]
assert len(optional) > 50, (
'The inventory documents over a hundred jwt-optional endpoints; the '
'spec should express them with an empty security requirement '
'alongside the scheme, not as bearer-required.')
"""A route whose auth is optional is published as optional, either way round.
WHY THE COUNT IS GONE. This asserted `len(optional) > 50`, on the premise
that optional auth was the majority case here - it said so in as many
words. It is not any more. The guest role and the permission model moved
the read surface to `permission:*`, which publishes as bearer-required and
is correct, and the optional count fell to 14. The assertion went red
without a single route being wrong.
A count was never the property anyone cares about. It goes red when the
codebase legitimately changes shape, and it stays green while an individual
optional route is published as bearer-required, which is the actual defect.
Lowering the number to 14 would only buy time until the next legitimate
move. So the premise is dropped and the per-route property is checked
instead, whatever the total: an operation whose auth says optional carries
BOTH the empty requirement and a scheme, and an operation that publishes
the empty requirement says optional in its auth.
"""
offenders = []
for path, verb, operation in operations(spec):
auth = documented_auth(operation)
security = operation.get('security') or []
openly = any(item == {} for item in security)
scheme = any(item for item in security)
if 'optional' in auth:
if not openly:
offenders.append(
'%s %s: auth says optional, published as required (%r)'
% (verb.upper(), path, security))
elif not scheme:
offenders.append(
'%s %s: auth says optional, published with no scheme at '
'all, so a caller is never told what the credential is'
% (verb.upper(), path))
elif openly:
offenders.append(
'%s %s: published as optional, but its auth says %r'
% (verb.upper(), path, auth))
assert not offenders, (
'Optional auth means "works without a credential, does more with one", '
'and OpenAPI writes it as the empty requirement ALONGSIDE the '
'scheme:\n ' + '\n '.join(offenders[:15]))
def test_an_operation_needing_no_credential_says_so_in_its_auth(spec):
"""`security: []` is a machine-readable claim that no credential is needed.
Nine operations published it while their handler required one, including
the two that hand a bootstrapping PC its agent package - `security: []` and
a 404 for anyone who does not present the single-use token. The generator
reached that by substring: the auth string started with the word 'none', in
a sentence that went on to name the token. This is the gate on that class
of defect from the spec side, where a reader or a generated client sees it.
"""
offenders = [
'%s %s: %r' % (verb.upper(), path, documented_auth(operation))
for path, verb, operation in operations(spec)
if not operation.get('security')
and 'none' not in documented_auth(operation)
and 'public' not in documented_auth(operation)
]
assert not offenders, (
'These operations publish "no credentials needed" while their auth '
'names a credential:\n ' + '\n '.join(offenders[:15]))
def test_an_endpoint_that_needs_no_token_cannot_fail_with_401(spec):

632
tests/test_plugin_migrations.py
View File

@@ -15,6 +15,7 @@ These tests pin that contract:
* `flask plugin upgrade-all` runs clean on a fresh DB and is idempotent.
"""
import ast
import re
from pathlib import Path
import pytest
@@ -487,3 +488,634 @@ def test_upgrade_all_seeds_permissions_and_links_them_to_scopes(
(plugin_manager._app, plugin_manager._db, plugin_manager.registry,
plugin_manager.loader, plugin_manager.migration_manager,
plugin_manager._registered_prefixes) = saved
# ---------------------------------------------------------------------------
# ADR-008 lane guard: a plugin migration may only do DDL on its OWN tables.
#
# The endpoints chain broke this and nothing noticed. 0021 (shipped, since
# rewritten to a no-op stub) added five client-certificate columns plus
# idx_assets_cert_fingerprint to the CORE assets table from inside a plugin
# chain, and 0023 drops them again. Both directions are guarded and idempotent
# now, so the breach self-heals on every database - but self-healing is not
# prevention, and the only thing that stopped the next one was that somebody
# happened to read the file. That is what this guard replaces.
#
# The rule, stated as ADR-008 states it: the core Alembic chain owns core
# tables, each plugin chain owns the tables listed for it in
# PLUGIN_TABLE_OWNERS, and no chain reaches across. `assets` is not in
# PLUGIN_TABLE_OWNERS['endpoints'] and cannot be, so the per-plugin MetaData
# filter never saw the columns either: the ownership map was right and the
# migration simply ignored it.
# ---------------------------------------------------------------------------
CORE_MODELS_DIR = Path(__file__).resolve().parent.parent / 'shopdb' / 'core' / 'models'
# Alembic op name -> position of the table name in its positional arguments.
# create_index/drop_index take the INDEX name first and the table second, and
# every constraint op takes the CONSTRAINT name first, which is the whole
# reason this is a map and not "args[0]".
#
# The constraint ops are here because they are DDL on somebody's table just as
# much as add_column is: create_foreign_key / drop_constraint /
# create_unique_constraint / create_check_constraint / create_primary_key can
# all add or drop a constraint on core `assets` from inside a plugin chain.
# Leaving them out let that pass silently.
#
# create_foreign_key's REFERENT table is deliberately not recorded. The DDL
# runs against the source table only; naming another lane's table as a foreign
# key target is a read of its shape, not a write to it, and plugins legitimately
# point at core parents (network 0002 references `models`).
_TABLE_ARG_POSITION = {
'create_table': 0,
'drop_table': 0,
'add_column': 0,
'drop_column': 0,
'alter_column': 0,
'rename_table': 0,
'create_index': 1,
'drop_index': 1,
'create_foreign_key': 1,
'drop_constraint': 1,
'create_unique_constraint': 1,
'create_check_constraint': 1,
'create_primary_key': 1,
}
# Keywords that carry the table when it is not passed positionally. 0023 uses
# `table_name='assets'`, which an args-only scan walks straight past. `source`
# and `source_table` are the two spellings Alembic has used for
# create_foreign_key's own table; `referent`/`referent_table` are absent on
# purpose, for the reason given above.
_TABLE_KEYWORDS = ('table_name', 'source', 'source_table', 'old_table_name')
# Op name -> position of the COLUMN name in its positional arguments, for the
# ops that name exactly one column. The allowlist keys on this as well as the
# table, so an entry excusing a shipped drop_column cannot also excuse a NEW
# drop_column on the same table in the same file.
_COLUMN_ARG_POSITION = {
'add_column': 1,
'drop_column': 1,
'alter_column': 1,
}
# Same, for the batch_alter_table forms, where the table moved onto the context
# manager and every positional argument shifted left by one.
_BATCH_COLUMN_ARG_POSITION = {
'add_column': 0,
'drop_column': 0,
'alter_column': 0,
}
# --- raw SQL -----------------------------------------------------------------
# op.execute() and Connection.exec_driver_sql() hand a string straight to the
# database, so an AST scan that only knows the op.* helpers sees zero ops for
# `op.execute('ALTER TABLE assets DROP COLUMN assetnumber')`. That is the
# cheapest possible way around every other guard in this file, so the statement
# text is parsed too.
#
# Only DDL against a named table counts. UPDATE/DELETE/INSERT/SELECT are data,
# not schema: three plugin revisions already run cross-checked DML
# (backups 0002, backups 0003, computers 0006) and none of them is a lane
# crossing.
#
# Unresolvable text is rendered as a sentinel character rather than dropped, so
# the difference between "interpolates the constraint name" and "interpolates
# the table name" survives into the regex. machines 0002 does the first
# (`f"ALTER TABLE equipment DROP FOREIGN KEY {name}"` - the table is literal
# text, the FK name is not) and stays resolvable; the dodge this guards against
# is the second (`f'ALTER TABLE {t} DROP COLUMN x'`), where the sentinel lands
# inside the captured table name and the op reports UNRESOLVED, which
# test_plugin_migration_ops_all_resolve_to_a_table treats as a failure.
_SQL_HOLE = '\x00'
# Identifier characters plus the sentinel, so an interpolated name is captured
# rather than silently truncated to the literal prefix beside it.
_SQL_IDENT = r'[A-Za-z0-9_$`"\[\]\.' + _SQL_HOLE + r']+'
# (op name, compiled pattern, number of table groups). Ordered only for
# readability; every pattern is tried against every statement.
_SQL_DDL_PATTERNS = (
('execute:alter_table',
re.compile(r'\bALTER\s+TABLE\s+(?:IF\s+EXISTS\s+)?(' + _SQL_IDENT + r')', re.I)),
('execute:drop_table',
re.compile(r'\bDROP\s+TABLE\s+(?:IF\s+EXISTS\s+)?(' + _SQL_IDENT + r')', re.I)),
('execute:create_table',
re.compile(r'\bCREATE\s+(?:TEMPORARY\s+)?TABLE\s+(?:IF\s+NOT\s+EXISTS\s+)?('
+ _SQL_IDENT + r')', re.I)),
('execute:rename_table',
re.compile(r'\bRENAME\s+TABLE\s+(' + _SQL_IDENT + r')\s+TO\s+('
+ _SQL_IDENT + r')', re.I)),
('execute:create_index',
re.compile(r'\bCREATE\s+(?:UNIQUE\s+|FULLTEXT\s+|SPATIAL\s+)?INDEX\s+'
+ _SQL_IDENT + r'\s+ON\s+(' + _SQL_IDENT + r')', re.I)),
('execute:drop_index',
re.compile(r'\bDROP\s+INDEX\s+' + _SQL_IDENT + r'\s+ON\s+('
+ _SQL_IDENT + r')', re.I)),
('execute:truncate_table',
re.compile(r'\bTRUNCATE\s+(?:TABLE\s+)?(' + _SQL_IDENT + r')', re.I)),
)
# Statement verbs that are data, not schema. A statement starting with one of
# these is skipped outright, even if its body happens to contain a word like
# "table"; anything starting with something unreadable is recorded UNRESOLVED,
# because `op.execute(sql)` with sql built elsewhere is the same dodge as an
# f-string.
_SQL_DML_VERBS = ('select', 'insert', 'update', 'delete', 'replace', 'with',
'set', 'commit', 'rollback', 'analyze', 'optimize', 'call',
'pragma', 'flush', 'use', 'show', 'start')
# Historical lane crossings that have already shipped to production. Every entry
# is closed - listed so the guard is green today and red for anything new, NOT
# as a precedent to extend. Adding a line here means a plugin chain wrote to
# somebody else's table, which needs an ADR amendment, not an allowlist edit.
#
# The key is (plugin, filename, op, table, column). The column is part of it on
# purpose: keyed on the first four alone, the endpoints 0023 drop_column entry
# would also excuse a brand new `op.drop_column('assets', 'assetnumber')`
# appended to that same shipped file, which is exactly the breach this guard
# exists to stop. `None` means the op names no single column (drop_index) or
# names one the scan cannot resolve.
#
# endpoints 0023 is the CLEANUP of the 0021/0022 client-certificate breach: it
# drops the five cert columns and any index over them from the core assets
# table. It is applied on WJ production, so it cannot be edited - the fix is
# that no 0025 may do the same thing.
#
# 0021 and 0022 themselves are NOT listed, and must not be: they were rewritten
# to stamp-only no-ops, so they present no DDL to scan. Their damage lives on
# only as the 0023 cleanup below.
#
# The drop_index half of that cleanup (0023:71-73) reads alarmingly: it drops
# ANY index on assets whose column set INTERSECTS the cert columns, and never
# recreates one. An index over (assetid, client_cert_fingerprint) is dead once
# the column goes, but an index that merely happened to include a cert column
# alongside columns that still matter would be silently lost. Settled here so
# the next reader does not re-derive it - only these things ever create an index
# on assets:
# * core 68b3947ae14f baseline: idx_asset_active, idx_asset_location,
# idx_asset_status, idx_asset_type_bu, ix_assets_assetnumber,
# ix_assets_serialnumber
# * core 7b02: ix_assets_gaugelabreference, ix_assets_maintenancereference
# * core 7d33: idx_assets_levelid
# * the shipped endpoints 0021: idx_assets_cert_fingerprint over
# (client_cert_fingerprint) alone. The shipped 0022 added two more cert
# columns and no index at all.
# * db.create_all off shopdb/core/models/asset.py. Between 18e5fdc00
# (2026-09-14) and 7480e4fe3 (2026-09-15) that model DID carry the five cert
# columns, with client_cert_fingerprint declared index=True, so create_all
# built ix_assets_client_cert_fingerprint on every database created in that
# window. That is a second cert index, not just the one 0021 named, and the
# inventory above is wrong without it.
# It changes nothing for this loop. ix_assets_client_cert_fingerprint is
# single-column over client_cert_fingerprint, which 0023 drops in the very next
# statement, so the index carries no column that outlives it and there is
# nothing to recreate. The conclusion holds: the only indexes the loop can drop
# are idx_assets_cert_fingerprint and ix_assets_client_cert_fingerprint, both
# dead by construction. No index was lost and no repair migration is owed.
#
# The filter is also unreachable by accident: every name in _CERT_COLUMNS is
# underscore-spelled, which this repo's column convention forbids, so no real
# assets column can collide with it.
ALLOWED_CROSS_LANE_OPS = {
('endpoints', '0023_endpoint_credentials.py', 'drop_index', 'assets', None),
('endpoints', '0023_endpoint_credentials.py', 'drop_column', 'assets',
'client_cert_fingerprint'),
('endpoints', '0023_endpoint_credentials.py', 'drop_column', 'assets',
'cert_issued_at'),
('endpoints', '0023_endpoint_credentials.py', 'drop_column', 'assets',
'cert_expires_at'),
('endpoints', '0023_endpoint_credentials.py', 'drop_column', 'assets',
'cert_revoked_at'),
('endpoints', '0023_endpoint_credentials.py', 'drop_column', 'assets',
'cert_revocation_reason'),
}
def _core_tablenames() -> set:
"""Every table the CORE chain owns, read statically off shopdb/core/models.
Picks up both `__tablename__` on a model and the bare `db.Table(...)`
association tables (userroles, rolepermissions, userpermissions), which have
no model class and would otherwise read as unowned.
"""
names = set()
for source in CORE_MODELS_DIR.glob('*.py'):
tree = ast.parse(source.read_text())
for node in ast.walk(tree):
if isinstance(node, ast.Assign):
targets = [t.id for t in node.targets if isinstance(t, ast.Name)]
if '__tablename__' in targets and isinstance(node.value, ast.Constant):
names.add(node.value.value)
if isinstance(node, ast.Call):
func = node.func
name = func.attr if isinstance(func, ast.Attribute) else getattr(func, 'id', None)
if name == 'Table' and node.args:
first = node.args[0]
if isinstance(first, ast.Constant) and isinstance(first.value, str):
names.add(first.value)
return names
def _table_owner_map() -> dict:
"""table name -> owning lane ('core' or a plugin name)."""
owners = {}
for plugin, tables in PLUGIN_TABLE_OWNERS.items():
for tablename in tables:
owners[tablename] = plugin
for tablename in _core_tablenames():
# A plugin entry wins: pre-cutover the core chain CREATED the plugin
# tables, so a name can look core-shaped and still belong to a plugin.
owners.setdefault(tablename, 'core')
return owners
def _module_constants(tree) -> dict:
"""Module-level `NAME = 'sometable'` bindings.
Half the chains name their tables through constants (CREDENTIALS,
REQUESTS, _APPS), so a scan that only understands string literals sees
almost nothing.
"""
consts = {}
for node in tree.body:
if isinstance(node, ast.Assign) and isinstance(node.value, ast.Constant) \
and isinstance(node.value.value, str):
for target in node.targets:
if isinstance(target, ast.Name):
consts[target.id] = node.value.value
return consts
def _module_sequences(tree) -> dict:
"""Module-level `NAME = ('a', 'b')` bindings of all-string sequences.
Needed for the column half of the allowlist key: endpoints 0023 drops its
five columns with `for column in _CERT_COLUMNS`, and without this the column
resolves to nothing and the allowlist entry degrades back to a blanket
"drop_column on assets from this file is fine".
"""
seqs = {}
for node in tree.body:
if not isinstance(node, ast.Assign):
continue
if not isinstance(node.value, (ast.Tuple, ast.List, ast.Set)):
continue
values = set()
for element in node.value.elts:
if not (isinstance(element, ast.Constant) and isinstance(element.value, str)):
values = set()
break
values.add(element.value)
if not values:
continue
for target in node.targets:
if isinstance(target, ast.Name):
seqs[target.id] = frozenset(values)
return seqs
def _literal(node, consts):
if isinstance(node, ast.Constant) and isinstance(node.value, str):
return node.value
if isinstance(node, ast.Name):
return consts.get(node.id)
return None
def _loop_bindings(tree, consts, seqs) -> dict:
"""Map each Call node to the loop variables in scope over a literal sequence.
`for table in (IMPACTS, TRANSITIONS, HEALTHSTATES): op.drop_table(table)`
(endpoints 0017) and the same shape in guides 0001 are real DDL over real
tables. Without this they resolve to nothing, and "resolves to nothing" has
to be a failure or the guard is dodgeable by writing a loop.
A bare module constant as the iterable (`for column in _CERT_COLUMNS`) is
resolved through `seqs` for the same reason.
"""
bindings = {}
for node in ast.walk(tree):
if not isinstance(node, ast.For) or not isinstance(node.target, ast.Name):
continue
values = set()
if isinstance(node.iter, (ast.Tuple, ast.List, ast.Set)):
for element in node.iter.elts:
value = _literal(element, consts)
if value is None:
values = set()
break
values.add(value)
elif isinstance(node.iter, ast.Name):
values = set(seqs.get(node.iter.id, ()))
if not values:
continue
# ast.walk is breadth-first from the top, so an inner loop rebinding the
# same name is visited after the outer one and correctly shadows it.
for sub in ast.walk(node):
if isinstance(sub, ast.Call):
bindings.setdefault(id(sub), {})[node.target.id] = frozenset(values)
return bindings
def _resolve_tables(node, consts, loopvars) -> frozenset:
"""Every table name an argument could be. Empty means unresolvable."""
if isinstance(node, ast.Constant) and isinstance(node.value, str):
return frozenset({node.value})
if isinstance(node, ast.Name):
if node.id in loopvars:
return loopvars[node.id]
if node.id in consts:
return frozenset({consts[node.id]})
return frozenset()
def _resolve_columns(opname, node, args, consts, loopvars, positions) -> frozenset:
"""The column name(s) an op names, or empty when it names none.
Empty is NOT a failure here the way it is for a table: most ops name no
column at all. It only widens the allowlist key to None.
"""
position = positions.get(opname)
if position is None or len(args) <= position:
return frozenset()
argument = args[position]
if opname == 'add_column':
# add_column takes sa.Column('name', ...), not a bare string.
if isinstance(argument, ast.Call) and argument.args:
argument = argument.args[0]
else:
return frozenset()
return _resolve_tables(argument, consts, loopvars)
def _flatten_sql(node, consts) -> str:
"""Statement text with every unresolvable fragment replaced by a sentinel.
Keeping the hole in place (instead of giving up on the whole string) is what
lets `f"ALTER TABLE equipment DROP FOREIGN KEY {name}"` resolve to
`equipment` while `f'ALTER TABLE {t} DROP COLUMN x'` resolves to nothing.
"""
if isinstance(node, ast.Constant):
return node.value if isinstance(node.value, str) else _SQL_HOLE
if isinstance(node, ast.JoinedStr):
return ''.join(_flatten_sql(value, consts) for value in node.values)
if isinstance(node, ast.FormattedValue):
return _SQL_HOLE
if isinstance(node, ast.BinOp) and isinstance(node.op, ast.Add):
return _flatten_sql(node.left, consts) + _flatten_sql(node.right, consts)
if isinstance(node, ast.Name):
return consts.get(node.id, _SQL_HOLE)
if isinstance(node, ast.Call):
# sa.text('...') / text('...') is a wrapper, not a hole: computers 0006
# passes its statement that way.
func = node.func
name = func.attr if isinstance(func, ast.Attribute) else getattr(func, 'id', None)
if name == 'text' and len(node.args) == 1:
return _flatten_sql(node.args[0], consts)
return _SQL_HOLE
def _clean_sql_identifier(raw: str) -> str:
"""`db`.`assets` -> assets. Keeps the sentinel if one landed in the name."""
name = raw.strip().rstrip('(;,')
if '.' in name:
name = name.rsplit('.', 1)[-1]
return name.strip('`"[]')
def _scan_raw_sql(node, consts) -> list:
"""(opname, frozenset-of-tables) for the DDL in one execute-style call.
An empty table set is UNRESOLVED and the caller fails on it. A statement
that is plainly DML returns nothing at all.
"""
statement = _flatten_sql(node, consts)
found = []
# Classify PER STATEMENT, not per call. Reading the first verb and bailing
# let a leading DML verb hide trailing DDL: op.execute("SET @x = 1; ALTER
# TABLE assets DROP COLUMN serialnumber") was skipped whole, because the
# scan saw SET, called it DML and returned nothing.
for part in statement.split(';'):
stripped = part.lstrip().lstrip('(')
verb = stripped.split(None, 1)[0].lower() if stripped.split() else ''
if not verb:
continue
if verb.startswith(_SQL_HOLE):
# The verb itself is interpolated, so nothing about this statement
# can be checked. Unresolved op rather than no op.
found.append(('execute:unknown', frozenset()))
continue
if verb in _SQL_DML_VERBS:
continue
for opname, pattern in _SQL_DDL_PATTERNS:
for match in pattern.finditer(part):
for group in match.groups():
name = _clean_sql_identifier(group)
if not name or _SQL_HOLE in name:
found.append((opname, frozenset()))
else:
found.append((opname, frozenset({name})))
return found
def _scan_plugin_migration_ops() -> list:
"""Every schema op in every plugin chain, as
(plugin, filename, lineno, opname, frozenset-of-tables,
frozenset-of-columns).
An empty table set means the argument could not be resolved statically;
the caller treats that as a failure rather than a pass. An empty column set
only means the op names no resolvable single column.
"""
found = []
for script in sorted(PLUGINS_DIR.glob('*/migrations/versions/*.py')):
plugin = script.parts[-4]
tree = ast.parse(script.read_text())
consts = _module_constants(tree)
seqs = _module_sequences(tree)
loops = _loop_bindings(tree, consts, seqs)
# batch_alter_table('assets') moves the table onto the context manager,
# so batch_op.drop_column('x') names a column where op.drop_column names
# a table. Resolve those against the `with` instead.
batched = {}
for node in ast.walk(tree):
if not isinstance(node, ast.With):
continue
for item in node.items:
call = item.context_expr
if not (isinstance(call, ast.Call) and isinstance(call.func, ast.Attribute)
and call.func.attr == 'batch_alter_table' and call.args):
continue
tables = _resolve_tables(call.args[0], consts, loops.get(id(call), {}))
alias = item.optional_vars.id if isinstance(item.optional_vars, ast.Name) else None
for sub in ast.walk(node):
if isinstance(sub, ast.Call) and isinstance(sub.func, ast.Attribute) \
and sub.func.attr in _TABLE_ARG_POSITION \
and isinstance(sub.func.value, ast.Name) \
and sub.func.value.id == alias:
batched[id(sub)] = tables
for node in ast.walk(tree):
if not (isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute)):
continue
opname = node.func.attr
base = node.func.value
# Raw SQL: op.execute(...), bind.execute(...) and any
# <connection>.exec_driver_sql(...).
#
# Both names are matched on the method name ALONE, without looking at
# what they are called on. Requiring the receiver to be literally `op`
# missed the repo's DOMINANT idiom: 24 call sites across endpoints
# 0006/0007/0010/0019, network 0003 and notifications 0005 all do
# `bind = op.get_bind()` and then `bind.execute(...)`, so a plugin
# could drop a column off the core assets table through a bind and
# the scan recorded nothing at all. Every shipped bind.execute passes
# a literal SELECT or UPDATE, which _SQL_DML_VERBS skips, so widening
# this costs no false positives.
if opname in ('exec_driver_sql', 'execute'):
if not node.args:
continue
for sqlop, tables in _scan_raw_sql(node.args[0], consts):
found.append((plugin, script.name, node.lineno, sqlop,
tables, frozenset()))
continue
if opname not in _TABLE_ARG_POSITION:
continue
loopvars = loops.get(id(node), {})
if id(node) in batched:
columns = _resolve_columns(opname, node, node.args, consts, loopvars,
_BATCH_COLUMN_ARG_POSITION)
found.append((plugin, script.name, node.lineno,
'batch_' + opname, batched[id(node)], columns))
continue
if not (isinstance(base, ast.Name) and base.id == 'op'):
continue
position = _TABLE_ARG_POSITION[opname]
tables = frozenset()
if len(node.args) > position:
tables = _resolve_tables(node.args[position], consts, loopvars)
for keyword in node.keywords:
if keyword.arg in _TABLE_KEYWORDS:
tables = tables | _resolve_tables(keyword.value, consts, loopvars)
columns = _resolve_columns(opname, node, node.args, consts, loopvars,
_COLUMN_ARG_POSITION)
found.append((plugin, script.name, node.lineno, opname, tables, columns))
if opname == 'rename_table' and len(node.args) > 1:
# Both ends of a rename are DDL on a table.
found.append((plugin, script.name, node.lineno, 'rename_table',
_resolve_tables(node.args[1], consts, loopvars),
frozenset()))
return found
def _cross_lane_ops() -> list:
"""Scanned ops whose table belongs to core or to a different plugin.
A table that is in neither the core models nor PLUGIN_TABLE_OWNERS is NOT a
crossing: that is a pre-rename name (machines 0002 renames equipment ->
machines) or a table since dropped, and neither reaches into another lane.
Ownership drift of a table that DOES still exist is already covered by
test_owned_tables_match_declared_models.
"""
owners = _table_owner_map()
crossings = []
for plugin, filename, lineno, opname, tables, columns in _scan_plugin_migration_ops():
for tablename in sorted(tables):
owner = owners.get(tablename)
if owner is None or owner == plugin:
continue
for column in (sorted(columns) or [None]):
crossings.append((plugin, filename, lineno, opname, tablename,
owner, column))
return crossings
def _crossing_keys() -> set:
return {(plugin, filename, opname, tablename, column)
for plugin, filename, _, opname, tablename, _, column in _cross_lane_ops()}
def test_plugin_migration_ops_all_resolve_to_a_table():
"""Every scanned op names a table the scan can actually resolve.
Without this the guard is dodgeable by accident: a table name built at
runtime resolves to nothing, and "nothing" would otherwise read as "touches
no foreign table" and pass. It is also what makes the raw-SQL scan real -
`op.execute(f'ALTER TABLE {t} ...')` lands here rather than going quietly
green.
"""
unresolved = [(plugin, filename, lineno, opname)
for plugin, filename, lineno, opname, tables, _
in _scan_plugin_migration_ops() if not tables]
assert not unresolved, (
'plugin migration ops whose table name could not be resolved '
f'statically: {unresolved}')
def test_scan_sees_the_known_lane_crossing():
"""The scanner is not vacuous: it still finds the 0023 assets cleanup.
If a refactor breaks the AST walk, every other assertion here goes green on
an empty result set. This one goes red.
"""
scanned = _scan_plugin_migration_ops()
assert len(scanned) > 100, f'scan found only {len(scanned)} ops; it is broken'
keys = _crossing_keys()
assert ALLOWED_CROSS_LANE_OPS <= keys, (
'the allowlisted historical crossings are no longer detected: '
f'{ALLOWED_CROSS_LANE_OPS - keys}')
def test_scan_reads_raw_sql_ddl():
"""The raw-SQL half is not vacuous either.
machines 0002 does its whole MySQL rename through exec_driver_sql. If the
statement parser stops working, those ops vanish and nothing else in this
file notices.
"""
raw = [(plugin, filename, opname, tuple(sorted(tables)))
for plugin, filename, _, opname, tables, _
in _scan_plugin_migration_ops() if opname.startswith('execute:')]
assert ('machines', '0002_rename_from_equipment.py', 'execute:rename_table',
('equipmenttypes',)) in raw, raw
assert ('machines', '0002_rename_from_equipment.py', 'execute:alter_table',
('machines',)) in raw, raw
# DML stays out of it, or the three shipped UPDATE statements would read as
# schema changes on tables their own plugin owns.
assert not [entry for entry in raw
if entry[0] == 'backups' or entry[1].startswith('0006_policy')], raw
def test_no_plugin_migration_touches_another_lanes_table():
"""ADR-008: a plugin chain does DDL only on the tables it owns.
Fails on create_table/drop_table/add_column/drop_column/alter_column/
create_index/drop_index/rename_table, the five constraint ops, the
batch_alter_table forms, and DDL inside op.execute/exec_driver_sql, against
a core table or another plugin's table. The allowlist above holds exactly
the entries that already shipped.
"""
offenders = [crossing for crossing in _cross_lane_ops()
if (crossing[0], crossing[1], crossing[3], crossing[4], crossing[6])
not in ALLOWED_CROSS_LANE_OPS]
assert not offenders, '\n'.join(
f'{plugin} chain: {filename}:{lineno} {opname} on {tablename!r}'
+ (f' column {column!r}' if column else '')
+ f', owned by {owner} (ADR-008)'
for plugin, filename, lineno, opname, tablename, owner, column in offenders)
def test_cross_lane_allowlist_has_no_dead_entries():
"""The allowlist does not outlive what it excuses.
An entry left behind after its migration is squashed or rewritten is a
standing licence to write to a core table from that file again.
"""
keys = _crossing_keys()
assert not (ALLOWED_CROSS_LANE_OPS - keys), (
'allowlist entries that match nothing in the tree: '
f'{ALLOWED_CROSS_LANE_OPS - keys}')

286
tests/test_plugins/test_endpoints_bootstrap_install.py
View File

@@ -11,6 +11,7 @@ import datetime
import hashlib
import io
import json
import logging
import zipfile
import pytest
@@ -27,17 +28,51 @@ TOKENS_URL = '/api/endpoints/siteagent/bootstrap/tokens'
SCRIPT_URL = '/api/endpoints/siteagent/bootstrap.ps1'
PACKAGE_URL = '/api/endpoints/siteagent/bootstrap/package'
BASE_URL = 'https://shopdb.test/shopdb'
TEMP_SCRIPT = r'C:\Windows\Temp\shopdb-bootstrap.ps1'
TOKEN_HEADER = 'X-Endpoint-Bootstrap-Token'
QUERY_SETTING = 'plugin.endpoints.bootstrapquerytoken'
# Admin-only staging, NOT C:\Windows\Temp: a standard user can pre-plant a
# path there and have it run elevated.
STAGE_DIR = r'C:\Program Files\ShopDB\bootstrap'
STAGED_SCRIPT = STAGE_DIR + r'\shopdb-bootstrap.ps1'
ADMIN_SID = 'S-1-5-32-544'
SYSTEM_SID = 'S-1-5-18'
# The marker the admin "Download .ps1" link carries, so the server can tell an
# admin's browser click apart from a bay still on a pre-change pasted command.
UI_LINK_QUERY = 'via=download'
def _expected_command(secret, base=BASE_URL):
"""Download, run, delete. Never `irm | iex` - Defender kills that shape."""
return ('powershell -ExecutionPolicy Bypass -c "'
"try {{ Invoke-WebRequest -UseBasicParsing "
"'{0}/api/endpoints/siteagent/bootstrap.ps1?t={1}' "
"-OutFile '{2}'; & '{2}' }} finally {{ "
"Remove-Item '{2}' -Force -ErrorAction SilentlyContinue }}\""
).format(base, secret, TEMP_SCRIPT)
"""Stage admin-only, download with the token in a header, check the owner,
run, delete. Never `irm | iex` - Defender kills that shape."""
return (
'powershell -ExecutionPolicy Bypass -c "'
"try {{ New-Item -ItemType Directory -Force -Path '{dir}' "
'| Out-Null; '
"icacls '{dir}' /inheritance:r /grant '*{admin}:(OI)(CI)F' "
"'*{system}:(OI)(CI)F' | Out-Null; "
'if ((Get-Variable LASTEXITCODE -ValueOnly) -ne 0) '
"{{ Write-Host 'Refusing to run: could not restrict "
"{dir} to administrators (icacls failed). Fix the directory "
"or delete it, then run the command again.' -ForegroundColor Red; "
'exit 6 }}; '
"Remove-Item '{path}' -Force -ErrorAction SilentlyContinue; "
"Invoke-WebRequest -UseBasicParsing "
"'{base}/api/endpoints/siteagent/bootstrap.ps1' "
"-Headers @{{'{header}'='{secret}'}} -OutFile '{path}'; "
"if ((Get-Acl '{path}').GetOwner("
'[Security.Principal.SecurityIdentifier]).Value -notin '
"@('{admin}','{system}',"
'[Security.Principal.WindowsIdentity]::GetCurrent().User.Value)) '
"{{ Write-Host 'Refusing to run: the staged bootstrap script is not "
"owned by an administrator.' -ForegroundColor Red; exit 5 }}; "
"& '{path}' }} finally {{ Remove-Item '{path}' -Force "
'-ErrorAction SilentlyContinue }}"'
).format(dir=STAGE_DIR, path=STAGED_SCRIPT, base=base, header=TOKEN_HEADER,
secret=secret, admin=ADMIN_SID, system=SYSTEM_SID)
def _token_header(secret):
return {TOKEN_HEADER: secret}
def _make_cert():
@@ -111,8 +146,8 @@ def test_generate_returns_the_secret_and_the_ready_made_command(
assert len(secret) == len('sdbb_') + 43
assert data['command'] == _expected_command(secret)
assert data['scripturl'] == (
'{0}/api/endpoints/siteagent/bootstrap.ps1?t={1}'.format(BASE_URL,
secret))
'{0}/api/endpoints/siteagent/bootstrap.ps1?t={1}&{2}'.format(
BASE_URL, secret, UI_LINK_QUERY))
token = data['token']
assert token['tokenprefix'] == secret[:12]
assert token['maxuses'] == 3
@@ -136,13 +171,74 @@ def test_the_command_downloads_and_runs_instead_of_piping_into_iex(
assert 'iex' not in command.lower()
assert '$' not in command
assert '\n' not in command
assert data['scripturl'] in command
assert data['secret'] in command
assert "Remove-Item '{0}' -Force".format(TEMP_SCRIPT) in command
assert "Remove-Item '{0}' -Force".format(STAGED_SCRIPT) in command
assert 'finally' in command
# The link the UI offers instead is the route that really serves it.
assert client.get(SCRIPT_URL,
query_string={'t': data['secret']}).status_code == 200
assert client.get(data['scripturl'].split(BASE_URL, 1)[1]
).status_code == 200
def test_the_command_stages_where_only_administrators_can_write(
client, db, auth_headers):
"""C:\\Windows\\Temp lets any standard user create the exact path the
command runs elevated, so the staging directory moved under Program Files,
is created with an admin-only DACL, and the command refuses to run a staged
file owned by anyone but an administrator."""
command = _secret(client, auth_headers)['command']
assert r'C:\Windows\Temp' not in command
assert "-OutFile '{0}'".format(STAGED_SCRIPT) in command
assert "New-Item -ItemType Directory -Force -Path '{0}'".format(
STAGE_DIR) in command
assert "icacls '{0}' /inheritance:r".format(STAGE_DIR) in command
# Well-known SIDs, so a renamed or localized group cannot widen the grant.
assert "/grant '*{0}:(OI)(CI)F' '*{1}:(OI)(CI)F'".format(
ADMIN_SID, SYSTEM_SID) in command
assert "Get-Acl '{0}'".format(STAGED_SCRIPT) in command
assert 'GetOwner([Security.Principal.SecurityIdentifier]).Value' in command
assert "-notin @('{0}','{1}',".format(ADMIN_SID, SYSTEM_SID) in command
assert '[Security.Principal.WindowsIdentity]::GetCurrent().User.Value' \
in command
assert 'not owned by an administrator' in command
assert 'exit 5' in command
# A leftover from an earlier run is cleared before the download, so the
# owner test cannot pass on stale content.
assert command.index("Remove-Item '{0}'".format(STAGED_SCRIPT)) < \
command.index('Invoke-WebRequest')
def test_the_command_sends_the_token_in_a_header_not_the_query_string(
client, db, auth_headers):
"""A query string lands verbatim in the nginx and gunicorn access logs."""
data = _secret(client, auth_headers)
command = data['command']
assert '?t=' not in command
assert "-Headers @{{'{0}'='{1}'}}".format(TOKEN_HEADER,
data['secret']) in command
assert "'{0}/api/endpoints/siteagent/bootstrap.ps1'".format(
BASE_URL) in command
# The browser download link is the one place that cannot send a header,
# and it is marked so its own fetches are not counted as un-migrated bays.
assert data['scripturl'].endswith(
'?t={0}&{1}'.format(data['secret'], UI_LINK_QUERY))
def test_a_failed_acl_hardening_stops_the_command(client, db, auth_headers):
"""icacls is a native command: it raises nothing and $ErrorActionPreference
never applies to it, so an unchecked call leaves a staging directory that
still inherits the parent DACL and the line downloads into it anyway. The
exit code is read sigil-free, because the whole line must hold no `$`."""
command = _secret(client, auth_headers)['command']
assert '(Get-Variable LASTEXITCODE -ValueOnly) -ne 0' in command
assert 'LASTEXITCODE' in command and '$LASTEXITCODE' not in command
assert 'icacls failed' in command
assert 'exit 6' in command
# Checked before anything is fetched into the directory, not after.
assert command.index('LASTEXITCODE') > command.index('icacls')
assert command.index('LASTEXITCODE') < command.index('Invoke-WebRequest')
def test_defaults_are_a_day_and_twenty_five_uses(client, db, auth_headers):
@@ -226,8 +322,8 @@ def test_script_is_served_for_a_good_token(client, db, auth_headers):
assert response.mimetype == 'text/plain'
script = response.get_data(as_text=True)
script.encode('ascii')
assert '{0}/api/endpoints/siteagent/bootstrap/package?t={1}'.format(
BASE_URL, secret) in script
assert '"{0}/api/endpoints/siteagent/bootstrap/package"'.format(
BASE_URL) in script
assert 'Run as administrator' in script
assert 'GE-Enforce' in script
assert 'SHOPDB_BOOTSTRAP_FORCE' in script
@@ -251,6 +347,163 @@ def test_script_bakes_the_staged_binary_hashes(client, db, auth_headers,
assert hashlib.sha256(traypayload).hexdigest() in script
def test_the_generated_script_fetches_the_package_with_the_header(
client, db, auth_headers):
"""Same reason as the one-liner: the package URL must not carry the token,
or every agent install writes it into an access log."""
secret = _secret(client, auth_headers)['secret']
script = client.get(SCRIPT_URL,
headers=_token_header(secret)).get_data(as_text=True)
assert '?t=' not in script
assert '$BootstrapPackageHeaders = @{{ "{0}" = "{1}" }}'.format(
TOKEN_HEADER, secret) in script
assert ('Invoke-WebRequest -Uri $BootstrapPackageUrl '
'-Headers $BootstrapPackageHeaders') in script
def test_both_routes_take_the_token_in_a_header(client, db, auth_headers,
servercert):
secret = _secret(client, auth_headers, maxuses=2)['secret']
script = client.get(SCRIPT_URL, headers=_token_header(secret))
assert script.status_code == 200
assert 'ShopDB endpoint agent bootstrap' in script.get_data(as_text=True)
assert 'install.ps1' in _files(
client.get(PACKAGE_URL, headers=_token_header(secret)))
assert EndpointBootstrapToken.query.one().usecount == 1
@pytest.mark.parametrize('url', [SCRIPT_URL, PACKAGE_URL])
def test_a_bad_header_token_is_not_found(client, db, auth_headers, url):
"""The header is read first, so a good ?t= cannot rescue a bad header - one
request carries one token."""
secret = _secret(client, auth_headers)['secret']
response = client.get(url, query_string={'t': secret},
headers=_token_header('sdbb_' + 'a' * 43))
assert response.status_code == 404
def test_the_query_form_still_works_and_is_audited_as_deprecated(
client, db, auth_headers, caplog):
"""A bay running a command pasted before this change must keep working, so
the query form stays accepted; it is flagged in the audit row and warned
about in the log, with no token value in either."""
secret = _secret(client, auth_headers)['secret']
with caplog.at_level(logging.WARNING):
assert client.get(SCRIPT_URL, query_string={'t': secret},
environ_overrides={'REMOTE_ADDR': '10.0.0.6'}
).status_code == 200
row = AuditLog.query.filter_by(entitytype='EndpointBootstrapToken',
action='requested').one()
assert 'query-string token, deprecated' in row.details
assert secret not in row.details
warnings = [record.getMessage() for record in caplog.records
if record.levelno >= logging.WARNING]
assert any('query string' in message and '10.0.0.6' in message
for message in warnings)
assert all(secret not in message for message in warnings)
def test_the_header_form_is_not_flagged_as_deprecated(client, db, auth_headers,
caplog):
secret = _secret(client, auth_headers)['secret']
with caplog.at_level(logging.WARNING):
client.get(SCRIPT_URL, headers=_token_header(secret))
row = AuditLog.query.filter_by(entitytype='EndpointBootstrapToken',
action='requested').one()
assert 'deprecated' not in row.details
assert not [record for record in caplog.records
if record.levelno >= logging.WARNING]
def test_the_admin_download_link_is_not_counted_as_an_un_migrated_bay(
client, db, auth_headers, caplog):
"""The rollout finds un-migrated bays by the deprecation WARNING, and the
UI Download .ps1 link has to use the query form. Marked, it logs at info
and is audited as the admin path, so an admin testing a token does not
invent a bay that is not there."""
data = _secret(client, auth_headers)
path = data['scripturl'].split(BASE_URL, 1)[1]
with caplog.at_level(logging.INFO):
assert client.get(path, environ_overrides={'REMOTE_ADDR': '10.0.0.7'}
).status_code == 200
row = AuditLog.query.filter_by(entitytype='EndpointBootstrapToken',
action='requested').one()
assert 'admin download link' in row.details
assert 'deprecated' not in row.details
assert data['secret'] not in row.details
assert not [record for record in caplog.records
if record.levelno >= logging.WARNING]
infos = [record.getMessage() for record in caplog.records
if record.levelno == logging.INFO]
assert any('Download .ps1' in message and '10.0.0.7' in message
for message in infos)
assert all(data['secret'] not in message for message in infos)
def test_no_log_or_audit_row_ever_carries_the_token_value(
client, db, auth_headers, caplog, servercert):
secret = _secret(client, auth_headers, maxuses=4)['secret']
with caplog.at_level(logging.DEBUG):
client.get(SCRIPT_URL, query_string={'t': secret})
client.get(SCRIPT_URL, headers=_token_header(secret))
client.get(PACKAGE_URL, query_string={'t': secret})
client.get(PACKAGE_URL, headers=_token_header(secret))
assert all(secret not in record.getMessage()
for record in caplog.records)
rows = AuditLog.query.filter_by(
entitytype='EndpointBootstrapToken').all()
assert rows
for row in rows:
assert secret not in (row.details or '')
# The 12-char display prefix is the identifier the UI already shows.
assert row.entityname == secret[:12]
@pytest.mark.parametrize('url', [SCRIPT_URL, PACKAGE_URL])
def test_the_query_form_can_be_turned_off_once_a_site_has_re_issued(
client, db, auth_headers, url):
"""Off is the hardened end state, not the default: the UI download link
uses the query form too, so a site only turns it off deliberately."""
secret = _secret(client, auth_headers)['secret']
Setting.set(QUERY_SETTING, False, valuetype='boolean')
refused = client.get(url, query_string={'t': secret})
assert refused.status_code == 400
assert TOKEN_HEADER in refused.get_json()['data']['error']['message']
# The admin Download .ps1 link is a query token too: the setting is about
# a live token landing in an access log, and that link puts one there.
assert client.get(url, query_string={'t': secret, 'via': 'download'}
).status_code == 400
# Refused before any lookup, so it confirms nothing about the token.
assert EndpointBootstrapToken.query.one().usecount == 0
assert AuditLog.query.filter_by(
entitytype='EndpointBootstrapToken', action='requested').count() == 0
assert client.get(url, headers=_token_header(secret)).status_code == 200
@pytest.mark.parametrize('url', [SCRIPT_URL, PACKAGE_URL])
def test_no_token_is_still_not_found_when_the_query_form_is_off(
client, db, auth_headers, url):
"""A bare GET presented no token, so it is unusable and answers 404 like
every other unusable request. Calling it a query token answered 400 naming
the header, which handed an anonymous caller the state of the switch."""
_secret(client, auth_headers)
Setting.set(QUERY_SETTING, False, valuetype='boolean')
assert client.get(url).status_code == 404
# An empty ?t= presented nothing either.
assert client.get(url, query_string={'t': ''}).status_code == 404
# And it is the same answer with the query form still on.
Setting.set(QUERY_SETTING, True, valuetype='boolean')
assert client.get(url).status_code == 404
def test_script_fetch_does_not_spend_a_use(client, db, auth_headers):
secret = _secret(client, auth_headers, maxuses=1)['secret']
for _ in range(3):
@@ -421,3 +674,4 @@ def test_the_command_falls_back_to_the_request_when_no_site_url_is_set(
base='https://localhost')
assert data['scripturl'].startswith(
'https://localhost/api/endpoints/siteagent/bootstrap.ps1?t=')
assert data['scripturl'].endswith('&' + UI_LINK_QUERY)

2
tests/test_plugins/test_endpoints_credentials.py
View File

@@ -689,6 +689,7 @@ def test_enrollment_listing_and_settings(client, db, app, keyheaders,
body = response.get_json()
data = body['data']
assert data['settings'] == {'sharedkeyallowed': True, 'autoenroll': True,
'revokeddeniessharedkey': False,
'maxpendingenrollments': 500}
assert data['counts'] == {'enrolled': 1, 'notenrolled': 1,
'pendingcomputers': 1, 'pending': 1}
@@ -711,6 +712,7 @@ def test_enrollment_listing_and_settings(client, db, app, keyheaders,
assert put.status_code == 200
assert put.get_json()['data'] == {'sharedkeyallowed': False,
'autoenroll': True,
'revokeddeniessharedkey': False,
'maxpendingenrollments': 500}
bad = client.put('/api/endpoints/enrollment/settings',
json={'autoenroll': 'yes'}, headers=auth_headers)

352
tests/test_plugins/test_endpoints_enrollment_denials.py Normal file
View File

@@ -0,0 +1,352 @@
"""What a denial means at the enrollment door, and how long it lasts.
Three denials meet here, and each one used to be undone by the PC on the other
side of it:
reject An admin refuses a request. The agent then sends a new nonce and
the row was re-armed as a fresh cycle - which, with the shared key
and auto-enroll on, auto-issued the credential the admin had just
refused. A rejection binds to the identity (hostname + serial), so
it survives any nonce; only an admin approving it clears it.
cap The pending queue is bounded, and at the cap a PC nobody has seen
before is refused. Abandoned rows come out first, so a queue that
filled with junk drains itself instead of blocking every new PC.
revoke A revoked credential falls back to the shared key, which is how
the un-migrated fleet and the rollback to an older agent keep
reporting. plugin.endpoints.revokeddeniessharedkey (off by
default) turns a revoke into an eviction instead.
"""
from datetime import timedelta
import pytest
from shopdb.core.models import Asset, AssetType, Setting
from plugins.endpoints.api import agentauth
from plugins.endpoints.api.enrollment import (
PENDING_CAP_SETTING, PENDING_STALE_AFTER,
)
from plugins.endpoints.models import EndpointCredential, EndpointEnrollRequest
COLLECTOR_KEY = 'testcollectorkey'
ENROLL_URL = '/api/endpoints/enroll'
ALERTS_URL = '/api/endpoints/alerts'
NONCE = 'n' * 40
OTHER_NONCE = 'm' * 40
@pytest.fixture
def keyheaders(app):
app.config['COLLECTOR_API_KEY'] = COLLECTOR_KEY
return {'X-API-Key': COLLECTOR_KEY}
@pytest.fixture(autouse=True)
def nolimiter(app):
"""The per-IP limiter counts in a process-wide cache that outlives a test.
Nothing here is about how FAST a PC asks, and another test in the same
worker may already have spent the budget for this window and address, so the
limiter is off for this module and put back as it was.
"""
key = 'ENDPOINTS_ENROLL_RATELIMIT_ENABLED'
previous = app.config.get(key)
app.config[key] = False
yield
if previous is None:
app.config.pop(key, None)
else:
app.config[key] = previous
def _computer(db, hostname, serialnumber='SER-1'):
from plugins.computers.models import Computer
atype = AssetType.query.filter_by(assettype='computer').first()
if atype is None:
atype = AssetType(assettype='computer')
db.session.add(atype)
db.session.flush()
asset = Asset(assetnumber=hostname, assettypeid=atype.assettypeid,
serialnumber=serialnumber, isactive=True)
db.session.add(asset)
db.session.flush()
db.session.add(Computer(assetid=asset.assetid, hostname=hostname))
db.session.commit()
return asset.assetid
def _body(hostname='BAY-01', serialnumber='SER-1', nonce=NONCE):
return {'hostname': hostname, 'serialnumber': serialnumber,
'enrollnonce': nonce, 'agentversion': '1.20.0'}
def _enroll(client, headers, **kwargs):
return client.post(ENROLL_URL, json=_body(**kwargs), headers=headers)
def _pending_request(client, db, keyheaders, **kwargs):
"""A request waiting for an admin, with auto-enroll back on afterwards.
Auto-enroll off is what puts a clean match in the queue instead of issuing
at once; turning it back on is what made the old code hand the retry a
credential, so every rejection test needs both halves.
"""
_setting(db, 'plugin.endpoints.autoenroll', False)
response = _enroll(client, keyheaders, **kwargs)
assert response.status_code == 202, response.get_data(as_text=True)
_setting(db, 'plugin.endpoints.autoenroll', True)
return response.get_json()['data']['requestid']
def _reject(client, auth_headers, requestid):
return client.post(
f'/api/endpoints/enrollment/requests/{requestid}/reject',
headers=auth_headers)
def _setting(db, key, value, valuetype='boolean'):
Setting.set(key, value, valuetype=valuetype, category='plugin.endpoints')
db.session.commit()
def _code(response):
return response.get_json()['data']['error']['code']
# -- a rejection binds to the identity ---------------------------------------
def test_a_new_nonce_does_not_lift_a_rejection(client, db, app, keyheaders,
auth_headers):
"""The bug: reinstall the agent and the retry was auto-issued the
credential an admin had just refused."""
_computer(db, 'BAY-01')
requestid = _pending_request(client, db, keyheaders)
assert _reject(client, auth_headers, requestid).status_code == 200
retry = _enroll(client, keyheaders, nonce=OTHER_NONCE)
assert retry.status_code == 202, retry.get_data(as_text=True)
assert retry.get_json()['data']['reason'] == 'rejected'
assert retry.get_json()['data']['requestid'] == requestid
assert EndpointCredential.query.count() == 0
row = db.session.get(EndpointEnrollRequest, requestid)
assert row.status == EndpointEnrollRequest.STATUS_REJECTED
assert row.decidedat is not None
# Still one row for this identity: the retry did not queue a second.
assert EndpointEnrollRequest.query.count() == 1
def test_a_rejection_survives_however_often_the_pc_asks(client, db, app,
keyheaders,
auth_headers):
_computer(db, 'BAY-01')
requestid = _pending_request(client, db, keyheaders)
_reject(client, auth_headers, requestid)
for index in range(5):
again = _enroll(client, keyheaders, nonce=('x%d' % index) * 20)
assert again.status_code == 202
assert again.get_json()['data']['reason'] == 'rejected'
assert EndpointCredential.query.count() == 0
assert EndpointEnrollRequest.query.count() == 1
def test_a_rejection_does_not_touch_a_different_pc(client, db, app,
keyheaders, auth_headers):
"""Identity is hostname AND serial, so one PC's rejection is its own."""
_computer(db, 'BAY-01', serialnumber='SER-1')
other = _computer(db, 'BAY-02', serialnumber='SER-2')
requestid = _pending_request(client, db, keyheaders)
_reject(client, auth_headers, requestid)
clean = _enroll(client, keyheaders, hostname='BAY-02',
serialnumber='SER-2')
assert clean.status_code == 201, clean.get_data(as_text=True)
assert EndpointCredential.query.filter_by(assetid=other).one() \
.revokedat is None
def test_an_admin_clears_a_rejection_by_approving_it(client, db, app,
keyheaders, auth_headers):
"""The way back is the admin's, and it works for the reinstalled agent that
now carries a different nonce."""
assetid = _computer(db, 'BAY-01')
requestid = _pending_request(client, db, keyheaders)
_reject(client, auth_headers, requestid)
_enroll(client, keyheaders, nonce=OTHER_NONCE)
approve = client.post(
f'/api/endpoints/enrollment/requests/{requestid}/approve',
json={'assetid': assetid}, headers=auth_headers)
assert approve.status_code == 200, approve.get_data(as_text=True)
issued = _enroll(client, keyheaders, nonce=OTHER_NONCE)
assert issued.status_code == 201, issued.get_data(as_text=True)
assert EndpointCredential.query.filter_by(assetid=assetid).one() \
.enrolledvia == 'approved'
def test_an_approval_out_of_a_rejection_still_binds_to_one_installation(
client, db, app, keyheaders, auth_headers):
"""It binds to the installation the PC was last seen with, so a third
machine claiming the same hostname and serial still cannot collect it."""
assetid = _computer(db, 'BAY-01')
requestid = _pending_request(client, db, keyheaders)
_reject(client, auth_headers, requestid)
_enroll(client, keyheaders, nonce=OTHER_NONCE)
client.post(f'/api/endpoints/enrollment/requests/{requestid}/approve',
json={'assetid': assetid}, headers=auth_headers)
impostor = _enroll(client, keyheaders, nonce='z' * 40)
assert impostor.status_code == 403
assert _code(impostor) == 'ENROLL_NONCE_MISMATCH'
assert EndpointCredential.query.count() == 0
issued = _enroll(client, keyheaders, nonce=OTHER_NONCE)
assert issued.status_code == 201, issued.get_data(as_text=True)
# -- the pending cap ages out abandoned rows ---------------------------------
def _stale(db, hostname, days):
from datetime import datetime, timezone
seen = (datetime.now(timezone.utc).replace(tzinfo=None)
- timedelta(days=days))
row = EndpointEnrollRequest(
hostname=hostname, serialnumber=hostname + '-SER',
enrollnonce=NONCE, status=EndpointEnrollRequest.STATUS_PENDING,
requestedat=seen, lastseenat=seen)
db.session.add(row)
db.session.commit()
return row
def test_the_cap_ages_out_abandoned_rows_before_refusing(client, db, app):
"""A queue full of PCs that stopped asking must not block the one that is."""
_setting(db, PENDING_CAP_SETTING, 2, valuetype='integer')
_stale(db, 'GONE-01', PENDING_STALE_AFTER.days + 1)
_stale(db, 'GONE-02', PENDING_STALE_AFTER.days + 3)
response = client.post(ENROLL_URL, json=_body(hostname='NEW-01',
serialnumber='NEW-SER'))
assert response.status_code == 202, response.get_data(as_text=True)
hostnames = {row.hostname for row in EndpointEnrollRequest.query.all()}
assert hostnames == {'NEW-01'}
def test_the_age_out_spares_rows_still_being_asked_for(client, db, app):
"""Only rows nothing has touched go, and a decided row never goes: the
rejection above has to outlive any queue pressure."""
_setting(db, PENDING_CAP_SETTING, 2, valuetype='integer')
fresh = _stale(db, 'LIVE-01', 0)
stillasking = _stale(db, 'LIVE-02', 0)
decided = _stale(db, 'GONE-01', PENDING_STALE_AFTER.days + 1)
decided.status = EndpointEnrollRequest.STATUS_REJECTED
db.session.commit()
refused = client.post(ENROLL_URL, json=_body(hostname='NEW-01',
serialnumber='NEW-SER'))
assert refused.status_code == 503
assert _code(refused) == 'ENROLL_PENDING_FULL'
for row in (fresh, stillasking, decided):
assert db.session.get(EndpointEnrollRequest, row.requestid) is not None
# -- revocation and the shared key -------------------------------------------
def test_a_revoked_pc_keeps_the_shared_key_by_default(client, db, app,
keyheaders,
auth_headers):
"""Today's behaviour, and the setting default: a revoke is a rotation, not
an eviction, so the PC keeps reporting while it waits to enroll again."""
assetid = _computer(db, 'BAY-01')
assert _enroll(client, keyheaders).status_code == 201
client.post(f'/api/endpoints/enrollment/assets/{assetid}/revoke',
headers=auth_headers)
assert client.get(ALERTS_URL + '?hostname=BAY-01',
headers=keyheaders).status_code == 200
def test_revoked_denies_shared_key_evicts_the_pc(client, db, app, keyheaders,
auth_headers):
assetid = _computer(db, 'BAY-01')
assert _enroll(client, keyheaders).status_code == 201
client.post(f'/api/endpoints/enrollment/assets/{assetid}/revoke',
json={'blockreenroll': True, 'reason': 'stolen'},
headers=auth_headers)
_setting(db, agentauth.REVOKED_DENIES_SHARED_KEY_SETTING, True)
refused = client.get(ALERTS_URL + '?hostname=BAY-01', headers=keyheaders)
assert refused.status_code == 401
assert _code(refused) == 'SHARED_KEY_REFUSED'
assert 'revoked' in refused.get_json()['data']['error']['message']
def test_a_pc_that_never_enrolled_is_never_evicted(client, db, app,
keyheaders, auth_headers):
"""The un-migrated fleet has no credential row at all, so the setting can be
turned on without silencing it."""
_computer(db, 'BAY-01', serialnumber='SER-1')
_computer(db, 'BAY-02', serialnumber='SER-2')
assetid = _computer(db, 'BAY-03', serialnumber='SER-3')
_enroll(client, keyheaders, hostname='BAY-03', serialnumber='SER-3')
client.post(f'/api/endpoints/enrollment/assets/{assetid}/revoke',
headers=auth_headers)
_setting(db, agentauth.REVOKED_DENIES_SHARED_KEY_SETTING, True)
assert client.get(ALERTS_URL + '?hostname=BAY-01',
headers=keyheaders).status_code == 200
assert client.get(ALERTS_URL + '?hostname=BAY-02',
headers=keyheaders).status_code == 200
assert client.get(ALERTS_URL + '?hostname=BAY-03',
headers=keyheaders).status_code == 401
def test_the_collector_contract_follows_the_same_rule(client, db, app,
keyheaders,
auth_headers):
"""The generic collector path asks the plugin, not bind_hostname, so it has
to answer the same way."""
from plugins.endpoints.plugin import EndpointsPlugin
assetid = _computer(db, 'BAY-01')
assert _enroll(client, keyheaders).status_code == 201
client.post(f'/api/endpoints/enrollment/assets/{assetid}/revoke',
headers=auth_headers)
refused = EndpointsPlugin().agent_shared_key_refused
with app.test_request_context():
assert refused('BAY-01') is False
_setting(db, agentauth.REVOKED_DENIES_SHARED_KEY_SETTING, True)
assert refused('BAY-01') is True
assert refused('UNKNOWN') is False
def test_the_eviction_switch_is_editable_and_defaults_to_today(client, db,
auth_headers):
"""A setting nobody can change is a constant with extra steps."""
read = client.get('/api/endpoints/enrollment',
headers=auth_headers).get_json()['data']
assert read['settings']['revokeddeniessharedkey'] is False
put = client.put('/api/endpoints/enrollment/settings',
json={'revokeddeniessharedkey': True},
headers=auth_headers)
assert put.status_code == 200, put.get_data(as_text=True)
assert put.get_json()['data']['revokeddeniessharedkey'] is True
assert Setting.get(agentauth.REVOKED_DENIES_SHARED_KEY_SETTING) is True
bad = client.put('/api/endpoints/enrollment/settings',
json={'revokeddeniessharedkey': 'yes'},
headers=auth_headers)
assert bad.status_code == 400

162
tests/test_plugins/test_endpoints_settings_declared.py Normal file
View File

@@ -0,0 +1,162 @@
"""The two endpoints switches are declared, seeded, and have a control.
Both were readable by their enforcement code before they were declared:
plugin.endpoints.revokeddeniessharedkey agentauth.revoked_denies_shared_key
plugin.endpoints.bootstrapquerytoken bootstrap._query_token_allowed
An undeclared key has no row until somebody writes one, and until it was
declared the settings API upserted it with valuetype 'string' - so
bool(Setting.get(key, True)) read the text 'false' as True and the switch did
nothing. Declaring it fixes both halves: the row is seeded at plugin
install/enable, and a PUT lands as a real boolean.
The UI half is checked by reading the pages. A switch with no control on an
air-gapped box can only be changed with SQL, which is not a setting.
"""
from pathlib import Path
import pytest
from plugins.endpoints.api import agentauth, bootstrap
from plugins.endpoints.plugin import EndpointsPlugin
from shopdb.core.models import Setting
REPO = Path(__file__).resolve().parents[2]
VIEWS = REPO / 'plugins' / 'endpoints' / 'frontend' / 'views'
REVOKED_KEY = agentauth.REVOKED_DENIES_SHARED_KEY_SETTING
QUERY_KEY = bootstrap.QUERY_TOKEN_SETTING
def _declared():
return {entry['key']: entry for entry in
EndpointsPlugin().get_settings_defaults()}
@pytest.mark.parametrize('key,value', [(REVOKED_KEY, 'false'),
(QUERY_KEY, 'true')])
def test_the_switch_is_declared_with_todays_behaviour(key, value):
entry = _declared().get(key)
assert entry is not None, key + ' is read but never declared'
assert entry['valuetype'] == 'boolean'
assert entry['category'] == 'plugin.endpoints'
# The declared default must be what the code does with no row at all, or
# seeding a site silently changes its behaviour.
assert entry['value'] == value
@pytest.mark.parametrize('key', [REVOKED_KEY, QUERY_KEY])
def test_the_description_names_what_turning_it_on_costs(key):
"""An operator on the shop floor decides from this text alone."""
description = _declared()[key]['description']
assert len(description) > 120, description
if key == REVOKED_KEY:
# The revoke + blockreenroll rollback onto an older agent stops working.
assert 'rollback' in description
assert 'enrolls again' in description
else:
assert 're-issued' in description or 're-issue' in description
def test_the_declared_defaults_match_what_the_code_reads_with_no_row(db):
"""No Setting rows at all: the readers fall back to these same values."""
assert Setting.query.filter_by(key=REVOKED_KEY).first() is None
assert Setting.query.filter_by(key=QUERY_KEY).first() is None
assert agentauth.revoked_denies_shared_key() is False
assert bootstrap._query_token_allowed() is True
def test_the_upgrade_pass_seeds_them_on_a_site_that_installed_earlier(app, db):
"""install/enable fire only on a state transition, so a fleet already
running this plugin gets these rows from `flask plugin upgrade-all`, which
calls sync_all_plugin_settings -> _seed_plugin_settings.
This drives _seed_plugin_settings on a manager built for this test rather
than the process-wide plugin_manager singleton, and that is deliberate.
create_app() re-points that singleton, eight test modules build their own
app, and whatever ran last on this xdist worker owns it: the symptoms are
"no such table: settings" (seeding against another app's in-memory SQLite)
and "'NoneType' object has no attribute 'get_all'" (registry left None).
Both land on whichever test follows, so they move with the split - this one
passed locally and failed on CI for exactly that reason.
init_app is not called here either: it re-registers the plugin blueprints,
and twice on one app raises "The name 'backups' is already registered".
Setting _app and _db is all the seeding routine reads.
"""
from shopdb.plugins import PluginManager
from plugins.endpoints.plugin import EndpointsPlugin
assert Setting.query.filter_by(key=QUERY_KEY).first() is None
manager = PluginManager()
manager._app = app
manager._db = db
assert manager._seed_plugin_settings(EndpointsPlugin()) is True
for key, value in ((REVOKED_KEY, False), (QUERY_KEY, True)):
row = Setting.query.filter_by(key=key).one()
assert row.valuetype == 'boolean'
assert row.category == 'plugin.endpoints'
assert row.get_typed_value() is value
def test_a_put_lands_as_a_boolean_and_flips_the_query_token(client, db,
auth_headers):
"""End to end for bootstrapquerytoken through the API the page uses.
The declaration is what makes the upsert boolean; as a string row the read
below would stay True whatever was saved.
"""
saved = client.put('/api/settings/' + QUERY_KEY, json={'value': False},
headers=auth_headers)
assert saved.status_code == 200, saved.get_json()
body = saved.get_json()['data']
assert body['value'] is False
assert body['valuetype'] == 'boolean'
assert body['category'] == 'plugin.endpoints'
assert bootstrap._query_token_allowed() is False
back = client.put('/api/settings/' + QUERY_KEY, json={'value': True},
headers=auth_headers)
assert back.get_json()['data']['value'] is True
assert bootstrap._query_token_allowed() is True
def test_the_settings_api_lists_the_declared_switch(client, db, auth_headers):
"""The agent-key page reads GET /api/settings/<key> to render the box."""
client.put('/api/settings/' + QUERY_KEY, json={'value': True},
headers=auth_headers)
resp = client.get('/api/settings/' + QUERY_KEY, headers=auth_headers)
assert resp.status_code == 200, resp.get_json()
assert resp.get_json()['data']['value'] is True
def test_the_enrollment_page_carries_the_eviction_checkbox():
page = (VIEWS / 'EnrollmentSettings.vue').read_text(encoding='utf-8')
assert 'toggle-revokeddenies' in page, 'no checkbox for the eviction switch'
assert 'onRevokedDeniesToggle' in page
# It sits with the two switches it belongs beside.
assert 'toggle-sharedkey' in page and 'toggle-autoenroll' in page
# The operator has to be told what it breaks before clicking it.
assert 'rollback' in page
assert 'enrolls again' in page
def test_the_agent_key_page_carries_the_query_token_control():
page = (VIEWS / 'AgentKeySettings.vue').read_text(encoding='utf-8')
assert QUERY_KEY in page, 'the page does not touch the setting'
assert 'toggle-querytoken' in page
assert 'onQueryTokenToggle' in page
# Turning it off breaks commands already handed out, so say so.
assert 're-issue' in page

56
tests/test_plugins/test_endpoints_siteagent_build.py
View File

@@ -289,6 +289,62 @@ def test_leaf_certificate_is_imported_into_root(client, db, auth_headers):
assert json.loads(files['appsettings.json'])['serverpins'] == [leafpin]
def test_installer_restricts_the_registry_key_to_system_and_admins(
client, db, auth_headers, servercert):
"""The shared key stays in HKLM until every consumer is on a per-PC
credential, so what matters meanwhile is who can read it. The key inherits a
DACL that grants BUILTIN\\Users read, which is the exposure the per-PC
migration exists to close - the installer must not leave it that way."""
files = _files(_build(client, auth_headers, collectorkey=SECRET,
includesharedkey='true'))
script = files['install.ps1'].decode('ascii')
assert 'function Protect-HklmShopdbKey' in script
assert 'Protect-HklmShopdbKey -SubKey "SOFTWARE\\GE\\ShopDB"' in script
# SYSTEM and Administrators by SID, so a localised Windows still matches.
assert '"S-1-5-18", "S-1-5-32-544"' in script
# Inheritance broken WITHOUT copying the inherited entries: copying them is
# what silently preserves the Users read.
assert 'SetAccessRuleProtection($true, $false)' in script
assert 'RemoveAccessRule($rule)' in script
def test_the_key_is_written_before_it_is_locked(client, db, auth_headers, servercert):
"""SetAccessControl replaces the DACL rather than merging, so a lock that ran
first would be undone by nothing - but a write that ran after a lock the
installer no longer holds rights for would fail. Order is the guarantee."""
script = _files(_build(client, auth_headers, collectorkey=SECRET,
includesharedkey='true'))['install.ps1'].decode('ascii')
write = script.index('-Name "CollectorKey" -Value $CollectorKey')
lock = script.index('Protect-HklmShopdbKey -SubKey')
assert write < lock
def test_the_registry_lock_failing_does_not_fail_the_install(
client, db, auth_headers, servercert):
"""A bay that cannot take the ACL still needs its agent. It must say plainly
that the key is readable there, rather than dying or staying quiet."""
script = _files(_build(client, auth_headers, collectorkey=SECRET,
includesharedkey='true'))['install.ps1'].decode('ascii')
lock = script.index('Protect-HklmShopdbKey -SubKey')
tail = script[lock:lock + 600]
assert 'catch' in tail
assert 'readable by any logged-on user' in tail
def test_the_keyless_package_still_locks_the_key(client, db, auth_headers, servercert):
"""ServerPins live in the same key and the registry is authoritative for
pinning, so a standard user must not be able to rewrite them either."""
files = _files(_build(client, auth_headers, collectorkey=SECRET,
includesharedkey=False))
script = files['install.ps1'].decode('ascii')
assert b'CollectorKey' not in files['install.ps1']
assert 'Protect-HklmShopdbKey -SubKey "SOFTWARE\\GE\\ShopDB"' in script
def test_install_script_helper_without_pins_or_key():
script = siteagent._install_script('https://shopdb.test', '', 900,
serverpins=[], includesharedkey=False)

212
tests/test_plugins/test_kiosk_network_allowlist.py Normal file
View File

@@ -0,0 +1,212 @@
"""Network-position admission for the two unauthenticated kiosks.
The device token covers unattended screens an administrator provisions once. It
does not cover the other caller: somebody opening the kiosk page on their own
PC, who never logs in and whose browser nobody provisioned. Per-browser
provisioning would mean touching every PC on the floor and losing it on the next
reimage, so those callers are admitted by where they are on the network instead.
What these tests hold down:
- the allowlist actually admits a caller with no token (the feature),
- it stays shut when unset, when the caller is outside it, and when the kiosk
itself is off (the flag still wins),
- a malformed entry is skipped rather than widening the match, and
- the two kiosks do not share an allowlist.
"""
import pytest
from shopdb.core.models import Setting
from shopdb.api import ip_in_cidrs
PLANT = '10.134.48.17'
PLANT_CIDR = '10.134.48.0/23'
ELSEWHERE = '10.9.100.5'
# -- the pure matcher ---------------------------------------------------------
# Separated from the request so the fail-closed edges can be stated plainly.
@pytest.mark.parametrize('address, cidrs, expected', [
(PLANT, PLANT_CIDR, True),
(PLANT, ' 10.48.249.0/26 , 10.134.48.0/23 ', True),
(ELSEWHERE, PLANT_CIDR, False),
# Empty or absent setting means the feature is off, not "allow everyone".
(PLANT, '', False),
(PLANT, None, False),
# A malformed entry must be skipped, never treated as a wildcard.
(PLANT, 'not-a-cidr', False),
(PLANT, 'not-a-cidr, 10.134.48.0/23', True),
# An unparseable caller address matches nothing.
('', PLANT_CIDR, False),
('unknown', PLANT_CIDR, False),
])
def test_ip_in_cidrs(address, cidrs, expected):
assert ip_in_cidrs(address, cidrs) is expected
# -- USB kiosk ----------------------------------------------------------------
@pytest.fixture
def usbkiosk(db):
"""Self-hosted USB, kiosk ON, a token set, one employee and one drive."""
from plugins.employees.models import DirectoryEmployee
from plugins.usb.models import USBDevice
db.session.add(Setting(key='usb_directory_mode', value='selfhosted',
valuetype='string', category='site'))
db.session.add(Setting(key='usb_kiosk_enabled', value='true',
valuetype='boolean', category='usb'))
db.session.add(Setting(key='usb_kiosk_token', value='a-real-token',
valuetype='string', category='usb'))
db.session.add(DirectoryEmployee(sso=270015376, firstname='Dana',
lastname='Reyes'))
db.session.add(USBDevice(serialnumber='USB-1', label='Blue Kingston',
ischeckedout=False))
db.session.commit()
return db
def _allow(db, key, value):
row = Setting.query.filter_by(key=key).first()
if row is None:
db.session.add(Setting(key=key, value=value, valuetype='string',
category='usb'))
else:
row.value = value
db.session.commit()
def _session(client, address):
"""A kiosk call carrying NO token, from `address`."""
return client.post('/api/usb/kiosk/session', json={'badge': '270015376'},
environ_base={'REMOTE_ADDR': address})
def test_usb_kiosk_refuses_tokenless_caller_when_no_allowlist(usbkiosk, client):
"""The state before this feature: no token, no way in."""
assert _session(client, PLANT).status_code == 404
def test_usb_kiosk_admits_tokenless_caller_on_allowed_network(usbkiosk, client):
"""THE FEATURE. Fails without the allowlist branch in _kiosk_open."""
_allow(usbkiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
assert _session(client, PLANT).status_code == 200
def test_usb_kiosk_refuses_caller_outside_the_allowlist(usbkiosk, client):
"""The PXE LAN is also 10.x, which is why the range has to be narrow."""
_allow(usbkiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
assert _session(client, ELSEWHERE).status_code == 404
def test_usb_kiosk_allowlist_does_not_override_the_off_switch(usbkiosk, client):
"""Turning the kiosk off closes it to everyone, allowlisted or not.
The flag is the site's decision to have this door at all; the allowlist only
decides who walks through an open one.
"""
_allow(usbkiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
Setting.query.filter_by(key='usb_kiosk_enabled').first().value = 'false'
usbkiosk.session.commit()
assert _session(client, PLANT).status_code == 404
def test_usb_kiosk_token_still_works_from_any_network(usbkiosk, client):
"""Wall screens keep working, and are not confined to the allowlist."""
response = client.post('/api/usb/kiosk/session',
json={'badge': '270015376'},
headers={'X-Kiosk-Token': 'a-real-token'},
environ_base={'REMOTE_ADDR': ELSEWHERE})
assert response.status_code == 200
def test_usb_allowlist_does_not_open_the_parts_kiosk(usbkiosk, client):
"""Separate screens, separate rooms, separate settings.
The tokens are deliberately separate so provisioning one does not grant the
other. An allowlist shared between them would hand back exactly that.
"""
_allow(usbkiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
Setting.query.filter_by(key='usb_kiosk_enabled').first().value = 'true'
usbkiosk.session.add(Setting(key='printedparts_kiosk_enabled', value='true',
valuetype='boolean', category='printedparts'))
usbkiosk.session.commit()
response = client.get('/api/printedparts/kiosk/lookup?code=ANY',
environ_base={'REMOTE_ADDR': PLANT})
assert response.status_code == 404
# -- what the page probes on load ---------------------------------------------
def test_empty_badge_probe_separates_refused_from_admitted(usbkiosk, client):
"""The kiosk page asks the server on load whether it may be used.
It cannot guess: the enabled flag is public and only says the site HAS a
kiosk, and a caller admitted by network carries no token to look for. So the
page sends a badge-less session request and reads the status.
That only works while the gate runs BEFORE badge handling - refused is 404,
admitted is the badge validation error. If someone reorders those, the page
starts telling admitted screens they are not set up, so hold it here.
"""
refused = client.post('/api/usb/kiosk/session', json={'badge': ''},
environ_base={'REMOTE_ADDR': ELSEWHERE})
assert refused.status_code == 404
_allow(usbkiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
admitted = client.post('/api/usb/kiosk/session', json={'badge': ''},
environ_base={'REMOTE_ADDR': PLANT})
assert admitted.status_code != 404
assert admitted.status_code < 500
def test_the_probe_records_nothing(usbkiosk, client):
"""A page load must not leave a checkout, a log row, or a resolved badge."""
from plugins.usb.models import USBCheckout
_allow(usbkiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
client.post('/api/usb/kiosk/session', json={'badge': ''},
environ_base={'REMOTE_ADDR': PLANT})
assert USBCheckout.query.count() == 0
# -- provisioning link --------------------------------------------------------
# The settings API masks any key named '*token*', so the settings page shows
# '********'. An administrator who copies that provisions a screen with the mask
# and gets a silent 404 - the failure that cost real time on 2026-09-16. These
# hold the way out: a named, permission-gated route that hands a working setup
# link to somebody already allowed to set the token.
def test_provisioning_link_does_not_hand_back_the_mask(usbkiosk, client,
auth_headers):
"""It used to answer '/usb-kiosk?token=<value>' and this test pinned that
literal string. The link has since become a single-use setup code, which is
a better answer to the same problem - a query string ends up in history,
referrers and proxy logs - so what is held here is the property rather than
the shape: a usable link, and never the mask."""
response = client.get('/api/usb/kiosk/provisioning-link',
headers=auth_headers)
assert response.status_code == 200
url = response.get_json()['data']['url']
assert '*' not in url
# The token itself must not ride in the URL either.
assert 'a-real-token' not in url
assert client.get(url).status_code == 302
def test_provisioning_link_needs_permission(usbkiosk, client):
"""Anonymous callers cannot read the token out of this route."""
response = client.get('/api/usb/kiosk/provisioning-link')
assert response.status_code in (401, 403)
def test_provisioning_link_says_so_when_no_token_is_set(usbkiosk, client,
auth_headers):
Setting.query.filter_by(key='usb_kiosk_token').first().value = ''
usbkiosk.session.commit()
response = client.get('/api/usb/kiosk/provisioning-link',
headers=auth_headers)
assert response.status_code == 400
assert 'Generate' in response.get_json()['data']['error']['message']

77
tests/test_plugins/test_manifest_core_version_floor.py Normal file
View File

@@ -0,0 +1,77 @@
"""usb and printedparts declare a core_version floor they actually need.
Both reach for the network-position allowlist that arrived on the contract
surface in 0.35.0 (commit "Admit kiosk callers by network position"): ip_in_cidrs
and request_ip_allowlisted landed together in that bump, so either one witnesses
the same floor. Their manifests claimed >=0.1.0 and >=0.16.0, so the loader would
have accepted them on a core whose shopdb.api has no such name - the plugin then
fails at first import of its routes instead of at the version check that exists
to catch exactly this.
The witness is named PER PLUGIN and not shared. usb needs the matcher itself,
because its admission is the allowlist minus the site's proxies and the
addresses no caller can have, which the one-call helper does not express;
printedparts is the plain case and uses ip_in_cidrs. A single shared
name went red the moment usb moved, which said "the floor is wrong" when the
floor was right.
"""
import json
import pathlib
import pytest
from packaging.specifiers import SpecifierSet
from packaging.version import Version
from shopdb import __contract_version__
PLUGINS = pathlib.Path(__file__).resolve().parents[2] / 'plugins'
# The contract version that added request_ip_allowlisted / ip_in_cidrs.
IP_ALLOWLIST_CONTRACT = '0.35.0'
# plugin, the 0.35.0 name it really imports, the floor that name implies
CASES = [
('usb', 'ip_in_cidrs', IP_ALLOWLIST_CONTRACT),
# Was request_ip_allowlisted until 2026-09-18, when the parts kiosk moved to
# client_ip + ip_in_cidrs so it could SUBTRACT proxy and loopback addresses
# before consulting the allowlist. Same contract release, same floor.
('printedparts', 'ip_in_cidrs', IP_ALLOWLIST_CONTRACT),
]
def _manifest(name):
return json.loads((PLUGINS / name / 'manifest.json').read_text())
@pytest.mark.parametrize('name,witness,needed', CASES)
def test_manifest_excludes_a_core_without_the_name(name, witness, needed):
"""A core one minor below the name's arrival must be refused."""
spec = SpecifierSet(_manifest(name)['core_version'])
too_old = Version(needed)
too_old = Version(f'{too_old.major}.{too_old.minor - 1}.0')
assert too_old not in spec
@pytest.mark.parametrize('name,witness,needed', CASES)
def test_manifest_admits_the_running_core(name, witness, needed):
"""And the floor must not have overshot what this core provides."""
spec = SpecifierSet(_manifest(name)['core_version'])
assert Version(__contract_version__) in spec
@pytest.mark.parametrize('name,witness,needed', CASES)
def test_plugin_really_imports_the_name(name, witness, needed):
"""Pins the floor to the reason for it: drop the import and this test says
the floor can come back down."""
sources = (PLUGINS / name).rglob('*.py')
assert any(witness in path.read_text() for path in sources), name
@pytest.mark.parametrize('name,witness,needed', CASES)
def test_the_name_is_on_the_contract_surface(name, witness, needed):
"""Each witness is a contract name, not an internal import a plugin reached
through - which is the only reason the manifest floor can speak for it."""
import shopdb.api
assert witness in shopdb.api.__all__

93
tests/test_plugins/test_notifications_alertscope_storage.py Normal file
View File

@@ -0,0 +1,93 @@
"""alertscope is TEXT in the database, a dict in Python.
Migration 0006 ships the column as TEXT because a site still runs MySQL 5.6,
which has no JSON type. The model declared db.JSON, so create-all built a column
the migration path never produces and the two disagreed about the same column.
"""
import json
from sqlalchemy import Text, inspect
from plugins.notifications.models.notification import Notification
def test_column_type_matches_the_migration(app):
"""The model's storage type must be the one migration 0006 creates."""
column = inspect(Notification).columns['alertscope']
assert isinstance(column.type.impl_instance
if hasattr(column.type, 'impl_instance')
else column.type.impl, Text)
def test_ddl_has_no_json_column(app, db):
"""MySQL 5.6 fails on a JSON column with a bare syntax error, so the
emitted DDL must not contain one."""
from sqlalchemy.schema import CreateTable
ddl = str(CreateTable(Notification.__table__).compile(db.engine))
alertscope_line = [line for line in ddl.splitlines()
if 'alertscope' in line][0]
assert 'JSON' not in alertscope_line.upper()
def test_dict_round_trips(app, db):
"""A scope assigned as a dict reads back as a dict."""
row = Notification(notification='fleet alert', alertfleet=True,
alerttimeout=0,
alertscope={'hostname': ['BAY-01', 'BAY-02']})
db.session.add(row)
db.session.commit()
notificationid = row.notificationid
db.session.expire_all()
reread = db.session.get(Notification, notificationid)
assert reread.alertscope == {'hostname': ['BAY-01', 'BAY-02']}
def test_stored_value_is_json_text(app, db):
"""What lands in the column is a JSON string, readable by a site that
queries the table directly."""
row = Notification(notification='fleet alert', alertfleet=True,
alerttimeout=0, alertscope={'computertype': 'CMM PC'})
db.session.add(row)
db.session.commit()
raw = db.session.execute(
db.text('SELECT alertscope FROM notifications WHERE notificationid = :i'),
{'i': row.notificationid}).scalar()
assert isinstance(raw, str)
assert json.loads(raw) == {'computertype': 'CMM PC'}
def test_null_scope_stays_null(app, db):
"""No scope means the whole fleet; it must not become the string 'null'."""
row = Notification(notification='fleet alert', alertfleet=True,
alerttimeout=0, alertscope=None)
db.session.add(row)
db.session.commit()
raw = db.session.execute(
db.text('SELECT alertscope FROM notifications WHERE notificationid = :i'),
{'i': row.notificationid}).scalar()
assert raw is None
assert row.alertscope is None
def test_unparseable_text_is_handed_back_raw(app, db):
"""A hand-edited row must be visible and fixable, not an exception on read.
The endpoints matcher already treats a non-dict scope as matching nothing.
"""
row = Notification(notification='fleet alert', alertfleet=True,
alerttimeout=0)
db.session.add(row)
db.session.commit()
db.session.execute(
db.text('UPDATE notifications SET alertscope = :v '
'WHERE notificationid = :i'),
{'v': 'not json at all', 'i': row.notificationid})
db.session.commit()
db.session.expire_all()
reread = db.session.get(Notification, row.notificationid)
assert reread.alertscope == 'not json at all'

92
tests/test_plugins/test_printedparts_kiosk_admission.py Normal file
View File

@@ -0,0 +1,92 @@
"""How the parts kiosk decides a caller is admitted by network position.
The USB kiosk closed this on 2026-09-18 and this one did not, so the two read
differently for no reason anybody chose. The hole is the same one: a reverse
proxy normally sits ON the network a site enters in the allowlist, so a request
wearing the proxy's own address matched the range and was admitted while
carrying no evidence at all about who sent it.
Shape of the requests here matches production: the layer in front (IIS with the
X-Forwarded-For rewrite, or nginx plus ProxyFix) has already resolved the caller
into REMOTE_ADDR, and the app never reads the header itself.
"""
import pytest
from shopdb.core.models import Setting
PLANT = '10.134.48.17'
PLANT_CIDR = '10.134.48.0/23'
ELSEWHERE = '10.9.100.5'
# Inside the allowlisted range, which is where a proxy actually lives and the
# reason the un-subtracted check was wrong.
PROXY = '10.134.48.5'
PROXY_CIDR = '10.134.48.5/32'
KIOSK_TOKEN = 'parts-kiosk-admission-token'
def _set(key, value):
Setting.set(key, value, valuetype='string', category='printedparts')
@pytest.fixture
def kiosk(app, db):
_set('printedparts_kiosk_enabled', 'true')
_set('printedparts_kiosk_token', KIOSK_TOKEN)
_set('printedparts_kiosk_allowed_cidrs', '')
_set('printedparts_kiosk_trusted_proxies', '')
return app
def _open_from(app, address):
"""Ask the gate directly: it is the unit under test, not a route's shape.
The request context comes from the REAL app, so Setting.get reaches the same
database the fixture wrote to.
"""
from plugins.printedparts.api.routes import _kiosk_open
with app.test_request_context('/', environ_base={'REMOTE_ADDR': address}):
return _kiosk_open()
def test_a_caller_on_an_allowed_network_is_admitted(kiosk):
_set('printedparts_kiosk_allowed_cidrs', PLANT_CIDR)
assert _open_from(kiosk, PLANT) is True
def test_a_caller_outside_every_allowed_network_is_refused(kiosk):
_set('printedparts_kiosk_allowed_cidrs', PLANT_CIDR)
assert _open_from(kiosk, ELSEWHERE) is False
def test_a_request_wearing_the_proxy_address_fails_closed(kiosk):
"""THE FIX. The proxy forwarded nothing, so its own address arrived - and it
sits inside the allowlist. No evidence about the sender means refuse."""
_set('printedparts_kiosk_allowed_cidrs', PLANT_CIDR)
_set('printedparts_kiosk_trusted_proxies', PROXY_CIDR)
assert _open_from(kiosk, PROXY) is False
# ... and the feature still works for a real caller on the same network, or
# the subtraction has taken the allowlist with it.
assert _open_from(kiosk, PLANT) is True
def test_loopback_is_never_a_kiosk_client(kiosk):
"""Needs no configuration: no site's kiosk is the server itself. This is
also the shape a misconfigured front end produces, where every request
arrives as 127.0.0.1."""
_set('printedparts_kiosk_allowed_cidrs', '127.0.0.0/8, ' + PLANT_CIDR)
assert _open_from(kiosk, '127.0.0.1') is False
def test_the_kiosk_flag_still_governs(kiosk):
"""Network position admits nobody while the kiosk is off."""
_set('printedparts_kiosk_allowed_cidrs', PLANT_CIDR)
_set('printedparts_kiosk_enabled', 'false')
assert _open_from(kiosk, PLANT) is False
def test_an_empty_allowlist_admits_nobody(kiosk):
"""Token-only, which is the shipped default."""
assert _open_from(kiosk, PLANT) is False

499
tests/test_plugins/test_usb_kiosk_admission.py Normal file
View File

@@ -0,0 +1,499 @@
"""How the USB kiosk decides a caller is admitted, and what that costs.
The kiosk writes without a login, so everything here is about the two things
standing in for one: where the caller is on the network, and a device token.
Three properties this file holds down, each of which was missing:
- a caller cannot choose its own address. X-Forwarded-For is a request
header; unless the site has said where its proxies are, it is ignored.
- a range that admits everybody cannot be saved by accident.
- the device token does not travel in a URL, and the badge-to-name answer
cannot be walked by whoever is admitted.
"""
import pytest
from shopdb.api import cache
from shopdb.core.models import Setting
PLANT = '10.134.48.17'
PLANT_CIDR = '10.134.48.0/23'
ELSEWHERE = '10.9.100.5'
# The reverse proxy sits ON the plant network, which is the normal place for it
# and the reason the old fail-closed claim was false: its own address matched
# the allowlist. A proxy parked outside the range, as this file used to have,
# proves nothing.
PROXY = '10.134.48.5'
PROXY_CIDR = '10.134.48.5/32'
KIOSK_TOKEN = 'usb-kiosk-admission-token'
@pytest.fixture
def kiosk(db):
"""Self-hosted USB, kiosk ON, a token set, one employee, one drive."""
from plugins.employees.models import DirectoryEmployee
from plugins.usb.models import USBDevice
# Session-scoped app means one SimpleCache for the whole run, and the badge
# limiter counts in it. Start every test with an empty budget.
cache.clear()
db.session.add(Setting(key='usb_directory_mode', value='selfhosted',
valuetype='string', category='site'))
db.session.add(Setting(key='usb_kiosk_enabled', value='true',
valuetype='boolean', category='usb'))
db.session.add(Setting(key='usb_kiosk_token', value=KIOSK_TOKEN,
valuetype='string', category='usb'))
db.session.add(DirectoryEmployee(sso=270015376, firstname='Dana',
lastname='Reyes'))
db.session.add(USBDevice(serialnumber='USB-1', label='Blue Kingston',
ischeckedout=False))
db.session.commit()
return db
def _set(db, key, value, valuetype='string'):
row = Setting.query.filter_by(key=key).first()
if row is None:
db.session.add(Setting(key=key, value=value, valuetype=valuetype,
category='usb'))
else:
row.value = value
db.session.commit()
def _session(client, address, badge='270015376', **headers):
return client.post('/api/usb/kiosk/session', json={'badge': badge},
environ_base={'REMOTE_ADDR': address},
headers=headers)
# -- where the caller is ------------------------------------------------------
# The stack that ships resolves this BEFORE Flask: IIS -> HttpPlatformHandler ->
# waitress on loopback with --trusted-proxy=127.0.0.1 --trusted-proxy-count=1,
# which takes the last X-Forwarded-For hop into REMOTE_ADDR. So REMOTE_ADDR in
# these tests is what that layer decided, and the header beside it is what the
# app is required to ignore.
def test_the_app_admits_on_the_address_the_wsgi_layer_resolved(kiosk, client):
"""Production shape: waitress already promoted the caller into REMOTE_ADDR."""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
assert _session(client, PLANT).status_code == 200
def test_a_caller_cannot_forward_itself_into_the_allowlist(kiosk, client):
"""THE FIX. The forwarding header is not evidence of anything here.
Without it, anyone who could reach the app sent
'X-Forwarded-For: <an allowlisted address>' and was admitted to an
unauthenticated write from anywhere on the network.
"""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
assert _session(client, ELSEWHERE,
**{'X-Forwarded-For': PLANT}).status_code == 404
def test_the_header_cannot_take_an_admitted_caller_away_either(kiosk, client):
"""Read in either direction, the header is simply not consulted.
Stated as its own case because a half-measure that only ignored the header
when it would have ADMITTED would still leave the app parsing it, and the
next person would wire the other branch back up.
"""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
assert _session(client, PLANT,
**{'X-Forwarded-For': ELSEWHERE}).status_code == 200
def test_declaring_a_proxy_does_not_turn_header_reading_back_on(kiosk, client):
"""usb_kiosk_trusted_proxies subtracts addresses; it never adds a source.
The version this replaced read X-Forwarded-For once this setting named the
peer. On the Windows stack the peer is never the proxy - waitress replaced
it with what the proxy forwarded - so that branch could not fire in
production, and anywhere it could it handed the header back its authority.
"""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
_set(kiosk, 'usb_kiosk_trusted_proxies', '10.20.30.40/32')
assert _session(client, '10.20.30.40',
**{'X-Forwarded-For': PLANT}).status_code == 404
def test_a_request_wearing_the_proxy_address_fails_closed(kiosk, client):
"""THE FIX, and the case the old "fails closed" test did not cover.
A proxy on the plant network forwards nothing - it is misconfigured, or
something bypassed it - so the request arrives as the proxy's own address,
which is inside usb_kiosk_allowed_cidrs. It carries no evidence about who
sent it and must be refused, not admitted.
"""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
_set(kiosk, 'usb_kiosk_trusted_proxies', PROXY_CIDR)
assert _session(client, PROXY).status_code == 404
# And a caller on the same allowlisted network is still admitted, or the
# subtraction has taken the feature with it.
assert _session(client, PLANT).status_code == 200
def test_the_proxy_is_refused_even_carrying_a_forwarding_header(kiosk, client):
"""Its own address is what arrived, whatever it says about somebody else."""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
_set(kiosk, 'usb_kiosk_trusted_proxies', PROXY_CIDR)
assert _session(client, PROXY,
**{'X-Forwarded-For': PLANT}).status_code == 404
def test_loopback_is_never_a_kiosk_client(kiosk, client):
"""The shipped Windows default with no X-Forwarded-For at all.
HttpPlatformHandler connects from 127.0.0.1, waitress has nothing to
promote, and every caller then wears the app server's own address. A site
whose allowlist happens to cover loopback must not turn that into admission
for everyone who can reach the server.
"""
_set(kiosk, 'usb_kiosk_allowed_cidrs', '127.0.0.0/8, ' + PLANT_CIDR)
assert _session(client, '127.0.0.1').status_code == 404
assert _session(client, PLANT).status_code == 200
def test_a_proxy_address_is_refused_even_when_nobody_declared_it(kiosk, client):
"""Honest limit, written down. Loopback is refused with no setting at all;
any OTHER proxy address is indistinguishable from a PC on that network, so
the site has to name it. This test is what says so out loud."""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
assert _session(client, PROXY).status_code == 200
_set(kiosk, 'usb_kiosk_trusted_proxies', PROXY_CIDR)
assert _session(client, PROXY).status_code == 404
def test_the_token_still_works_from_a_proxy_address(kiosk, client):
"""Subtracting proxies is about admission BY NETWORK. A screen presenting
its device token has said who it is and is not affected."""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
_set(kiosk, 'usb_kiosk_trusted_proxies', PROXY_CIDR)
assert _session(client, PROXY,
**{'X-Kiosk-Token': KIOSK_TOKEN}).status_code == 200
def test_todays_deployment_is_unchanged(kiosk, client):
"""No trusted proxy set - the default - still admits by connecting peer."""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
assert _session(client, PLANT).status_code == 200
# -- what may be saved in an allowlist ---------------------------------------
@pytest.mark.parametrize('value, refused', [
('', False),
('192.0.2.0/24', False),
(' 10.134.48.0/23 , 10.48.249.0/26 ', False),
('0.0.0.0/0', True),
('::/0', True),
('10.0.0.0/8', True),
('192.0.2.0/24, 0.0.0.0/0', True),
('not-a-cidr', True),
])
def test_kiosk_cidr_problem(value, refused):
from plugins.usb.api.routes import kiosk_cidr_problem
assert (kiosk_cidr_problem(value) is not None) is refused
def test_a_wide_range_is_allowed_when_a_site_says_so_deliberately():
from plugins.usb.api.routes import kiosk_cidr_problem
assert kiosk_cidr_problem('10.0.0.0/8', allowwide=True) is None
# Never, though: the override is about a big site, not about no bound.
assert kiosk_cidr_problem('0.0.0.0/0', allowwide=True) is not None
def test_saving_everything_is_refused_with_a_reason(kiosk, client, auth_headers):
"""THE FIX, at the place the administrator is standing."""
response = client.put('/api/settings/usb_kiosk_allowed_cidrs',
json={'value': '0.0.0.0/0'}, headers=auth_headers)
assert response.status_code == 400
message = response.get_json()['data']['error']['message']
assert 'without a login' in message
assert Setting.query.filter_by(key='usb_kiosk_allowed_cidrs').first() is None
def test_a_sane_range_still_saves(kiosk, client, auth_headers):
response = client.put('/api/settings/usb_kiosk_allowed_cidrs',
json={'value': PLANT_CIDR}, headers=auth_headers)
assert response.status_code == 200
assert Setting.get('usb_kiosk_allowed_cidrs') == PLANT_CIDR
def test_the_override_lets_a_flat_site_save_its_range(kiosk, client,
auth_headers):
assert client.put('/api/settings/usb_kiosk_allowed_cidrs',
json={'value': '10.0.0.0/8'},
headers=auth_headers).status_code == 400
_set(kiosk, 'usb_kiosk_allow_wide_cidrs', 'true', valuetype='boolean')
assert client.put('/api/settings/usb_kiosk_allowed_cidrs',
json={'value': '10.0.0.0/8'},
headers=auth_headers).status_code == 200
def test_an_override_turned_off_stays_off(kiosk, client, auth_headers):
"""Stored as a string, 'false' is a truthy Python value. It must not be one."""
_set(kiosk, 'usb_kiosk_allow_wide_cidrs', 'false')
assert client.put('/api/settings/usb_kiosk_allowed_cidrs',
json={'value': '10.0.0.0/8'},
headers=auth_headers).status_code == 400
def test_the_trusted_proxy_list_is_validated_too(kiosk, client, auth_headers):
"""A proxy list that matches everything would trust every caller's header."""
assert client.put('/api/settings/usb_kiosk_trusted_proxies',
json={'value': '0.0.0.0/0'},
headers=auth_headers).status_code == 400
def test_other_settings_are_not_touched_by_the_guard(kiosk, client,
auth_headers):
"""The before_request hook runs on every save; it may only judge its own."""
response = client.put('/api/settings/site_name',
json={'value': '0.0.0.0/0'}, headers=auth_headers)
assert response.status_code == 200
# -- who may find out what the rules are --------------------------------------
# The validator runs in a before_request, which is ahead of the view and so
# ahead of @require_permission. That ordering answered an anonymous caller 400
# with the reason a range was refused, while an anonymous caller who happened to
# send a sane range fell through to 401 - the difference being a free oracle on
# the allowlist rules.
def test_an_anonymous_probe_is_refused_before_the_value_is_judged(kiosk, client):
"""THE FIX. Authentication decides first."""
refused = client.put('/api/settings/usb_kiosk_allowed_cidrs',
json={'value': '0.0.0.0/0'})
assert refused.status_code in (401, 403)
assert 'without a login' not in refused.get_data(as_text=True)
def test_the_probe_cannot_tell_a_bad_range_from_a_good_one(kiosk, client):
"""Both answers must be the same one, or the oracle is still there."""
bad = client.put('/api/settings/usb_kiosk_allowed_cidrs',
json={'value': '0.0.0.0/0'})
good = client.put('/api/settings/usb_kiosk_allowed_cidrs',
json={'value': PLANT_CIDR})
assert bad.status_code == good.status_code
assert (bad.get_json()['data']['error']['code']
== good.get_json()['data']['error']['code'])
assert Setting.query.filter_by(key='usb_kiosk_allowed_cidrs').first() is None
# -- the badge budget is a setting, not a config file -------------------------
# It was app-config only, which on an air-gapped server means editing
# instance/config.py and restarting IIS. In practice that is no off switch at
# all, and the only lever anybody had was turning the whole kiosk off.
def test_the_badge_limiter_settings_are_declared(kiosk):
"""Declared where the app seeds defaults, or nothing creates the rows."""
from plugins.usb.plugin import USBPlugin
declared = {entry['key']: entry
for entry in USBPlugin().get_settings_defaults()}
assert declared['usb_kiosk_badge_limit_enabled']['value'] == 'true'
assert declared['usb_kiosk_badge_limit_enabled']['valuetype'] == 'boolean'
assert declared['usb_kiosk_badge_miss_max']['value'] == '15'
assert declared['usb_kiosk_badge_miss_max']['valuetype'] == 'integer'
def test_the_budget_setting_moves_the_bound(kiosk, client, app):
"""A site dials the number down and the kiosk stops there."""
from plugins.usb.api.routes import KIOSK_BADGE_MISS_MAX
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
_set(kiosk, KIOSK_BADGE_MISS_MAX, '2', valuetype='integer')
# The app-config override the suite normally uses has to be out of the way,
# or it would answer instead of the setting.
previous = app.config.pop('USB_KIOSK_BADGE_MISS_MAX', None)
app.config['USB_KIOSK_BADGE_LIMIT_ENABLED'] = True
try:
for _ in range(2):
assert _session(client, PLANT, badge='111111111').status_code == 422
assert _session(client, PLANT, badge='111111111').status_code == 429
finally:
app.config.pop('USB_KIOSK_BADGE_LIMIT_ENABLED', None)
if previous is not None:
app.config['USB_KIOSK_BADGE_MISS_MAX'] = previous
def test_a_nonsense_budget_falls_back_to_the_shipped_one(kiosk, app):
"""Zero is not "no limit" here - it would refuse the first scan of every
window - and a settings page can store a blank or a typo just as easily."""
from plugins.usb.api.routes import (BADGE_MISS_MAX, KIOSK_BADGE_MISS_MAX,
_badge_miss_budget)
previous = app.config.pop('USB_KIOSK_BADGE_MISS_MAX', None)
try:
for stored in ('', '0', '-4', 'lots'):
_set(kiosk, KIOSK_BADGE_MISS_MAX, stored, valuetype='integer')
with app.test_request_context('/'):
assert _badge_miss_budget()[1] == BADGE_MISS_MAX
finally:
if previous is not None:
app.config['USB_KIOSK_BADGE_MISS_MAX'] = previous
def test_the_kill_switch_is_reachable_without_editing_a_config_file(kiosk, app):
"""And it defaults to today's behaviour: on.
Off under TESTING is an arrangement for this suite, not the default, so the
first assertion here is also what stops "the limiter is off" quietly
becoming true everywhere.
"""
from plugins.usb.api.routes import (KIOSK_BADGE_LIMIT_ENABLED,
_badge_limit_enabled)
previous = app.config.pop('USB_KIOSK_BADGE_LIMIT_ENABLED', None)
try:
with app.test_request_context('/'):
app.config['TESTING'] = False
try:
assert _badge_limit_enabled() is True
_set(kiosk, KIOSK_BADGE_LIMIT_ENABLED, 'false',
valuetype='boolean')
assert _badge_limit_enabled() is False
finally:
app.config['TESTING'] = True
finally:
if previous is not None:
app.config['USB_KIOSK_BADGE_LIMIT_ENABLED'] = previous
# -- provisioning a screen ----------------------------------------------------
def test_the_setup_link_does_not_carry_the_token(kiosk, client, auth_headers):
"""THE FIX. A query string ends up in history, referrers and proxy logs."""
response = client.get('/api/usb/kiosk/provisioning-link',
headers=auth_headers)
assert response.status_code == 200
url = response.get_json()['data']['url']
assert KIOSK_TOKEN not in url
assert url.startswith('/api/usb/kiosk/setup?code=')
def test_opening_the_setup_link_provisions_that_browser(kiosk, client,
auth_headers):
"""One open on the screen, and its later calls are admitted with no token."""
url = client.get('/api/usb/kiosk/provisioning-link',
headers=auth_headers).get_json()['data']['url']
redirected = client.get(url)
assert redirected.status_code == 302
assert redirected.headers['Location'].endswith('/usb-kiosk')
cookie = redirected.headers['Set-Cookie']
assert 'usbkiosktoken=' + KIOSK_TOKEN in cookie
assert 'HttpOnly' in cookie
# No X-Kiosk-Token header, no allowlist: the cookie is the whole credential.
assert _session(client, ELSEWHERE).status_code == 200
def test_a_setup_code_is_good_once(kiosk, client, auth_headers):
url = client.get('/api/usb/kiosk/provisioning-link',
headers=auth_headers).get_json()['data']['url']
assert client.get(url).status_code == 302
second = client.get(url)
assert second.status_code == 410
assert 'Generate' in second.get_json()['data']['error']['message']
def test_a_made_up_setup_code_provisions_nothing(kiosk, client):
response = client.get('/api/usb/kiosk/setup?code=not-a-real-code')
assert response.status_code == 410
assert 'Set-Cookie' not in response.headers
def test_an_expired_setup_code_is_dead(kiosk, client, auth_headers):
from plugins.usb.api.routes import KIOSK_SETUP_CODE
url = client.get('/api/usb/kiosk/provisioning-link',
headers=auth_headers).get_json()['data']['url']
pending = Setting.get(KIOSK_SETUP_CODE)
_set(kiosk, KIOSK_SETUP_CODE, pending.split('|')[0] + '|1')
assert client.get(url).status_code == 410
def test_the_old_header_still_admits_screens_already_on_the_floor(kiosk,
client):
"""Screens provisioned before this change must not need a visit."""
assert _session(client, ELSEWHERE,
**{'X-Kiosk-Token': KIOSK_TOKEN}).status_code == 200
# -- the badge-to-name answer -------------------------------------------------
@pytest.fixture
def smallbudget(app):
"""Three misses, so the test states the rule rather than counting to 15.
Also turns the limiter on: it defaults off under TESTING, because the
counter lives in a cache that outlives a test and is shared across whatever
else runs on the same worker. Same arrangement as the login limiter.
"""
previous = {key: app.config.get(key)
for key in ('USB_KIOSK_BADGE_MISS_MAX',
'USB_KIOSK_BADGE_LIMIT_ENABLED')}
app.config['USB_KIOSK_BADGE_MISS_MAX'] = 3
app.config['USB_KIOSK_BADGE_LIMIT_ENABLED'] = True
yield
for key, value in previous.items():
if value is None:
app.config.pop(key, None)
else:
app.config[key] = value
def test_badge_guessing_is_bounded_per_source(kiosk, client, smallbudget):
"""THE FIX. The kiosk answers with a NAME, so it is a lookup to be walked.
Admission by network position can mean a whole VLAN, not one screen by the
cabinet, which is what makes an unbounded lookup worth somebody's loop.
"""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
for _ in range(3):
assert _session(client, PLANT, badge='111111111').status_code == 422
refused = _session(client, PLANT, badge='111111111')
assert refused.status_code == 429
# And a badge that WOULD resolve gets nothing either, or the bound is a
# speed bump with a hole in it.
assert _session(client, PLANT).status_code == 429
def test_the_bound_is_per_caller_not_per_kiosk(kiosk, client, smallbudget):
"""One PC guessing must not shut the cabinet screen down."""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
for _ in range(4):
_session(client, PLANT, badge='111111111')
assert _session(client, PLANT).status_code == 429
assert _session(client, '10.134.48.99').status_code == 200
def test_ordinary_use_never_spends_the_budget(kiosk, client, smallbudget):
"""Only a badge this site has never heard of counts."""
_set(kiosk, 'usb_kiosk_allowed_cidrs', PLANT_CIDR)
for _ in range(6):
assert _session(client, PLANT).status_code == 200
def test_enabling_the_plugin_on_a_running_server_still_works():
"""init_app also runs when an operator switches the plugin on live.
Flask refuses a setup method once the app has served a request, and the
loader turns any exception out of init_app into a plugin that failed to
load - so an unguarded before_request would have made "enable USB" fail on
every running server.
"""
from flask import Flask
from plugins.usb.plugin import USBPlugin
running = Flask(__name__)
running.test_client().get('/')
USBPlugin().init_app(running, None)

63
tests/test_plugins/test_usb_reports.py
View File

@@ -137,3 +137,66 @@ def test_the_report_is_registered_so_it_appears_in_the_catalogue(client, usbdata
assert report['id'] == 'usb-audit'
# The endpoint is what the reports page fetches; a wrong one renders blank.
assert report['endpoint'] == '/api/usb/reports/audit'
# -- sanitized / virus-scan state in the audit ---------------------------------
# CMMC evidence: the report answered who had which drive and when, but not
# whether it was wiped or scanned on the way back - which is the question the
# record exists to answer. Both stores held the values; neither reported them.
def test_audit_reports_wipe_and_scan_state_on_returns(db, client, auth_headers):
from plugins.usb.models import USBDevice, USBCheckout
from shopdb.core.models import Setting
from datetime import datetime, timedelta
db.session.add(Setting(key='usb_directory_mode', value='selfhosted',
valuetype='string', category='site'))
device = USBDevice(serialnumber='USB-9', label='Grey SanDisk')
db.session.add(device)
db.session.flush()
now = datetime.utcnow()
db.session.add(USBCheckout(
usbdeviceid=device.usbdeviceid, sso='270015376', checkoutname='Dana',
checkouttime=now - timedelta(hours=2), checkintime=now - timedelta(hours=1),
waswiped=True, scannedviruses=False))
db.session.commit()
response = client.get('/api/usb/reports/audit?days=7', headers=auth_headers)
assert response.status_code == 200
data = response.get_json()['data']
assert 'sanitized' in data['columns']
assert 'scanned_viruses' in data['columns']
checkin = [r for r in data['rows'] if r['action'] == 'check-in'][0]
checkout = [r for r in data['rows'] if r['action'] == 'check-out'][0]
assert checkin['sanitized'] == 'yes'
assert checkin['scanned_viruses'] == 'no'
# A take carries neither: both are recorded when the drive comes back.
assert checkout['sanitized'] == ''
assert checkout['scanned_viruses'] == ''
def test_unrecorded_wipe_is_blank_not_no(db, client, auth_headers):
"""A null means nobody wrote it down. Rendering that as 'no' would assert
something no one ever claimed, in the one record kept to be trustworthy."""
from plugins.usb.models import USBDevice, USBCheckout
from shopdb.core.models import Setting
from datetime import datetime, timedelta
db.session.add(Setting(key='usb_directory_mode', value='selfhosted',
valuetype='string', category='site'))
device = USBDevice(serialnumber='USB-10', label='Old stock')
db.session.add(device)
db.session.flush()
now = datetime.utcnow()
db.session.add(USBCheckout(
usbdeviceid=device.usbdeviceid, sso='270015376', checkoutname='Dana',
checkouttime=now - timedelta(hours=3), checkintime=now - timedelta(hours=2),
waswiped=None, scannedviruses=None))
db.session.commit()
response = client.get('/api/usb/reports/audit?days=7', headers=auth_headers)
checkin = [r for r in response.get_json()['data']['rows']
if r['action'] == 'check-in'][0]
assert checkin['sanitized'] == ''
assert checkin['scanned_viruses'] == ''