The app has a dark theme. /api/docs was the one page that stayed white when the
rest of the site went dark, which is exactly the page somebody has open beside
the app while they work.
Redoc does not read prefers-color-scheme, and the <redoc spec-url> element it
was using takes no theme at all - so the page now initialises through
Redoc.init, which does. Light keeps the default look; dark names the colours
Redoc otherwise derives from the background and gets wrong: body text, the
sidebar, and the right-hand sample panel, which came out dark-on-dark.
It re-initialises on a live scheme change rather than restyling, because Redoc
bakes its colours into styled-components at init - a CSS override reaches the
chrome and misses the content. The listener has the addListener fallback for the
older MediaQueryList, since a shop-floor kiosk browser is exactly where this
gets read.
Nothing new is loaded: same vendored bundle, same spec, no font, no CDN. A test
pins that, because theming is the kind of change that quietly introduces one.
The try-it page is left light. Swagger UI 5 ships no dark theme, so it would
mean vendoring a community stylesheet that fights upstream on every bump - and
it is a development tool that is off in production anyway.
/api/docs on the production site handed out
https://<host>/api/apitokens for an app that lives at /shopdb. Copy that URL
and it 404s; Swagger's try-it fired at the same wrong address.
The mount is a DEPLOYMENT fact and the spec is a BUILD artifact.
gen_openapi.py writes servers: [{"url": "/"}] because at generation time nobody
knows where the app will be hung, and wsgi.py only moves MOUNT_PATH from
PATH_INFO into SCRIPT_NAME when the process starts. Served as written, the file
was right for a root install and wrong for every subpath one - which is how
production runs.
The endpoint now sets `servers` per request from request.script_root: empty at
root, /shopdb under the subpath method, and correct behind a proxy that sets the
prefix. The paths themselves stay mount-free, because the prefix belongs in the
server entry and duplicating it into each path would double it.
The file is about a megabyte, so it is parsed once and cached by mtime, and each
request gets a shallow copy with only `servers` replaced - two requests arriving
on different mounts cannot race on one dict.
A test pins both cases and the fact that paths stay unprefixed. Planting a wrong
server URL fails it.
/api/docs renders the spec with Redoc, which is read-only by design. The thing
people mean when they ask for Swagger is firing a request from the page, so
Swagger UI now sits beside it at /api/docs/try, reading the same spec - one
contract, two renderers.
It is OFF outside development and 404s rather than rendering a disabled page: a
production box holds live fleet data, and a docs page that can POST to it is a
different object from a docs page that cannot.
Vendored, not linked. swagger-ui-bundle.min.js and swagger-ui.css are served
from staticdocs/ beside the existing Redoc bundle, because production has no
internet. validatorUrl is null: left unset, Swagger UI hands the spec to the
public swagger validator to render a badge, which here is a request that cannot
succeed and elsewhere would ship the internal API surface off the box.
THE VENDORED REDOC BUNDLE WAS ALREADY REACHING THE INTERNET, found while
proving this page does not. redoc.standalone.js fetched
cdn.redoc.ly/redoc/logo-mini.svg for the "API docs by Redocly" mark, so every
load of /api/docs on the air-gapped server has been waiting on a request that
cannot complete. Replaced with an inline transparent GIF.
Three gate holes closed, each one a dodge that worked:
- the page is built by interpolating url_for output into a JS string literal,
unescaped. SCRIPT_NAME is proxy-controlled and ProxyFix honours
X-Forwarded-Prefix, so a mount path carrying a quote closed the literal and
executed what followed. The spec URL is now json.dumps'd and the rest
HTML-escaped; the injected quote renders as \" and is inert
- the "no internet" assertion on the page scanned for https?://, so
<script src="//cdn.jsdelivr.net/..."> - the form a copy-paste from any CDN's
own documentation produces - matched nothing and passed
- the same assertion on the vendored files was a five-host CDN allowlist. An
upstream bump to any other host walked through it, and the one real leak in
that directory was on no list. It now matches the SHAPES a browser fetches -
@import, css url(), src/href, @font-face - whatever the host, ignoring the
data: URIs this CSS is full of
And the Redoc page's link to the try-it view is built with url_for, which raises
BuildError when those routes are not mounted - every production box. Made
unconditional it would 500 the docs page in exactly the deployment nobody tests
locally, and no test caught that. One does now.
Report IP is an icon on every Public Desktop in the company, so a help desk
caller asking for it to be re-run is asking for something they could already
do themselves at the keyboard. Getting there through ShopDB meant holding
endpoints.runscript, which is the permission that lets someone push ANY script
in the library - the agent self-upgrade included. Handing that over to run a
desktop shortcut is the bundling the permission split was created to take
apart, so the work was simply not being done through the tool.
The allowance is on one named script, never on the run_script action. The
script id rides in the caller-controlled request body, so relaxing the action
would grant the whole library and quietly rebuild the endpoints.deploy bundle.
The enqueue route resolves the target to a real Script row first and matches
the STORED name, because matching what the requester typed would let anyone
name their way past the grant.
Granting on a name is only defensible while the name cannot be made to mean
something else. The body is source-controlled at
plugins/endpoints/api/templates/report-ip-shortcut.ps1, seeded from there,
pinned by SHA-256 on delivery, and the managed fields are re-asserted on every
seed - so an operator cannot flip the row to runcontext=system, deactivate it,
or edit the body and leave the name still pushable.
The template lives under the plugin rather than agent/ because the image copies
plugins/ but not agent/: reading it from agent/ works on a developer checkout
and fails in the deployed container.
The script runs in the signed-in user's session. This is not a preference.
Launched as SYSTEM the tool re-enters the interactive session and the launcher
exits 0 immediately, reporting success having observed nothing - the same
session-0 boundary the collector already documents for mapped drives. User
context fails closed when nobody is logged on, which is the honest answer.
The seed runs in the migrate service alongside the other seeders, so an
existing site picks the script up on the upgrade it already performs.
Turning "Shared key allowed" off refused the key on the endpoints plugin and
left four core collector routes open to it. Anything still holding the old key
could keep rewriting logged-in user, last boot time, serial number and installed
app inventory for ANY hostname, while the admin believed the shared key was off.
Data integrity only - no command execution, no credential issuance - but it is a
hole in a boundary the product advertises as closed.
It cannot simply be closed. Both GE-Enforce reporters still authenticate with
that key, so refusing it fleet-wide would stop asset and printer reporting
everywhere, and on an air-gapped site mid-migration that is the wrong order of
operations. collector.legacysharedkeyrefused therefore DEFAULTS TO TODAY'S
BEHAVIOUR, and its description names exactly what turning it on breaks and when
it should be turned on: after the reporters move to per-PC credentials, not
before.
A legacy route used with the shared key now logs a warning, so a site can find
out who still depends on the key before flipping anything. On a stock Windows
deployment that warning may name the proxy rather than the caller; the token
prefix in the same line is the identifier to go by.
The switch is seeded with the core defaults and has a control on the
authentication settings page. Undeclared, it was a row that did not exist, and
this UI has no generic key/value editor, so seeding alone would have rendered
nothing for an operator to click.
Two contract declarations corrected while in here:
notifications.alertscope was declared db.JSON in the model while the migration
that creates it ships TEXT, and those migrations state the constraint in their
own docstrings: no JSON columns. The model now matches the shipped column rather
than the schema being bent to the model.
usb and printedparts both import a name that arrived in contract 0.35.0 while
declaring core_version floors of 0.1.0 and 0.16.0, so the loader would have
accepted them on a core that cannot satisfy them - an ImportError at blueprint
registration instead of a refusal with a reason. Both floors now say what the
plugin actually needs.
The device token assumes a handful of fixed screens an administrator
provisions once. That is not how these kiosks are used: most people open the
page on their own PC, and there is no login, so per-browser provisioning would
mean touching every PC on the floor and losing it again on the next reimage or
cleared profile. A screen in that state did not look refused either - the USB
kiosk rendered a working page whose every call answered 404.
Both kiosks now admit a caller on either ground, the same shape the GE-Enforce
client endpoints already use: the per-device token, or a source address inside
an allowlist. Wall screens keep the token and are not confined to the
allowlist; everyone else is admitted by where they are on the network. Still
gated on the site's enable flag, so nothing opens by itself, and both
allowlists ship empty - token-only until a site enters its ranges.
- shopdb.api gains ip_in_cidrs and request_ip_allowlisted (contract 0.35.0).
On the surface because both plugins need the identical gate, and a CIDR
parser copied into two of them is where a fail-open creeps in later. They
fail closed on an empty setting, an unparseable caller address and a
malformed entry, and they read remote_addr rather than raw X-Forwarded-For -
so the IIS rewrite rule is what makes them meaningful, which the module says
at length.
- Separate settings per kiosk, usb_kiosk_allowed_cidrs and
printedparts_kiosk_allowed_cidrs. One must not grant the other, for the same
reason the tokens are separate.
- Both settings pages get a picker listing the network plugin's subnets with
their VLANs. It stores resolved CIDR strings, not subnet ids: the setting has
to work at a site that does not run that plugin and must not break when an
inventory row is deleted. Entries matching no subnet are shown rather than
hidden, since a stale one left by an inventory edit looks exactly like a
hand-typed range.
- USBKiosk says it was not admitted instead of pretending to work, which
PartsKiosk already did.
- The USB settings page keyed text-vs-boolean off the token alone. Adding a
second text setting to the boolean path would have saved it as the word
'true' - the bug that replaced the kiosk token and 404'd every provisioned
screen. Both load and save now key off a list, with a spec on it.
The trade, stated plainly in the admin UI: anyone on an allowed network can
check a drive in or out against a badge number they guess, and the record
carries that person's name. Badges are attribution, not authorization. Keeping
the ranges narrow is what bounds it; login is the alternative and was ruled out
because these users do not sign in.
Finding F12 of the architectural review, checked against the code and found
worse than reported: the dynamic component import cannot be rewritten by the
bundler, so the built file ships the unresolved path and a plugin tab answers
404 in every production build, staged or not. It works only under the dev
server. Nothing shipping is affected, because the one plugin that declares a
tab also registers the same page as an ordinary route, and that is the one
sites reach.
- The ADR draft now carries a "Constraints as implemented" section: only the
model catalog hub registers plugin tabs, registration happens when that hub
mounts so a tab is not deep-linkable and does not survive a refresh, and the
build finding above with the bundler's own warning. The two ways out are
stated as the open decision rather than taken.
- The hook's docstring reads as instructions to a plugin author, and its
example no longer names a path that does not exist at build time.
- A route declared for a hub that cannot render it is dropped with an error
naming the plugin, the hub and the hubs that do work, and the same is logged
at startup. Enabling the plugin still succeeds: a tab nobody can reach is no
reason to take a working backend offline.
- Tests hold the two halves together: no bundled plugin may declare a hub the
frontend does not implement, and the list in the server must equal the hubs
that really register routes, so the two cannot drift apart again.
Findings F2, F4, F5, F6, F7, F8, F9, F10, F11 and F13 from
wj-architectural-review.md. F1/F1a (the non-sargable LOWER pattern) and F3
(the warranty sync holding a pooled connection) are being handled separately
and are untouched here.
- A collector check-in that raced another for the same PC lost everything:
the unique index on the application fingerprint raised, nothing caught it,
and the whole check-in rolled back, discarding state, events and hardware,
answering the agent with form copy meant for a person. The insert now runs
in a savepoint, reconciles what the other transaction already wrote, one row
per savepoint so a second collision skips a row rather than the batch, and
the check-in commits with a warning. This is the shape the delete side of
the same function already had.
- Enrollment is deliberately unauthenticated, so it now carries a per-address
rate limit and a cap on pending requests, and the cap is a setting an admin
can see and change. A flood can no longer bury the approval queue or grow
the table without bound.
- The fleet alert poll no longer formats debug strings on every request.
- BusinessUnit is on the contract surface (0.34.0), so the endpoints plugin
stops reaching into a core table with raw SQL.
- A test walks every asset type descriptor and proves the endpoint it names
really refuses a caller without the declared permission.
- The plugin settings routes now need a permission; they were telling anyone
which modules a site runs and where its components live.
- The connection pool is sized explicitly with a short timeout, and the
arithmetic against the server's connection ceiling is written next to the
worker count instead of being left to defaults nobody chose.
- Fleet alerts carry both the correct lowercase keys and the old camelCase
pair. Trays older than 1.7.0 match keys case-sensitively and would have gone
quiet on every PC not yet upgraded; the old pair goes when they are.
- The dev server proxy points at localhost again, with environment variables
for the remote-box workflow, and one developer's address is out of the
server certificate page.
An admin generates a short-lived token in Settings > Endpoints > Endpoints and
copies the command it produces. A tech runs that one line in an elevated
PowerShell on a PC with no agent:
powershell -ExecutionPolicy Bypass -c "irm https://<server>/shopdb/api/
endpoints/siteagent/bootstrap.ps1?t=<token> | iex"
The script refuses to run unelevated, refuses a PC that already has the agent
(a managed bay gets it from GE-Enforce), downloads the package the existing
builder produces, checks both binaries against the hashes the server baked in,
installs with NO shared key, and reports what happened. The PC then asks to
enroll and waits for an admin to approve it, which the script watches for
locally for two minutes before exiting.
- New endpointbootstraptokens table and migration
endpoints0024bootstraptokens: hash only, expiry, use count, revoke.
- Admin routes to generate, list and revoke tokens; the secret is shown once.
- Two token-only routes: the script, and the package (which spends a use).
Anything wrong with the token answers 404 rather than confirming it exists,
both are rate limited per address and both are audited with the caller.
- The package builder is now shared rather than duplicated, and a bootstrap
package can never carry a collector key.
Per-PC credentials (replaces the 09-14 client-certificate attempt):
- POST /api/endpoints/enroll issues a credential to a PC that matches exactly
one computer by hostname and BIOS serial and has none yet. Anything else
(unknown or duplicate hostname, serial mismatch, already enrolled) waits as
a request an admin approves or rejects. The agent's persisted nonce binds an
approved request to the PC that asked.
- One auth helper serves every agent route. With X-Endpoint-Credential the
asset comes from the credential and a request naming another PC gets 403
ASSET_MISMATCH. The shared key still works for PCs that have not enrolled,
is refused (401 SHARED_KEY_REFUSED) for any PC that has, and can be switched
off. A revoked credential gets 401 CREDENTIAL_INVALID so the agent
re-enrolls.
- The core collector accepts the credential for the computers, printers and
endpoints collectors through two new plugin hooks exposed on shopdb.api
(contract 0.33.0), and refuses the shared key for enrolled PCs.
- Admin API: enrollment list, approve, reject, revoke (un-enroll), settings.
Migration API: per-PC readiness with blockers, and upgrade waves that
package the agent once and queue the existing agent upgrade per PC.
- Migration 0023 creates endpointcredentials, endpointenrollrequests and
endpointmigrationwaves, drops the certificate columns from assets where
test databases have them, and renames app_fingerprint to appfingerprint.
0021 and 0022 keep their ids but no longer alter the core assets table.
- Removed: client_cert.py, certificate renewal, the unauthenticated
/test/client-cert route, nginx client certificate forwarding and the agent
certificate script. The client-cert header could be forged on IIS, the key
was readable by users, and the shared key remained a fallback.
- GET /settings/server-cert returns the SPKI SHA-256 pin for GE-Enforce.
Review fixes:
- report_command_result used g without importing it; every result was a 500.
- collect_log refuses setup answer files, appsettings, agent directories and
key files at enqueue.
- Fleet alerts: alerttimeout and alertfleet agree with the migration, the
scope is validated and a malformed scope no longer reaches the whole fleet,
and the ack race no longer raises NameError.
- A refused break-glass override leaves no hold and no audit row.
- Agent upgrade checks locks and permissions before repackaging, and writes
the package atomically.
- Server certificate upload validates the pair before replacing files;
generation no longer builds an openssl config from user input; audit rows
are actually written.
- Health evaluation runs in a savepoint so a caught error cannot lose the
check-in.
- Required-app policies are found by appid, survive renames, and deleting an
app deactivates policies that still name it. The UI and API now say such a
policy only takes effect inside a business policy.
- Backfill handles duplicate fingerprints and rolls back a failed batch.
- Plugin settings routes skip a plugin whose hook raises.
- The setup token is no longer printed to the log.
- The naming gate checks top-level files again and handles odd file names.
- Removed root scratch scripts, one of which carried a token.
- Rewrote the fleet alert scheduling and app delta tests on real fixtures.
Addresses all critical and high-severity findings from Fable security review.
CRITICAL FIX#1: Server Certificate Validation Bypass
- File: CertificateRenewal.cs
- Issue: Agent accepted ANY server certificate when ServerCertThumbprint not configured
- Fix: Now validates against Windows trusted roots when pinning not configured
- Impact: Prevents MITM attacks during renewal requests
CRITICAL FIX#2: Certificate Revocation Check Incomplete
- Files: asset.py, client_cert.py, migration 0022
- Issue: Revocation check used hasattr() and silently skipped when columns missing
- Fix: Applied migration 0022, uncommented revocation columns, removed hasattr()
- Impact: Revoked certificates can no longer authenticate or renew
CRITICAL FIX#3: Expired Certificates Cannot Renew
- Files: routes.py (renewal endpoint), CertificateRenewal.cs
- Issue: Expired certs fail at TLS handshake before reaching renewal endpoint
- Fix: Added API key + hostname fallback for renewal when cert expired
- Agent now sends hostname in request body for API key authentication
- Impact: Agents can renew expired certificates, 30-day window becomes effective
HIGH FIX#4: Hostname Injection Vulnerability
- File: routes.py
- Issue: No validation before passing hostname to certificate generation
- Fix: Added regex validation - 1-64 chars, alphanumeric plus dot/hyphen/underscore
- Impact: Prevents injection attacks via malicious hostnames
HIGH FIX#5: Rate Limit Bypass via Clock Manipulation
- File: routes.py
- Issue: Naive UTC datetime arithmetic vulnerable to timezone changes
- Fix: Use timezone-aware datetime with explicit UTC conversion
- Impact: Rate limit cannot be bypassed via clock manipulation
HIGH FIX#6: Hard Process Termination Loses In-Flight Work
- File: Program.cs
- Issue: Environment.Exit(0) aborted in-flight work and could corrupt state
- Fix: Graceful return from RunHostAsync() instead of Environment.Exit()
- Impact: Service exits cleanly, no lost telemetry or corrupted local state
HIGH FIX#7: Missing Database Transaction Isolation
- File: routes.py
- Issue: Race condition in concurrent renewal requests
- Fix: Added with_for_update() row-level locking for both auth paths
- Impact: Prevents concurrent renewals from bypassing rate limit
MEDIUM FIX#8: Missing Audit Logging for Failures
- File: routes.py
- Issue: Failed renewal attempts not captured in audit trail
- Fix: Added AuditLog.log() for cert generation and database update failures
- Impact: Security monitoring can detect repeated renewal failures
SECURITY HARDENING:
- Server cert validation: Never accepts certificates blindly
- Revocation enforcement: Fail closed if columns missing
- Expired cert handling: Dual-auth allows renewal while maintaining security
- Input validation: Hostname must match safe character set
- Transaction safety: Row locks prevent race conditions
- Audit trail: All failures logged for security monitoring
OPERATIONAL IMPROVEMENTS:
- Agents can now renew expired certificates (no more manual intervention)
- Graceful shutdown prevents data loss during renewal
- Rate limiting immune to timezone issues
- Comprehensive audit trail for troubleshooting
ADR COMPLIANCE:
- Maintains ADR-001 (Asset as platform contract)
- Maintains ADR-006 (Collector contract - backward compatible)
- Database naming (snake_case columns)
- No unicode/emojis
- Security best practices: ALL CRITICAL/HIGH issues resolved
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
CRITICAL FIXES:
- Enforce cert authentication on collector telemetry endpoint
- Add authorization check in report_command_result (prefer cert over hostname)
- Implement server certificate pinning in C# agents (optional)
- Add constant-time fingerprint comparison to prevent timing attacks
- Validate fingerprint format (40 or 64 hex chars only)
AUTHENTICATION:
- Created shopdb/core/auth/client_cert.py with authenticate_by_client_cert()
- Extracts fingerprint from nginx header (SHA1), IIS environ, or ARR header
- Validates against assets.client_cert_fingerprint in database
- Checks certificate expiration (cert_expires_at column)
- Falls back to API key when cert not present (dual-auth during rollout)
AUTHORIZATION:
- shopdb/core/api/collector.py: Added cert auth to generic_collect() before
API key check. Sets g.cert_authenticated_asset_id when cert validates.
- plugins/endpoints/plugin.py: apply_collector_payload() compares
g.cert_authenticated_asset_id with payload's resolved assetid. Raises
PermissionError if mismatch (prevents agent A from submitting data for
asset B by changing hostname field).
- plugins/endpoints/api/routes.py: report_command_result() prefers
g.cert_authenticated_asset_id over hostname parameter. Requires either
cert or hostname for authorization.
AGENT CHANGES:
- AgentConfig.cs: Added ServerCertThumbprint config for certificate pinning
- CollectorClient.cs: Validates server cert thumbprint when configured
- CommandClient.cs: Validates server cert on WebSocket TLS
- FleetAlertClient.cs: Validates server cert on WebSocket TLS
- When ServerCertThumbprint is empty, accepts all certs (rollout compat)
REVOCATION SUPPORT:
- Added cert_revoked_at and cert_revocation_reason columns to assets table
- Migration 0022_client_cert_revocation.py adds columns
- Revocation check uses hasattr() for backward compatibility
- Revocation requires manual database UPDATE (instant kill switch)
SECURITY HARDENING:
- Constant-time fingerprint comparison via hmac.compare_digest()
- Format validation: must be 40 (SHA1) or 64 (SHA256) hex chars
- SHA1 documented as nginx limitation, acceptable for air-gapped self-signed
- Audit logging for failed authentication attempts (AuditLog.log)
DEPLOYMENT:
- Dual-auth during rollout: cert OR API key both work
- Agents without certs continue using API key (backward compatible)
- No impact on existing fleet until certs provisioned
- Rollout: server first, pilot certs, fleet rollout, retire API key
Addresses Fable security review findings: 1 critical + 5 high vulnerabilities
related to authentication bypass, authorization flaws, and timing attacks.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Bugfix: Fix AttributeError in /api/plugins/settings-routes endpoint
- Changed get_enabled_plugin_instances() to get_all_plugins()
- Added explicit enabled check with pm.registry.is_enabled()
- Test failure: test_no_route_answers_a_nonsense_filter_with_a_server_error
Optimization: Enable parallel test execution (Phase 1 of pytest speedup)
- Added pytest-xdist configuration to pytest.ini
- Enabled -n auto (auto-detect CPU cores, typically 8 workers)
- Added slow marker for future test categorization
- Expected speedup: 4-8x (20 min -> 2.5-5 min)
Root cause: 2857 tests recreate entire schema per test (0.4-0.6s overhead each)
- Phase 2 (savepoint isolation) will achieve 80x total speedup
- See docs/PYTEST-OPTIMIZATION-PLAN.md for full analysis
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Plugins can now register their settings pages dynamically via get_settings_routes()
hook, eliminating hardcoded imports that break lean builds.
Backend:
- Add get_settings_routes() hook to BasePlugin (shopdb/plugins/base.py)
- Implement in ComputersPlugin for Operating Systems management
- Add /api/plugins/settings-routes endpoint to expose plugin routes
Frontend:
- ModelCatalogHub fetches plugin routes and dynamically adds them via router.addRoute()
- Tabs render dynamically from pluginTabs ref
- Components lazy-load when clicked
Benefits:
- Lean builds work without manual route exclusions
- Plugins self-contain their UI extensions
- No core code changes needed when adding plugin settings pages
- Follows plugin architecture best practices
Fixes TODO from commit a03fd3de4f
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
An operator could see that an endpoint was behind the published build but
had no way to act on it. update_app resolves against Application rows, and
the agent lives in the version store, so there was nothing to push.
Bridge the two: the staged build is packaged and registered as a reserved
Application row, and a managed PowerShell script is seeded to install it.
The endpoint downloads that package over the existing collector-authenticated
app-package route, so no parallel delivery path is introduced.
The new route prepares the delivery and then DELEGATES to
enqueue_endpoint_command as a digest-pinned run_script. It deliberately does
not implement its own enqueue: a second path would duplicate, and eventually
drift from, the endpoint lock, the Configuration Policy gate, stale-command
reaping and the at-most-one-in-flight rule. A fleet-wide agent swap is the
last action that should skip a quarantine check. Tests pin that delegation
rather than only the happy path.
Because it lands as run_script, the caller needs endpoints.runscript on top
of endpoints.agentupdate. That is intended: this delivers arbitrary SYSTEM
PowerShell by the same mechanism as any other script push. The audit event
is written only on an actual 201, so a refused push does not record an
upgrade that never happened.
Replacing the agent is self-modifying the control plane - the process
receiving the command is the one being swapped. The script therefore
re-launches itself detached and exits immediately, because the agent runs
run_script as a direct child and holds its stdout/stderr pipes while
waiting on it. Expect the command to report queued rather than success:
the process that would report completion is stopped mid-run. Judge the
result by the version the endpoint reports afterward.
Tray relaunch does not use the installer's `explorer.exe <path>` trick.
That works in install-shopdbagent.ps1 only because the installer runs as
the user - note its own `-ne "SYSTEM"` guard. This script always runs as
SYSTEM in session 0, where the hand-off reaches no shell and exits 0, so
Start-Process reported success for a relaunch that never happened. It also
cannot descend into the session with a token-duplicating launch:
UserContextSpool documents that WTSQueryUserToken is blocked by endpoint
protection on these hosts. Instead a one-shot scheduled task targets the
pinned ShopFloor account, matching how the user-context spool already
resolves that user, and the tray must run as that account or the
mapped-drive probe reports the wrong (empty) drive table. The relaunch is
verified by polling for the process rather than assumed, and stays
best-effort: the Run key still starts the tray at next logon, and a late
tray is not worth rolling back a good agent upgrade.
panel_status now carries agentbehind/agentversion/agentcurrent so the UI
gates the action on a flag rather than parsing "(behind X)" out of a label.
The button is confirmed by typing UPDATE, naming both versions and the
service interruption, matching how run_script and system_restart are gated.
Verified on live endpoints: upgraded 1.13.0.0 and 1.15.0.0 machines to
1.16.0.0. run_script delivery, digest fetch and system context all predate
1.13.0.0, so no minimum agent version is declared for this path.
Gate collect_log on a minimum agent version and bridge the agent version
store to the applications table so a staged build can actually be pushed.
update_app resolves an install source from the applications table alone; it
cannot see agentstore. A build could therefore be staged and current while no
application row existed at all, which is what the fleet was in: the agent was
unpushable. Staging is now the single source of truth - it packages the
current build and upserts the row in the same request, so the row cannot
describe a build that is no longer staged.
PARTIAL BUILD ADVISORY
This is steps 1-4 of a longer rollout and is deliberately incomplete:
- There is no push trigger yet. The registered package is resolvable and
verifiable, but nothing enqueues an upgrade; the "behind current build"
indicator is still display-only.
- The upgrade template script, its script row, and the enqueue route
(endpoints.agentupdate) are not written.
- Blocking constraint for the next slice: run_script executes
powershell.exe -File <script> with NO argument channel, so a delivered
upgrade script cannot be passed BaseUrl/CollectorKey and must either
source them itself or run_script must be extended first.
Nothing here touches a live endpoint. Staging only changes what a future
push would contain.
Details:
- Package members carry a fixed timestamp. zipfile stamps the current mtime
by default, so identical input produced a different archive digest on
every call. update_app pins a download by SHA-256, so a drifting hash
would rewrite installersha256 out from under an endpoint mid-download and
fail its verification.
- All three routes that can change 'current' re-register (upload,
adopt-legacy, rollback). Covering only upload would leave the row pinned
to the build an operator just rolled away from, and the push would ship
it anyway - silent drift.
- Registration cannot fail an upload. The binary is staged and audited by
the time it runs, so a packaging problem is reported in the response
payload rather than raised; returning an error would tell the operator a
successful upload was rejected.
- The package is written to a temp name and os.replace'd, so an interrupted
write cannot leave a truncated archive at the path the row advertises.
- The row is isinstallable but hidden: machine-managed, and it must not
appear in the software catalogue beside real licensed applications.
- Added application_package_location() to the shopdb.api contract surface
(lazy, matching collector_authorized) returning the package directory and
URL prefix together. agentregistry reached into shopdb.core internals,
violating the plugin import contract; sourcing both from one place also
stops the write location and installpath from drifting apart.
collect_log is refused at enqueue below its minimum agent version, with
numeric comparison - string ordering misplaces 1.9.0.0 against 1.16.0. The
refusal is mirrored in the UI so a command slot is not spent on a call that
is guaranteed to fail.
Test isolation: create_app('testing') resolves instance_path to the real repo
instance/ directory, so agent upload tests write into the developer's actual
store. The new tests redirect both the store and package output into tmp_path
and include a guard test that fails if that isolation regresses. NOTE: the
pre-existing test_endpoints_agentupload.py still has this leak and deposits
1.5.0.0 and 1.6.3 into instance/siteagent/ on a full run - not fixed here.
Suite: 2834 passed, 11 skipped. The two failures (test_docs_publishable, and
a symlink scan needing Windows privileges) were confirmed pre-existing by
stashing this work and reproducing them.
The new table was declared as a model and created by migration 7d51, but never added to the explicit ownership map. test_owned_tables_match_declared_models caught the drift.
Not cosmetic bookkeeping: the per-plugin migration uses this map to decide which tables it owns, so an unlisted table is invisible to plugin-scoped baseline and prune. The prune path would have treated endpointlogcollections as unowned - collected logs surviving or vanishing on a schema operation that should have been deterministic about them.
Log Collector: a new collect_log action reads a plaintext log file from a
known path and returns its tail for viewing and download. Rides the existing
endpoints.updateapp grant rather than inventing one, since collecting a log is
the same trust level as pushing an application. Collected logs land in their
own table with per-asset retention of 20 instead of overloading the command
audit row, and the agent refuses UNC, traversal, wildcards, directories and
non-text content independently of the server check.
collect_log is POLICY_EXEMPT. Every existing Configuration Policy predates the
action and so cannot name it, and the intersection in ALLOWED_ACTIONS_FOR drops
anything unnamed - so every policied endpoint refused log collection outright,
including for admins, who saw a permission-shaped error for a grant they held.
Exemption from policy is not exemption from permission: the elevated grant
still applies, and a regression test asserts both halves so a future edit
cannot quietly loosen the grant while "fixing" the policy gate.
Script output capture: successful runs can return output, gated per script by
captureoutput. Failures always returned it; success never did, which left a
green badge as the only evidence a run produced.
Timestamps: Activity and Operator Reports rendered date-only because the shared
formatter's 'date' branch drops the clock time. Run Scripts additionally
labels its timestamp, since a bare time meant "completed" on a terminal row but
"queued" on an in-flight one - indistinguishable, and the wrong reading.
Agent 1.16.0. Minor: collect_log is a new action name, so an older agent asked
for it fails closed on the unknown-action default.
The wj merge brought in a half-built fleet alert feature: server-side UI
behaviour still being stood up over a delivery proof of concept. It left
five red tests. This clears them and fixes a real hole underneath.
The cross-machine guard never ran. acknowledge_fleet_alert read .assetid
off _resolve_asset_for_hostname, which returns the id itself, so the
check raised AttributeError on every call and answered 500. That guard is
what stops one endpoint acknowledging another's alert, and the collector
key is shared across the fleet, so until now any bay could clear an alert
on another machine's behalf. Compare the id directly.
Gate both alert routes the way the rest of the agent channel is gated.
They are collector-key authenticated, not JWT+role, because a shop floor
bay has no user session - the same shape as claim_next_command and
upload_operator_report. Declared in the authz allowlists with the reason,
rather than inventing a permission no agent could hold.
Settle the delivery contract ahead of the durable agent work. The poll
response is authoritative for a host: every alert that should be live on
it right now. An alert absent from the feed is the agent's instruction to
retire the pending toast, which is how a server-side close or cancel
reaches a machine already displaying one. Acknowledgement is terminal per
product decision, so a close after an ack is a no-op and the agent can
retry a queued ack safely. A toast that merely timed out is not an
acknowledgement.
Stop reaching past the plugin contract surface for the mailer. One import
was a plain redirect; the other pulled a private helper to pre-check
recipients, duplicating what send_alert already does. It also returned
early when no email recipients existed, which silently suppressed the
independent webhook leg - misconfigured SMTP was dropping Teams alerts.
Register NotificationAcknowledgement in get_models() and give the
notifications plugin ownership of notificationacknowledgements, then move
the expected head to notifications0006fleetalerts.
Add the coverage whose absence let the .assetid bug ship. Thirteen tests
over gating, authoritative-feed withdrawal, terminal and idempotent
acknowledgement, per-asset scoping, and the cross-machine guard. Five of
them fail against the pre-fix handler.
Remaining suite failures are pre-existing and local: the symlink scan
needs elevation, and the docs check wants files absent from this checkout.
A run_script command was judged solely on PowerShell's exit code, and
PowerShell treats most errors as non-terminating: it writes the error,
runs the next line, and still exits zero. The account-provisioning
script reported succeeded having provisioned nothing, and because no
output was captured there was no evidence to contradict the verdict.
Scripts now carry an errormode. Strict prepends $ErrorActionPreference
= 'Stop' so the first error ends the run and the exit code means what
it says. It is opt-in per script rather than global because bodies
authored against the permissive semantics rely on them - tolerating a
failing cleanup, probing with a cmdlet that errors when a thing is
absent, looping past a bad item - and would start aborting midway,
leaving endpoints half-configured. The default is unchanged, so every
existing script keeps running exactly as before.
The prologue is applied after digest verification, so the SHA-256 still
validates the bytes the server delivered.
stdout/stderr is captured on both paths: redirected streams for SYSTEM,
a transcript file under the ShopFloor profile for user-session runs,
since CreateProcessAsUser cannot inherit the service's pipes across the
session boundary. Output is always logged locally and returned to the
server only on failure, so routine output stays on the machine that
produced it instead of filling command history.
Also adds the optional per-script working directory, which the same
surfaces needed, and a collapsible explainer on the scripts page - the
dropdown is not self-explanatory, and an operator who does not know
about non-terminating errors has no reason to ever leave the default.
Agent 1.14.0. An older agent ignoring errormode keeps today's behaviour.
The script library was gated on applications.*, so anyone holding the
software-delivery grant could rewrite the bytes of a script that a separately
trusted operator later pushed to a bay. The push was gated on
endpoints.runscript; the CONTENT of what got pushed was not. An installer is a
vendor artefact, but a script here is arbitrary PowerShell this site wrote that
the agent runs as SYSTEM, so the write and the push must require the same
trust.
All six script routes now require endpoints.runscript, including the body
download - reading the bytes is not a lesser act than listing them. Core names
the permission as a string rather than importing it from the endpoints plugin
so the module stays importable with the plugin absent.
This tightens access and there is deliberately no backfill: the operator scope
reaches applications.create/edit, so operator-scope holders lose the library.
Granting them endpoints.runscript on upgrade would undo the change and hand
every operator remote SYSTEM code execution, since the same grant gates the
push. A test pins that no prebuilt scope confers it.
Adds the System Scripts settings page and card. The route requires the
permission rather than admin, because the grant is meant for non-admin
operators and the sibling admin gate would lock them out of the library the
grant exists for. The settings icon map gains terminal and key; the agent-key
card had been asking for key since it shipped and silently drawing a puzzle
piece.
The library had no test coverage before this.
Operator report screenshots are an uncontrolled capture of whatever was
on the operator's screen when they filed the report, so they can carry
part numbers, customer names, or other shopfloor content. The rest of
endpoints.view is uniform machine-collected telemetry. Reading a PC's
CPU load does not imply a right to read someone's screen, so the reports
and their images move to endpoints.operatorreports.
The image route is the real control point, not the panel: the panel only
decides whether a tab is drawn, while the bytes stay reachable by URL to
anyone who can guess an id. Both GET routes now require the new grant.
Migration 7d47 backfills the grant to everyone holding endpoints.view so
nobody loses a panel they were already using; tightening from there is a
per-account decision in the user editor. The grant is in the scope
denylist, so it is a per-user direct grant and Reporter cannot hold it.
Also adds test_user_editor_checkboxes_match_the_server_action_list. The
SCOPE_ENDPOINT_ACTIONS comment claimed a sync guard kept the Vue
checkbox list aligned with the Python tuple, but no such test existed.
The lists had already drifted: endpoints.operatorreports reached a
running container with no way to grant it through the UI. The guard
catches drift in both directions - a name only in Python ships the
feature dark, and a name only in Vue is worse, since the admin ticks a
box and believes access was granted while the server refuses to store it.
Soft delete left orphaned executables on disk and let a deleted name collide
with a fresh import, so delete is now a hard delete that removes the
associated files. The destructive paths (delete, revert-to-record) require a
typed OK because neither can be undone.
Installers keep their filename at import and nest per application/version,
which stops repeat imports doubling up. Version now comes from the same PE
detection the endpoints agent uses; when no version can be read the caller is
asked for one rather than silently storing a blank.
Applications also surface Version and a monitorable toggle, with launch path
and process name required only while monitorable is ticked. Fleet shows the
PC alias, matching PC Details.
The Applications page had no delete control even though the endpoint and
the API client both already existed, so the gap was UI-only.
The button is gated on applications.delete via hasPermission rather than
on the admin role, mirroring the backend guard: admins bypass it anyway,
and a non-admin role granted the permission should get the button instead
of a control that would only 403.
Deleting also withdraws the generated install policy. Delete here is a
soft delete, and the policy is a separate row that did not follow, so a
deactivated application kept installing itself across the fleet with
nothing on screen to explain why. A customised policy is still left
alone, on the same ownership terms as unticking the box.
Ticking "Required on all PCs" now generates a configuration policy that
installs the application on every non-server endpoint it matches, rather
than only flagging a software-compliance report row.
Because that save is a fleet-wide deployment, it requires explicit
acknowledgement. The gate is enforced in the API, not only in the form:
a confirmation dialog that can be sidestepped by POSTing directly is
decoration, not a safeguard. It fires only on the transition INTO the
required state, since re-saving an already-required application deploys
nothing new.
Generated policies carry a new 'autoapp' source. They stay editable, and
hand-editing one transfers ownership to 'local' so it survives a later
uncheck; 'shipped' remains read-only. Existing required applications are
not migrated retroactively - policies are only created on save.
Also adds the monitorable fields that the shipped-policy breakup will
build on, and repairs a mangled non-ASCII byte in copilot-instructions.md
that was failing the style gate.
on_install/on_enable fire only on a state transition, so a permission declared by a later plugin version never reached an already-installed site through them. Until someone ran flask seed permissions by hand the name did not exist, and any route guarding on it refused every caller. Settings already had sync_all_plugin_settings() on the upgrade path for exactly this reason; permissions now have the counterpart.
The scope relink is the half that actually bites. Creating the Permission row only makes the name exist - the prebuilt scopes derive their grants from the catalog, so a permission seeded without a following Role.seedscopes() belongs to nobody but admin. The plugin installs clean and its UI is invisible to every non-admin, which reads as a broken plugin rather than a seeding gap. Ordering matches flask seed permissions: seedscopes only links rows that already exist, so it runs last. Additive, so a site's deliberate role edits survive the deploy, and best-effort per plugin so one bad plugin cannot abort a deploy.
Test deletes an existing plugin permission, runs upgrade-all, and asserts both that the row returns and that it is granted to the technician scope. Verified it fails without the fix rather than passing vacuously.
Exports caller_is_role on the plugin authorization surface. Additive name, so a minor bump. Scoped personal tokens are always refused by it, matching require_role, which stops a plugin from accidentally letting a scoped token inherit role authority from its owner.
Registers the tables added by the endpoint lock and policy hold work (endpointlocks, policyactionholds) plus the endpoint health tables in the alembic template, so plugin migrations see them and the ownership guard does not report them as unclaimed.
Adds caller_is_role(), the body-answering counterpart to require_role, for routes that serve everyone but treat one class of caller differently. A scoped personal token is always refused here, matching require_role: scopes gate individual permissions, not roles. That invariant stops a leaked scoped agent token from inheriting override authority just because its owner is an admin - a scoped PAT is a narrower caller than its owner, admin included.
Frontend enqueueCommand now carries policyoverride with overridereason. Both are required together server-side and an override without a stated reason is rejected, so the caller must collect the reason before sending rather than discovering the refusal after the fact.
A newly created user held no roles and no permissions, so the sidebar filtered every nav item away and the account appeared broken. Adds three prebuilt scopes (Reporter, Operator, Technician) derived from the live permission catalog by verb rather than hand-listed, so a new plugin permission lands in the right scope automatically instead of silently going ungranted. users.view is granted to all three since a user who cannot see the directory cannot collaborate; users.create/edit/delete stay admin-only. Role.seedscopes() is additive and idempotent, seeded via flask seed permissions, and preserves custom grants on rerun.
Reporter guard: the auth model is additive across admin, direct grants and roles, so Reporter cannot be expressed as merely fewer grants - a direct endpoint-action grant would silently re-arm a read-only account. Rather than change the global permission predicate and alter every route's semantics, _check_scope_grants() rejects the invalid combination at write time on both create and update. The update path validates the effective post-update state, so demoting a Technician to Reporter is refused while a stale endpoint grant remains rather than quietly leaving a read-only user with write authority.
Validation runs before the User object is constructed. Assigning user.roles first put the relationship into the session autoflush path while the User was still unattached; nothing broke because the refusal returns before db.session.add(), but it left a trap where any query added below that point would autoflush a half-built user and surface later as an orphan row. Verified with -W error::sqlalchemy.exc.SAWarning: 25 passed.
Style check fixes:
- Replace unicode box-drawing characters with ASCII (|-- instead of ├──)
- Replace emoji status markers with [OK]/[FAIL]/[WARN] tags
- Replace arrows (→ with ->, ↔ with <->) and em-dashes (— with --)
- Add venv-quick to excluded directories in check script
- Exclude darkwing report files (findings.sarif, report.md)
Backup finalization fix:
- Fix finalize-restore endpoint to not claim the upgrade sentinel
- Endpoint now checks for pending upgrade and returns {queued: true}
- Leaves UPGRADE_SENTINEL for background watcher to claim and process
- Prevents inline Alembic runs that can outlast request timeout
Claude Code context:
- Add .claude/ directory with INDEX.md and QUICK-REFERENCE.md
- Add CLAUDE.md with project overview and conventions
- Helps Claude Code understand project structure and rules
Fixes failing test: test_the_endpoint_does_not_run_the_upgrade_itself
Fixes naming/style CI check violations
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Restore backup finalization functionality that was simplified in commit
b973561cb7:
- Add UPGRADE_SENTINEL, UPGRADE_CLAIM, UPGRADE_FAILED constants
- Restore _claim_upgrade() for atomic sentinel claim via os.rename()
- Restore _known_plugin_revisions() for plugin chain compatibility checks
- Restore _finalize_claimed() for upgrade execution with failure handling
- Update finalize-restore endpoint to use claim mechanism
- Update _is_compatible() to check plugin revisions and reject empty datarev
These changes prevent race conditions when multiple workers attempt to
finalize the same restore, and add compatibility validation for plugin
migration chains during restore operations.
Fixes 13 failing tests in test_shopdbbackups_finalize.py
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The test_shopdbbackups_finalize.py tests were failing because the backup
finalization code was missing the atomic claim/lock mechanism to prevent
race conditions when multiple workers try to run the post-restore schema
upgrade simultaneously.
Added missing constants and functions:
- UPGRADE_SENTINEL: .upgrade-requested marker from backup sidecar
- UPGRADE_CLAIM: .upgrade-inprogress lock to prevent races
- UPGRADE_FAILED: .upgrade-failed marker for failed upgrades
- _claim_upgrade(): Atomically claim upgrade using filesystem rename
- _finalize_claimed(): Run upgrade and handle success/failure
- _known_plugin_revisions(): Walk plugin migration histories
Updated finalize-restore endpoint:
- Uses atomic claim mechanism to prevent concurrent upgrades
- Supports re-arming failed upgrades for retry
- Returns proper response format (queued/retried flags)
- Checks if backup store is configured (404 if missing)
Updated _is_compatible():
- Checks plugin revisions when present in manifest
- Refuses sets from newer plugin chains
- Handles empty datarev as error case (not legacy fallback)
- Maintains backward compatibility with pre-datarev sets
These changes restore the functionality that was present in commit 993353d
but was simplified/removed in later refactors. The tests document the
critical race-prevention behavior that production backups need.
Fixes 13 failing tests in test_shopdbbackups_finalize.py
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add defense-in-depth validation to prevent path traversal attacks when
enabling/installing plugins via the UI or CLI.
Layer 1 - Input Validation (enable_plugin):
- Validate plugin names with regex ^[a-zA-Z0-9_-]+$
- Reject reserved names (., .., empty string)
- Blocks attempts like ../../../etc/passwd at entry point
Layer 2 - Path Boundary Check (load_manifest):
- Verify resolved manifest path stays within plugins_dir
- Defense in depth even if name validation is bypassed
- Compatible with Python 3.8+ (fallback for < 3.9)
Risk Assessment: LOW
- Requires admin role (already trusted)
- Multiple existing validation layers
- No known exploitation in the wild
Rationale: Defense in depth best practice. Admins are trusted, but input
validation should not rely solely on authorization checks.
Tested:
- Valid names pass (backups, my-plugin, plugin-name)
- Path traversal blocked (../../../etc/passwd)
- Reserved names blocked (..)
- Both validation layers confirmed working
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
enable_plugin() now auto-installs available plugins before enabling them.
This fixes the UX issue where clicking Enable on an AVAILABLE plugin would
fail with 'Plugin is not installed'. Now it installs automatically (runs
migrations, seeds permissions) then enables it.
Before: User had to manually install via CLI first
After: Enable button works on AVAILABLE plugins (one-click install+enable)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
wilm's backup implementation doesn't expose _backups_configured or
start_restore_finalizer - those were wj-specific. Since we took wilm's
complete backup system, remove the initialization code.
Details tab card layout
-----------------------
The Details tab sat on .content-grid, a two-column CSS grid. Grid packs
items into implicit rows, so every row is as tall as its tallest card and
align-items:start leaves the shorter one with dead space beneath it:
Warranty is short, Endpoint Status is tall, and Policy had to wait for the
taller of the two rather than sliding up. That read as a staggered layout.
Switched that one tab to multicol via a .details-flow modifier, which has no
rows at all - each card starts where the previous one ended, so the holes
close while cards still size to their content.
Multicol is deliberately NOT applied to .content-grid globally. The note by
that rule in style.css records why it was abandoned there: a column-span:all
child (.pap-span, used by table panels) overlaps the shorter column when the
two are unequal. That cannot happen on this tab - every panel in the details
group is a keyvalue or list render, and spansFullWidth() only returns true
for table (or a tabs panel holding one), so nothing in the subtree spans.
Monitors and Service/Process DO carry table panels and stay on grid.
The modifier is scoped to PCDetail, so the nine other detail views on
.content-grid are untouched.
Chosen over JS masonry on purpose. Masonry packs perfectly but has to
measure every card and re-measure on resize, on panel collapse and on every
async panel load - and these panels fetch after paint, so it would mean
continuous forced reflow on a page that already polls. This is pure CSS:
no measurement, no observer, no runtime cost. The tradeoff accepted is that
column bottoms can end at slightly different heights.
Cards carry break-inside: avoid so none fragments across the column break,
and the 25px spacing returns as a margin because multicol has no gap between
stacked items the way grid does.
IP Configuration card
---------------------
Removed: the data has no purpose on the page anymore. Taken out at the root
rather than hidden - the panel declaration, the /asset/<id>/ipconfig route
and the _IPCONFIG_COLUMNS constant all go, since a route with no consumer is
dead surface that still answers requests.
test_the_two_cards_do_not_duplicate_columns_beyond_the_address existed only
to police the history/config column split; with one card gone it guards
nothing, so it and its import go with it. The panel-count assertion in
test_endpoints_asset_panels_are_computer_scoped moves 15 -> 14, which is the
test doing its job and catching the removal.
IP History is unchanged and still carries the address list.
Not addressed here: three tzset() tests in test_endpoints_plugin.py fail on
Windows because time.tzset() is Unix-only. Verified pre-existing against
unmodified code; they pass in the Linux container.
Mapped drives stopped reporting because the user-context path failed silently
in three separate places, and none of them left a trace to read back.
- Probe: launch via -EncodedCommand instead of staging a temp .ps1 and running
powershell.exe -File. A missing or quarantined script file fails BEFORE any
statement runs, so the script's own try/catch never sees it and the only
symptom is exit -196608.
- Spool: pin to the ShopFloor profile. Resolving it through WTSQueryUserToken
is privileged, endpoint protection blocks it, and on failure the tray
(writer) and service (reader) silently disagree about the path.
- Spool: consume after a successful parse, so a stopped tray cannot keep
re-reporting stale mapped drives as current. Unparseable files are left in
place for the next cycle.
- Logging: user-session components cannot write to Program Files or to the
SYSTEM-owned ProgramData log directory, and the file sink degrades to a
silent no-op when it cannot create its directory. Tray and probe now log to
C:\Users\ShopFloor\ShopDB\Logs, installs transcript to ProgramData, and the
sink records a SinkProblem instead of swallowing UnauthorizedAccessException.
- Installer: provision the operator spool/log directories with ShopFloor:Modify
before the tray first runs, and stop the transcript via trap so a throw
cannot orphan exactly the failed install worth reading.
- Upload cap 64 -> 128 MB: the self-contained tray publishes at ~69 MB and was
rejected on SIZE, but the size check returns before the PE is parsed, so the
operator only ever saw a version-detection failure.
AGENT-CONTRACT.md gains a normative section defining the three security
contexts, their log destinations, the fixed spool path and consume-on-read.
Agent 1.8.0 -> 1.8.1, tray 1.1.0 -> 1.1.1 so the deployed build is
identifiable from a log line.
asset_panels() rewrites any group missing from ASSET_PANEL_GROUPS to 'details' before the client sees it, and Phase 3 never added 'serviceprocess' there. Both panels were relabelled on the wire, so the tab could not appear no matter how consistent the frontend was. That is why the tab was still missing after the schema fix - a second, unrelated cause wearing the same symptom.
Operator Reports moves out of System Monitors into a tab of its own: that tab is machine-collected health, this is a person reporting something, and the thumbnail grid dwarfed the fact cards it sat between. Full width, since it is alone in the tab.
panelGroups.spec.js claimed to cover four files and checked two; it now reads the server tuple as well. The plugin test's hand-copied set was a fifth copy and now derives from the server tuple instead.
A restore could load a database dump whose schema was older than the running
code. The manifest stamped schemarev from the code image on disk, so the
compatibility check compared the code against itself and always passed. A set
captured from a database at 7d37_import_batches_and_journal restored under
code at 7d41_drop_guest_tools_secrets, leaving the data behind the code: every
query on a table the newer migrations touched failed with 1054 Unknown column,
which surfaced as empty configuration-policy dropdowns and a failure to load
data on edit PC.
The authoritative signal is the alembic_version of the database that was
dumped, so manifests now carry datarev alongside schemarev, backup.sh reads it
from the database and warns when it has drifted from the image, and the gate
asks whether the running code knows that revision. Data behind the code is
allowed because it can be rolled forward; data ahead of the code is refused
because it cannot.
Rolling it forward has to happen somewhere with an app context. The backup
sidecar is mysql plus restic plus alembic only and has no Flask, so restore.sh
loads the dump and parks the run in an upgrading phase behind an
.upgrade-requested sentinel rather than declaring a success it cannot verify.
The api container owns the Flask CLI and already runs these exact idempotent
commands on every deploy, so a finalize endpoint there runs db upgrade plus
plugin upgrade-all and stamps the terminal phase. The UI drives it on seeing
the upgrading phase, so recovery needs no operator intervention.
The live simulation runs one codebase against itself, so its dump is always at
the same revision as the code restoring it and it can never reproduce this
drift. The gate therefore gets a standalone tester that builds throwaway
migration chains and dumps offline, invoked as a simulation preflight. It
caught a real bug in the legacy dump-parse fallback, where the revision was
matched loosely enough to return the alembic_version table name instead of the
value it holds.
The watcher scripts now write live status and append-only run history onto the shared volume; the api exposes them via /shopdbbackups/status and /history. The restore modal shows an indeterminate spinner with phase labels, and two cards list backup and restore run records. Timestamps render in UTC to match the Zulu-stamped set ids, the history directory is excluded from the set list, and a stale-status guard stops a previous run's done phase from collapsing the spinner on the next restore.
alembic.ini is a Flask-Migrate config with no script_location, so ScriptDirectory.from_config raised and _current_schema_revisions swallowed it and returned no heads. Every set then failed the subset check and was wrongly blocked as taken on a newer schema than the running code. Set script_location to the migrations directory - the same versions tree the manifest scans - so live heads resolve and compatible sets restore.
Backups are per-site and not part of the base stack, so they ship as a separate compose overlay with an isolated restic REST server the app can only reach through the backup utility (no shell or db creds). The sidecar image trusts the corporate proxy root CA at build time so its Oracle yum, github, and PyPI fetches succeed behind TLS interception; shell scripts are pinned to LF via .gitattributes so the container entrypoints do not die on a CRLF shebang.
