39aaa5d0cb1af3dfed2951ffef4098511df222ca
570 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
39aaa5d0cb |
Ship the API docs the container never had, and theme them like the product
/api/docs was broken in every Docker deployment, including the air-gapped ones.
docs.py resolves openapi.json and llms.txt from the repo root, and the image
copies shopdb, plugins, migrations, scripts and wsgi.py but never docs. The page
rendered the Redoc shell and then answered 404 fetching its own specification.
Only the two files the app actually serves are copied; the rest of that tree is
prose with no business in a runtime image.
The theme was invented colours that matched nothing. It now maps the product's
own tokens, and reads the theme the way the app does: the application stores an
explicit light/dark choice and stamps data-theme, so choosing light on a
dark-desktop kiosk used to give a dark documentation page. The operating system
setting is the fallback, exactly as it is in the app.
Light-mode primary, success and error are a shade darker than the product
tokens, because those tokens are designed as fills and fail contrast as text on
the nested panels where type names and required labels sit. Measured with a
browser harness rather than judged by eye: every text node in both themes now
clears WCAG AA, from an initial 5603 reported failures - most of which were the
harness itself ignoring alpha, which is its own lesson about trusting a number.
Four Redoc elements carry hardcoded colours with no theme key and keep targeted
rules, selected by tag and attribute rather than by generated class hashes that
change on every build.
The build now compiles the Python. Docker never imports the application, so a
syntax error sails through every layer and surfaces as a crash-looping
container; that happened, four restarts, while working on this file.
Redoc's upstream licence file was never vendored although its bundle header
points at it. The placeholder added here says exactly that rather than inventing
the third-party notices it is supposed to carry, and the dead redocly.com badge
is hidden: the licence wants the notice in the distribution, not a rendered
link that a site with no internet cannot follow.
(cherry picked from commit
|
||
|
cproudlock
|
be78e45320 |
Rework the first-run setup wizard around what a site cannot run without
The wizard configured five of the product's 129 settings. A site that finished
it still had no timezone, no authentication, no mail server, no backup
destination and no endpoint agent policy, and nothing told the operator so.
Six steps now: Administrator, Site, Authentication, Features, Mail and Backups,
Review. Everything else moves to a standing setup checklist at
/settings/setupchecklist, because a wizard that asked for all 129 settings would
be worse than one that asked for none.
Two new endpoints back it. /setup/checklist reports twelve areas as
ok/partial/todo/na with a link to the page that fixes each, and /setup/health
reports account counts plus whether a plugin toggle has reached every worker.
Both take care not to lie:
- Several settings are SEEDED non-empty (the shipped logo, a placeholder floor
plan, every identifier field), so "is it set" compares against the shipped
default rather than emptiness.
- plugin.endpoints.sharedkeyallowed DEFAULTS TO ALLOWED when its row is absent,
so an absent row is reported as the fleet-wide key being accepted. Reporting
it as refused told an admin the opposite of the truth about the credential
they are migrating away from.
- A plugin is held at "starting" until several DISTINCT workers report it live.
Counting replies instead let one recycled worker answer four times and declare
a change complete while other workers still served the old routes.
Recommended plugins are declared in each plugin's manifest rather than listed in
the wizard, so adding a plugin cannot leave a stale list in another file.
Also fixed: the logo preview was invisible, because the image borrowed a class
that hardcodes a white background while the shipped logo is white; and signing
in did not re-read setup state, so an admin already browsing was never steered
to the wizard and a hard refresh was the only cure.
(cherry picked from commit
|
||
|
cproudlock
|
9b7da9f2d2 |
Make a plugin toggle take effect without a manual restart
Enabling a plugin could not make its routes live. Flask refuses
register_blueprint once the app has served a request, and each gunicorn worker
holds its own app object, so registering in the worker that served the toggle
would leave the others answering 404 for the same URL.
Worse, the window before the restart was a window in which workers disagreed
about reality. Four defects, all of which answered 200 while going wrong:
- Enabling a not-yet-installed plugin ALWAYS 500'd on a running app:
install_plugin ran the migrations, updated the registry, and only then called
register_blueprint and hit Flask's assertion. The toggle half-applied and
reported failure.
- Enabling one plugin could hang the entire site. @jwt_required does SELECT
users, so the request's transaction sits open holding a shared metadata lock;
the plugin's migration runs on its own Alembic connection and needs an
EXCLUSIVE lock on users for a foreign key. Each waits on the other, and MySQL
grants metadata locks in request order, so every later login queued behind the
pending exclusive. Observed twice as a total outage from one plugin toggle.
- Each worker rewrote the WHOLE registry file from a snapshot taken at boot, so
concurrent enables erased each other: seven enables persisted three, with
HTTP 200 for all seven.
- Dependency checks read that same stale snapshot, refusing a plugin whose
dependency another worker had just enabled.
Routes still require the process to restart; what changes is who initiates it.
Each worker fingerprints instance/plugins.json at boot and exits gracefully when
it changes, and the supervisor replaces it. That is the primitive gunicorn's own
--reload uses. Recycles are staggered per worker so the pool rolls rather than
dropping at once, and the exit is scoped to gunicorn: on Windows sys.exit() in a
daemon thread ends the thread rather than the process, so IIS keeps the manual
restart until that path is proven on the Windows VM.
The registry now merges under an exclusive lock and replaces atomically, saves
only the entry being changed, and re-reads before any decision is made from it.
ADR-020 records the decision; ADR-013 is amended where it said the UI must tell
the operator to restart. The restart requirement and the honesty requirement
both stand - only the manual step is removed, and only where it is proven.
(cherry picked from commit
|
||
|
cproudlock
|
0eed80f3b9 |
Create endpointlogcollections in the chain that owns it
A fresh install could not build its schema. Core revision 7d51_endpoint_logcollections creates endpointlogcollections with a foreign key onto endpointcommands, but PLUGIN_TABLE_OWNERS lists both tables as owned by the endpoints plugin, and only the plugin's 0002 revision creates endpointcommands. Core migrations run before any plugin chain, and a plugin chain only runs once that plugin is enabled, which cannot happen before the schema exists. So there is no ordering in which the core revision can succeed on an empty database. MySQL rejects it with 1824, 'Failed to open the referenced table endpointcommands'. That is the first command the README gives an adopting site, on the branch that exists for sites to adopt. Nothing caught it because the suite runs on sqlite:///:memory: and SQLite accepts a foreign key onto a table that does not exist. The failure is MySQL-only and therefore invisible to every test. Fix: - endpoints revision 0025 creates the table in the plugin chain, after 0002 has created its foreign key target. Guarded, so it is a no-op on sites that already applied the core revision. - 7d51 skips when endpointcommands is absent, which is exactly the fresh install case 0025 then covers. Its downgrade is guarded to match. - EXPECTED_HEAD_REVISION records the new endpoints head. Verified against real MySQL 8.4, not SQLite: on an empty database the README sequence now completes, 7d51 stamps without creating anything, and enabling the endpoints plugin builds endpointlogcollections with endpointlogcollections_ibfk_2 pointing at endpointcommands. |
||
|
cproudlock
|
bd6e736798 |
Say on the front page that a site runs containers
The README gave Windows/IIS and Docker equal billing, and led with the installer under "Fast path (Docker)" - which reads as though the slower, more official route is the .exe. That is backwards now: Docker is what a new site gets and IIS is the legacy path, supported for the sites already on it. Changed where a reader looks first: two lines under the title saying a site runs containers, Distribution naming the installer as legacy, and the Docker section retitled "Standing up a site" rather than a fast path. The venv instructions are marked development-only, because they are for working on ShopDB rather than running it. Nothing about the IIS material was removed. It is what production runs today. |
||
|
cproudlock
|
be1eb9b366 |
Let the API docs follow the system theme
The app has a dark theme. /api/docs was the one page that stayed white when the rest of the site went dark, which is exactly the page somebody has open beside the app while they work. Redoc does not read prefers-color-scheme, and the <redoc spec-url> element it was using takes no theme at all - so the page now initialises through Redoc.init, which does. Light keeps the default look; dark names the colours Redoc otherwise derives from the background and gets wrong: body text, the sidebar, and the right-hand sample panel, which came out dark-on-dark. It re-initialises on a live scheme change rather than restyling, because Redoc bakes its colours into styled-components at init - a CSS override reaches the chrome and misses the content. The listener has the addListener fallback for the older MediaQueryList, since a shop-floor kiosk browser is exactly where this gets read. Nothing new is loaded: same vendored bundle, same spec, no font, no CDN. A test pins that, because theming is the kind of change that quietly introduces one. The try-it page is left light. Swagger UI 5 ships no dark theme, so it would mean vendoring a community stylesheet that fights upstream on every bump - and it is a development tool that is off in production anyway. |
||
|
cproudlock
|
76f085e224 |
Serve a spec that knows where the instance is mounted
/api/docs on the production site handed out
https://<host>/api/apitokens for an app that lives at /shopdb. Copy that URL
and it 404s; Swagger's try-it fired at the same wrong address.
The mount is a DEPLOYMENT fact and the spec is a BUILD artifact.
gen_openapi.py writes servers: [{"url": "/"}] because at generation time nobody
knows where the app will be hung, and wsgi.py only moves MOUNT_PATH from
PATH_INFO into SCRIPT_NAME when the process starts. Served as written, the file
was right for a root install and wrong for every subpath one - which is how
production runs.
The endpoint now sets `servers` per request from request.script_root: empty at
root, /shopdb under the subpath method, and correct behind a proxy that sets the
prefix. The paths themselves stay mount-free, because the prefix belongs in the
server entry and duplicating it into each path would double it.
The file is about a megabyte, so it is parsed once and cached by mtime, and each
request gets a shallow copy with only `servers` replaced - two requests arriving
on different mounts cannot race on one dict.
A test pins both cases and the fact that paths stay unprefixed. Planting a wrong
server URL fails it.
|
||
|
cproudlock
|
f9d6b57f0f |
Move the documentation to the line that ships, and correct it against the code
The docs lived on one lineage and the fleet code on another. That is why six pages described a shared fleet key as how an agent authenticates, why COLLECTOR-AUTH had to warn that the per-PC credential code was not in the repository it sat in, and why ADR-019 recorded a decision whose implementation was somewhere else. Documentation that cannot see the code it documents goes stale by construction, and no amount of editing fixes that. So the corpus moves here: 84 files, 56 pages, 20 ADRs, the proposals, INDEX.md, llms.txt and both gates. Filenames are unchanged, because a wiki page name is derived from the filename and a rename breaks every link anybody saved. THE POINT OF THE MOVE IS THE CORRECTION IT ENABLES. Every caveat that said "not in this tree" was false the moment the files landed beside the code, so each was rewritten from what is actually here: - COLLECTOR-AUTH gained the real enroll request body, the two bounds on the unauthenticated route (per-IP rate limit, pending cap with stale rows dropped before a real PC is refused), and the rejected pending reason that binds to the endpoint identity rather than the nonce. - ADR-019 now describes an implementation that is present, including its tests, and says plainly that what remains is the FLEET migration, not the code. - proposals/device-identity claimed phase 1 was unbuilt. plugins/computers declares deviceid on the collector schema and prefers it over the older fallbacks. Corrected in the page, its summary and the index. WINDOWS/IIS IS NOW DOCUMENTED AS THE LEGACY PATH, per the product decision, and carefully: it is what production runs today. Every Windows page opens by saying it is supported, not for a new site, and where a new site should go instead. No procedure was cut - an operator on an IIS box at 2am needs these pages to work exactly as well tomorrow. MIGRATE-TOPOLOGY stops being a footnote and becomes the off-ramp, which is why it nearly doubled: what was there did not match how the Docker stack really works here. The Docker material that already lived on this line was kept and given ownership rather than buried under the ported Windows-centric prose: DEPLOY now describes the four services, the migrate service that runs the upgrade chain before api starts, TLS terminated by nginx, and the plugin count corrected from thirteen to seventeen - a number both lineages had wrong. ROADMAP is rewritten from evidence rather than memory: what has shipped, read out of the changelog, the ADRs and the manifests, and ahead of it exactly two items - finalize a distributed release and polish the setup wizard. PROJECT-MAP is generated and was stale; the GENERATOR was what was wrong. --check passes now. API-REFERENCE is retired to a pointer. /api/docs serves 573 operations with derived auth, query parameters and harvested examples, and a hand-written page competing with that is a liability. The file stays so its wiki page and its inbound links survive. AND THE MOVE TURNED ON A SCRUB THAT HAD NEVER RUN HERE. test_docs_publishable keys on docs/*.md, which this line did not have, so it skipped - its own comment predicted what would surface. Porting the corpus activated it and it found 48 lines that must not reach a published wiki: branch names abbreviating the site, assistant-tooling paths, the site name in two deployment pages, and an employee ID in a Windows path that the scrub does not even look for. |
||
|
cproudlock
|
68567d9a27 |
Add a try-it API page, and stop the docs page reaching for the internet
/api/docs renders the spec with Redoc, which is read-only by design. The thing people mean when they ask for Swagger is firing a request from the page, so Swagger UI now sits beside it at /api/docs/try, reading the same spec - one contract, two renderers. It is OFF outside development and 404s rather than rendering a disabled page: a production box holds live fleet data, and a docs page that can POST to it is a different object from a docs page that cannot. Vendored, not linked. swagger-ui-bundle.min.js and swagger-ui.css are served from staticdocs/ beside the existing Redoc bundle, because production has no internet. validatorUrl is null: left unset, Swagger UI hands the spec to the public swagger validator to render a badge, which here is a request that cannot succeed and elsewhere would ship the internal API surface off the box. THE VENDORED REDOC BUNDLE WAS ALREADY REACHING THE INTERNET, found while proving this page does not. redoc.standalone.js fetched cdn.redoc.ly/redoc/logo-mini.svg for the "API docs by Redocly" mark, so every load of /api/docs on the air-gapped server has been waiting on a request that cannot complete. Replaced with an inline transparent GIF. Three gate holes closed, each one a dodge that worked: - the page is built by interpolating url_for output into a JS string literal, unescaped. SCRIPT_NAME is proxy-controlled and ProxyFix honours X-Forwarded-Prefix, so a mount path carrying a quote closed the literal and executed what followed. The spec URL is now json.dumps'd and the rest HTML-escaped; the injected quote renders as \" and is inert - the "no internet" assertion on the page scanned for https?://, so <script src="//cdn.jsdelivr.net/..."> - the form a copy-paste from any CDN's own documentation produces - matched nothing and passed - the same assertion on the vendored files was a five-host CDN allowlist. An upstream bump to any other host walked through it, and the one real leak in that directory was on no list. It now matches the SHAPES a browser fetches - @import, css url(), src/href, @font-face - whatever the host, ignoring the data: URIs this CSS is full of And the Redoc page's link to the try-it view is built with url_for, which raises BuildError when those routes are not mounted - every production box. Made unconditional it would 500 the docs page in exactly the deployment nobody tests locally, and no test caught that. One does now. |
||
|
cproudlock
|
5aa748cab5 |
Publish examples and query parameters, and stop the harvest leaking secrets
The spec described 573 operations with a schema each and almost nothing a
reader could copy: 12 operations carried an example, and no operation described
a query parameter at all, because a Flask handler's query string is invisible to
url_map. A route whose real interface is ?code= documented nothing.
Both are now derived from evidence rather than written by hand.
QUERY PARAMETERS come from each view's AST - request.args.get, the subscript
form, getlist, and this repo's own helpers - and a read that does not resolve to
a literal name is not emitted. 0 to 87 operations, 350 parameter objects. The
enumerations some handlers validate (sort=assetnumber|hostname) are deliberately
NOT published: deriving the accepted set reliably needs more than a literal
read, and a wrong enum is exactly the invented contract this work avoids.
EXAMPLES are harvested from the test suite, which already sends real bodies and
asserts real responses, so they are transcripts rather than inventions. 12 to
about 324. The harvest is re-runnable and optional; generation works without it.
THE HARVEST PUBLISHED TWO REAL CREDENTIALS, and that is the important part of
this commit. A bootstrap token and a kiosk setup code reached docs/openapi.json
inside URL values - a file served at /api/docs on production and copied to a
wiki. Neither scrubber layer could see them: the keys are 'scripturl' and 'url',
which name no secret, and the opaque-value rule excludes ':' and '/' on purpose
so that example URLs survive. Nothing was committed.
Four changes came out of it:
- a URL-credential rule that scrubs the parameter VALUE and keeps the URL, so
?t= is redacted and ?page=2&perpage=50 still reads as an example
- redaction runs on the way OUT as well as on the way in. Harvested examples
live in the inventory and a refresh preserves them, so a scrubber fixed after
a harvest protects only future harvests while the stored copy keeps
publishing. The published file can no longer carry a secret the current rules
would catch, whatever the inventory holds
- the leak gate now holds its own patterns instead of importing the generator's.
Checked with the scrubber's own rules it could only ever assert that redact()
ran, and it was green while both tokens sat in the file. It now knows the
product's token prefixes, a URL-credential shape, and a crude entropy
backstop. Replanting the exact credential that shipped fails it
- a DELETE only declares a body when the HANDLER reads one, derived from its
AST. A transcript proves the client sent something: the suite posts {} to
routes that ignore it, and that published "this route reads a JSON body" on
DELETE /api/endpoints/agentkey, whose handler never calls get_json.
249 operations still carry no example, because no test exercises them with a
JSON body. The honest fix is a test, not a hand-written example, and the next
harvest picks each one up for free.
|
||
|
cproudlock
|
f16b463736 |
Stop a settings test depending on which tests ran before it
CI failed one test that passes here: "no such table: settings", seeding a
Setting into a database with no tables. The test was not wrong about the
product, it was asking the process-wide plugin_manager to do the work.
create_app() re-points that singleton, and eight test modules build their own
app, so it belongs to whichever ran last on the xdist worker.
_seed_plugin_settings then opens `with self._app.app_context()` on a different
app's in-memory SQLite. A sibling symptom is the registry coming back None
("'NoneType' object has no attribute 'get_all'"). Both land on whichever test
follows, never on the module that caused them, so they move with the split -
green on one machine, red on another.
The test now builds a manager for itself, sets _app and _db, and calls
_seed_plugin_settings directly. That is the routine `flask plugin upgrade-all`
runs, so what is under test has not changed, and the assertion no longer depends
on worker order. init_app is deliberately not called: it re-registers the plugin
blueprints and a second call on one app raises "The name 'backups' is already
registered".
This does NOT fix the underlying fragility, and that is a separate decision.
Running tests/test_plugins serially today gives 115 failed and 542 errors, from
the same cause; under -n auto it is nearly invisible because modules land in
separate processes. The repair is to stop create_app mutating a module-level
singleton, and it belongs in its own change rather than riding along here.
|
||
|
cproudlock
|
69a7997dc4 |
Build the plugin set a real site runs, and write down what shipped
The lean-build job proved the delete-a-plugin guarantee with two plugins and four hand-picked marker components. It now builds a real twelve-plugin set, and the assertions follow a rule instead of a hunch: one component from EVERY omitted plugin must be absent, one from EVERY chosen plugin must be present, and the backend tree is checked both ways. An omitted set that grows without its check growing is how a leak ships. CHANGELOG had not been touched across the whole range, so the largest security-behaviour change in the product - per-PC endpoint credentials, the shared-key refusal for enrolled PCs, server pinning, agent 1.20.1 and tray 1.7.1 - had no release record. An [Unreleased] section now carries it, written from the commits rather than from memory. CLAUDE.md and the agent quick-reference stated that ADR-016 bans storing credentials in the registry. It does not, and this fleet depends on HKLM for its collector key and server pins: the next person following that text removes them and silently breaks reporting and pinning on every un-enrolled PC. Corrected, along with two other statements that were false on this branch - the database is MySQL, not PostgreSQL, and column names are lowercase concatenated, not snake_case, which is the opposite of what the gate enforces. The wiki drafts are updated against the code as it stands, each old finding given a verdict with a file and line rather than left as an assertion from a week ago, and a new page records the ADR conformance result - including that the ADR set itself is not on this branch, so nothing here enforces it except the naming and style script. They are also scrubbed of the things that must not reach a published wiki: the site name, the internal git server, developer home paths and the assistant tooling. They now pass the repository's own publishability patterns, which is what the scrub is for. |
||
|
cproudlock
|
8fc6df8f0a |
Describe the API this branch actually serves
docs/openapi.json is served live at /api/docs on the production box and documented none of the routes added since 2026-09-11. It is generated from docs/api-inventory.json, which is not tracked on this branch, so the generator had no input - and tests/test_api_inventory_parity.py skipped itself for the same reason. No route here had ever been checked against the published contract. The inventory is now generated from this branch's own url_map rather than carried over from elsewhere, with each route's auth derived from the decorators and gates on its view rather than guessed. A route whose gate cannot be classified is marked for review, not quietly called public. gen_openapi.py can refresh it from the live app, so this does not rot again, and human-authored prose survives a refresh. The parity test now RUNS instead of skipping. The thing that mattered more than the missing pages: nine operations were published with "security: []" - the machine-readable claim that no credentials are needed - while their handlers require one. Two of them serve the bootstrap installer and the agent package. A generated client, a firewall plan or anything else reading the spec would have been told those are anonymous. Required auth is now expressed as required, and a token-or-network gate is expressed as optional auth with the network half named in the description, which is what it actually is. A test re-derives auth from the live app and fails when the committed inventory disagrees, because the parity check only ever compared route lists: a wrong auth string passed everything. test_openapi_spec.py asserted that optional auth was the majority of the surface, a premise the guest-role and permission model has made false. It now holds the property that matters - an optional-auth route is expressed as optional - rather than a count that was true once. gen_openapi.py also loads .env before building the app, like every other script that builds one, or a plain python run falls back to the hardcoded localhost URL. |
||
|
cproudlock
|
85f69038d8 |
Check the things the gates were reporting green without checking
Four gates that were not gating. NO MIGRATION COLUMN NAME HAD EVER BEEN CHECKED. The naming gate matched db.Column attribute assignments only, so every column a migration creates has bypassed the lowercase-concatenated rule since the gate was written. It now reads migrations too, in the shapes they are actually written: sa.Column in create_table and add_column, alter_column's new_column_name, the batch equivalents, the keyword form, and a literal with a leading capital or a string prefix. Run against the tree it finds nothing, which is now a fact rather than an artefact of the pattern. THE ADR-015 SITE-LITERAL SCAN DID NOT REACH THE C# AGENT, twice over: agent/ was not in the directory list, and a file-level --exclude alongside the --include list changes which files grep matches at all, so the .cs include was not doing what it looked like it was doing. A hostname or base URL compiled into the agent would have shipped to another site with no CI signal. NO PLUGIN MIGRATION WAS CHECKED FOR TOUCHING SOMEBODY ELSE'S TABLES. ADR-008 says plugin chains own plugin schema and the core chain is reserved for core tables; the endpoints chain broke that, added and then dropped columns on the core assets table, and nothing would have stopped the next one. The scan walks every plugin chain, resolves each DDL op to its table, and fails when that table belongs to core or to another plugin - including through op.execute and through a bind, which is this repo's dominant idiom and the shape a check keyed on the receiver name would have missed entirely. Statements are classified one at a time, so a leading DML verb cannot hide trailing DDL. The two historical crossings are allowlisted by plugin, file, op, table AND column, so the shipped history stays green and anything new goes red. AND THE GATE RUNNER ITSELF PASSED WITHOUT RUNNING ANYTHING. In a checkout with no venv, `python` is often not on PATH; the pytest line ends in `|| true`, so the run died 127, the report was one command-not-found line, every gate matched nothing, and it still printed "All gates ran." and exited 0. That exit code is what everything else trusts. It now refuses to run without an interpreter that can import pytest. Recorded rather than fixed, with the reasoning in both files: the publishability scrub keys on docs/*.md, so prose that lives only in docs/wiki-drafts/ does not turn it on. Whether those drafts are internal or headed for publication is a question about what they are for, not a bug to close by rewriting them. |
||
|
cproudlock
|
2f98188feb |
Let a site close the legacy collector routes, once it can afford to
Turning "Shared key allowed" off refused the key on the endpoints plugin and left four core collector routes open to it. Anything still holding the old key could keep rewriting logged-in user, last boot time, serial number and installed app inventory for ANY hostname, while the admin believed the shared key was off. Data integrity only - no command execution, no credential issuance - but it is a hole in a boundary the product advertises as closed. It cannot simply be closed. Both GE-Enforce reporters still authenticate with that key, so refusing it fleet-wide would stop asset and printer reporting everywhere, and on an air-gapped site mid-migration that is the wrong order of operations. collector.legacysharedkeyrefused therefore DEFAULTS TO TODAY'S BEHAVIOUR, and its description names exactly what turning it on breaks and when it should be turned on: after the reporters move to per-PC credentials, not before. A legacy route used with the shared key now logs a warning, so a site can find out who still depends on the key before flipping anything. On a stock Windows deployment that warning may name the proxy rather than the caller; the token prefix in the same line is the identifier to go by. The switch is seeded with the core defaults and has a control on the authentication settings page. Undeclared, it was a row that did not exist, and this UI has no generic key/value editor, so seeding alone would have rendered nothing for an operator to click. Two contract declarations corrected while in here: notifications.alertscope was declared db.JSON in the model while the migration that creates it ships TEXT, and those migrations state the constraint in their own docstrings: no JSON columns. The model now matches the shipped column rather than the schema being bent to the model. usb and printedparts both import a name that arrived in contract 0.35.0 while declaring core_version floors of 0.1.0 and 0.16.0, so the loader would have accepted them on a core that cannot satisfy them - an ImportError at blueprint registration instead of a refusal with a reason. Both floors now say what the plugin actually needs. |
||
|
cproudlock
|
7ac1ee3077 |
Refuse a kiosk caller wearing an address that belongs to the path
Both kiosks admit a check-in or check-out with no login when the caller sits in a configured network. The subtraction that makes that safe was missing. A reverse proxy normally sits ON the plant network a site enters in the allowlist. When the proxy forwards nothing - misconfigured, or something bypassed it - the request arrives wearing the proxy's own address, which matches the allowlist, while carrying no evidence whatever about who sent it. That is the one case that has to refuse, and it was being admitted. The previous fail-closed test only looked closed because the proxy in it happened to sit outside the range. Addresses that belong to the path are now subtracted before the allowlist is consulted: loopback always, since no site's kiosk is the server itself, plus whatever the site declares as its proxies. USB and printedparts get the same treatment - they had diverged for no reason anybody chose. Three more, same surface: A range that admits everybody can no longer be saved by accident. 0.0.0.0/0 and ::/0 are refused outright, anything wider than a site plausibly needs is refused unless an explicit override is set, and the refusal says why. The kiosk token no longer travels in a URL. The provisioning link carries a single-use setup code redeemed on first open, so the token stays out of browser history, referrers and proxy logs. The route is declared public deliberately: the secret is the code, the way a password-reset link is the secret. The badge-to-name lookup is bounded. It answers without a login and is now reachable from a whole network rather than one screen, so a source that keeps missing gets refused rather than being allowed to walk the directory. On by default, budget and kill switch are settings. All four new settings carry controls on their settings pages. A setting that can stop the kiosk, or widen it, must not be reachable only by hand-editing rows. The installer's client-IP advice is corrected: it described the opposite of what the shipped waitress flags do, and named only GE-Enforce as the thing that breaks when the rule is absent, when the two kiosk allowlists depend on it too. |
||
|
cproudlock
|
987f5c0d0f |
Stop the one-line install handing a standard user a way in
The bootstrap command staged its script into C:\Windows\Temp and then ran it elevated. That directory grants Users create-file with CREATOR OWNER full control, so somebody without admin rights can pre-plant the exact path the elevated run is about to execute. It now stages under Program Files, creates the directory with an explicit DACL when it is missing, and refuses to run a staged file that is not owned by an admin principal. A failed hardening is loud rather than silent: the run stops instead of carrying on with a file it could not protect. The token moved out of the query string into a header. In a query string it lands verbatim in the reverse proxy and IIS logs, which is a credential written to a file nobody thinks of as a secret store. The query form is still accepted, because bays provisioned before this change are holding install commands that use it, and it logs that it was used - without logging the token - so a site can see who still needs a re-issued command before turning it off with plugin.endpoints.bootstrapquerytoken. Two smaller things this also closes: A request with no token at all answered 400 naming the header, which told an anonymous caller the state of that switch and contradicted the module's own promise that anything unusable answers 404. No token is now 404. The 400 is kept for a token genuinely presented in the query form while the query form is off, which is a caller who needs to know their command is stale. The rollout documentation was republishing the vulnerable one-liner verbatim, so a tech following the runbook reinstated exactly what this fixes. Both documents now carry the current command. Also corrects the operator-facing copy for the token, which named a query argument and a header that do not exist: it is ?t= and X-Endpoint-Bootstrap-Token, per bootstrap.py's own constants. Anybody following the old text got a 404 with no reason given. Both switches added in this range are now declared with the other plugin settings and reachable from their settings page. Read through Setting.get but never declared, they were rows that did not exist until someone wrote one by hand, which on an air-gapped box means the feature was unreachable. |
||
|
cproudlock
|
3f80425f30 |
Make an admin's refusal stick, and let a revoke actually deny
Three holes in the enrollment path, all of the same shape: a decision the admin made did not survive contact with a PC that simply asked again. REJECTION WAS UNDONE BY A NEW NONCE. A rejected request re-armed itself as a fresh cycle when the PC came back with a different nonce, and with the shared key and auto-enroll on it was then issued the credential the admin had just refused. Rejection now binds to the endpoint identity - hostname plus BIOS serial - not to the nonce, so the PC stays refused however often it reinstalls. An admin clears it deliberately, through the approve path that already exists; a genuinely different PC is untouched. THE PENDING QUEUE COULD BE FILLED UNTIL IT REFUSED EVERY NEW PC. /enroll takes no auth by design - a bay that has never spoken to the server has nothing to authenticate with - so anything that can reach the server could sit at the cap and lock out real enrollments fleet-wide. Rows nobody has touched for seven days now age out before the cap refuses anyone, and the count of what went is audited. Per-identity dedup and the cap itself were already there and are left alone. REVOKE DID NOT STOP A LIVE AGENT. It fell back to the shared key and the server admitted it again, so revoking a compromised PC changed nothing while that key still worked. plugin.endpoints.revokeddeniessharedkey makes a revoked host refused on the shared key too. That switch DEFAULTS OFF, and that is the important part. Turning it on would silence any bay revoked for a key rotation, and would break the documented revoke-then-rollback path to an older agent - on an air-gapped site, mid migration, that is a fleet outage. A never-enrolled bay is never affected either way: with no credential row at all the shared key still works, which is the whole un-migrated fleet. The checkbox sits next to the other two enrollment switches and says what turning it on costs, because a setting that can stop the floor should not be reachable only through the API. |
||
|
cproudlock
|
fe4f44f855 |
Lock the shared fleet key away from the people standing at the bay
The generated install.ps1 wrote the collector key to HKLM\SOFTWARE\GE\ShopDB and left it at the DACL the key inherits, which grants BUILTIN\Users read. Any operator logged on to any bay could read the fleet key, and that key still authenticates an un-enrolled PC: it claims queued commands and downloads script bodies and installer packages. The per-PC credential work exists to retire that key, but it stays until every consumer is off it, so meanwhile what matters is who can read it. Protect-HklmShopdbKey breaks inheritance, DROPS the inherited entries rather than copying them - copying is what silently preserves the Users read - and states SYSTEM and Administrators by SID in both registry views. By SID, so a localised Windows still matches. Everything that legitimately reads this key is SYSTEM (the service, the GE-Enforce reporters) or an administrator (the adoption tool, Set-ShopDBServerPins.ps1). The tray reads no registry at all - it talks to the service over the named pipe - so nothing loses access. It runs for the key-less package too, because ServerPins live in the same key and the registry is authoritative for pinning: a standard user must not be able to rewrite the pins either. A bay that cannot take the ACL still installs. It says so, loudly, rather than failing the install or staying quiet, because an agent that is not installed is a worse outcome than a key that is still readable on that one machine. Note what this does NOT fix: bays that already carry the key keep their inherited DACL until they are reinstalled or upgraded through a package built after this change. |
||
|
cproudlock
|
35d4fcd41c |
Report whether a returned drive was wiped and scanned
The USB audit answered who had which drive and when, but not whether it came back wiped or virus-scanned. That is the question the record is kept to answer: removable-media custody is CMMC evidence, and custody without the state of the media on return is half the story. Both stores already held the values - the external checkinoutlog in sanitized and scanned_viruses, the app's own usbcheckouts in waswiped and scannedviruses - and neither mode selected them. Both now report them, in the JSON, the CSV and the column list, so the two directory modes still answer in one voice. Two things the rendering has to get right: - Only a return carries them. The wipe and the scan happen when the drive comes back, so a check-out row shows blanks rather than implying a fact about an event that had not happened yet. - Unknown is not "no". Both stores keep these nullable deliberately: a null means nobody wrote it down, which is a different claim from a drive that came back unwiped. Rendering an unrecorded return as "No" would put an assertion in an audit record that no one ever made, so unknown stays blank and reads as the gap it is. A test holds that, because "no" is the tempting default. auditflag lives in selfhosted.py rather than routes.py: routes already imports selfhosted, so the helper had to sit on that side or the import would be circular. |
||
|
cproudlock
|
0f56653345 |
Say when a kiosk screen is not set up, and hand out the setup link
Two failures from the same root: an administrator cannot see the kiosk token, and a screen that was provisioned wrongly does not say so. The settings API masks every key whose name contains 'token', so the settings page shows '********'. The real value is visible for one moment - after Generate, before Save - and an administrator who copies the field at any other time provisions a screen with the literal mask. The kiosk then answers 404, exactly as it answers a caller it has never heard of, because the route deliberately does not advertise itself. Nothing anywhere says what went wrong. - Each kiosk gains a permission-gated provisioning-link route that returns the real setup URL, token included, to somebody who already holds settings.edit. The mask stays: a token has no business in an ordinary settings list. The copy step just stops depending on catching the one moment it is on screen. Revealing it writes an audit row, since that is when the value left the database. Both settings pages get a "Copy setup link" button, which falls back to showing the link when the clipboard is refused - it is, over plain http and in locked-down browsers. - The USB kiosk asks the server on load whether this screen may be used, and says plainly when it may not. It rendered a keypad and only failed once somebody had typed their SSO, which reads as broken rather than as not set up. It cannot guess: the enabled flag is public and only says the site has a kiosk, and a caller admitted by network holds no token to look for, so the parts kiosk's localStorage check would warn exactly the callers the allowlist exists to serve. The probe is a badge-less session request - the gate runs before any badge handling, so refused is 404 and admitted is the validation error, with no lookup and nothing recorded. A test pins that ordering, because reordering it would start telling admitted screens they are not set up. Verified in a browser, not only in tests: provisioning with the mask lands on the refused screen, a correct token gets the drive list and survives a plain revisit, and a tokenless caller inside the allowlist gets straight in. |
||
|
cproudlock
|
da91a29155 |
Admit kiosk callers by network position, not only by device token
The device token assumes a handful of fixed screens an administrator provisions once. That is not how these kiosks are used: most people open the page on their own PC, and there is no login, so per-browser provisioning would mean touching every PC on the floor and losing it again on the next reimage or cleared profile. A screen in that state did not look refused either - the USB kiosk rendered a working page whose every call answered 404. Both kiosks now admit a caller on either ground, the same shape the GE-Enforce client endpoints already use: the per-device token, or a source address inside an allowlist. Wall screens keep the token and are not confined to the allowlist; everyone else is admitted by where they are on the network. Still gated on the site's enable flag, so nothing opens by itself, and both allowlists ship empty - token-only until a site enters its ranges. - shopdb.api gains ip_in_cidrs and request_ip_allowlisted (contract 0.35.0). On the surface because both plugins need the identical gate, and a CIDR parser copied into two of them is where a fail-open creeps in later. They fail closed on an empty setting, an unparseable caller address and a malformed entry, and they read remote_addr rather than raw X-Forwarded-For - so the IIS rewrite rule is what makes them meaningful, which the module says at length. - Separate settings per kiosk, usb_kiosk_allowed_cidrs and printedparts_kiosk_allowed_cidrs. One must not grant the other, for the same reason the tokens are separate. - Both settings pages get a picker listing the network plugin's subnets with their VLANs. It stores resolved CIDR strings, not subnet ids: the setting has to work at a site that does not run that plugin and must not break when an inventory row is deleted. Entries matching no subnet are shown rather than hidden, since a stale one left by an inventory edit looks exactly like a hand-typed range. - USBKiosk says it was not admitted instead of pretending to work, which PartsKiosk already did. - The USB settings page keyed text-vs-boolean off the token alone. Adding a second text setting to the boolean path would have saved it as the word 'true' - the bug that replaced the kiosk token and 404'd every provisioned screen. Both load and save now key off a list, with a spec on it. The trade, stated plainly in the admin UI: anyone on an allowed network can check a drive in or out against a badge number they guess, and the record carries that person's name. Badges are attribution, not authorization. Keeping the ranges narrow is what bounds it; login is the alternative and was ruled out because these users do not sign in. |
||
|
cproudlock
|
158ee0507e |
Move the agent and tray to 1.20.1 and 1.7.1
No behaviour change. The 1.20.0 and 1.7.0 binaries were built and shipped to the display scope before two fixes landed - the fast retry while an enrollment is pending, and the fleet alert key rename - so a rebuild carried different bytes under the same version number. Detection is by hash and copes with that, but nobody reading a version number can tell two 1.20.0.0 files apart, and the runbook asks for one build across the share, the agent store and the display profile. The version moves instead. |
||
|
cproudlock
|
1a6ac67cab |
Say what a plugin settings tab can actually do, and catch a declaration that cannot work
Finding F12 of the architectural review, checked against the code and found worse than reported: the dynamic component import cannot be rewritten by the bundler, so the built file ships the unresolved path and a plugin tab answers 404 in every production build, staged or not. It works only under the dev server. Nothing shipping is affected, because the one plugin that declares a tab also registers the same page as an ordinary route, and that is the one sites reach. - The ADR draft now carries a "Constraints as implemented" section: only the model catalog hub registers plugin tabs, registration happens when that hub mounts so a tab is not deep-linkable and does not survive a refresh, and the build finding above with the bundler's own warning. The two ways out are stated as the open decision rather than taken. - The hook's docstring reads as instructions to a plugin author, and its example no longer names a path that does not exist at build time. - A route declared for a hub that cannot render it is dropped with an error naming the plugin, the hub and the hubs that do work, and the same is logged at startup. Enabling the plugin still succeeds: a tab nobody can reach is no reason to take a working backend offline. - Tests hold the two halves together: no bundled plugin may declare a hub the frontend does not implement, and the list in the server must equal the hubs that really register routes, so the two cannot drift apart again. |
||
|
cproudlock
|
066a19052b |
Act on the wj architectural review: races, limits, pool and contract gaps
Findings F2, F4, F5, F6, F7, F8, F9, F10, F11 and F13 from wj-architectural-review.md. F1/F1a (the non-sargable LOWER pattern) and F3 (the warranty sync holding a pooled connection) are being handled separately and are untouched here. - A collector check-in that raced another for the same PC lost everything: the unique index on the application fingerprint raised, nothing caught it, and the whole check-in rolled back, discarding state, events and hardware, answering the agent with form copy meant for a person. The insert now runs in a savepoint, reconciles what the other transaction already wrote, one row per savepoint so a second collision skips a row rather than the batch, and the check-in commits with a warning. This is the shape the delete side of the same function already had. - Enrollment is deliberately unauthenticated, so it now carries a per-address rate limit and a cap on pending requests, and the cap is a setting an admin can see and change. A flood can no longer bury the approval queue or grow the table without bound. - The fleet alert poll no longer formats debug strings on every request. - BusinessUnit is on the contract surface (0.34.0), so the endpoints plugin stops reaching into a core table with raw SQL. - A test walks every asset type descriptor and proves the endpoint it names really refuses a caller without the declared permission. - The plugin settings routes now need a permission; they were telling anyone which modules a site runs and where its components live. - The connection pool is sized explicitly with a short timeout, and the arithmetic against the server's connection ceiling is written next to the worker count instead of being left to defaults nobody chose. - Fleet alerts carry both the correct lowercase keys and the old camelCase pair. Trays older than 1.7.0 match keys case-sensitively and would have gone quiet on every PC not yet upgraded; the old pair goes when they are. - The dev server proxy points at localhost again, with environment variables for the remote-box workflow, and one developer's address is out of the server certificate page. |
||
|
cproudlock
|
2557d1967d |
Make the one-line install survive Defender, and approval leave a PC that works
All three found testing the bootstrap installer on Windows. - The published command used the irm-pipe-iex form, which Microsoft Defender detects as Trojan:Win32/Commando.A!ml and kills before it runs, every time, on a default install. The command now downloads the script to a file, runs it, and removes it in a finally block, which Defender does not object to. The UI also offers the script as a plain download for a tech who would rather save it. - Approving a request only linked the asset, so a PC whose pending reason was "unknown hostname" - which is every PC installed this way - kept getting 403 on every call afterwards, because its computer record had no hostname to match the credential against. Approval now stamps the hostname when the record has none, refuses with a 409 naming both when the record has a different one, and overwrites only when the admin confirms. The Credentials tab offers that confirmation. - A hostname that resolves to no record now says so instead of reading as an impostor, while still refusing the request. - While waiting for approval the agent retried on the 15-minute telemetry cadence, so a PC approved immediately enrolled a quarter of an hour later and the installer always ended on "still pending". It now retries about every minute for the first ten minutes, then every five to thirty minutes, then every fifteen, with jitter. A rate limit or server error still uses the slower backoff. |
||
|
cproudlock
|
ca57b7f728 |
Install an agent on a one-off PC from a single command
An admin generates a short-lived token in Settings > Endpoints > Endpoints and
copies the command it produces. A tech runs that one line in an elevated
PowerShell on a PC with no agent:
powershell -ExecutionPolicy Bypass -c "irm https://<server>/shopdb/api/
endpoints/siteagent/bootstrap.ps1?t=<token> | iex"
The script refuses to run unelevated, refuses a PC that already has the agent
(a managed bay gets it from GE-Enforce), downloads the package the existing
builder produces, checks both binaries against the hashes the server baked in,
installs with NO shared key, and reports what happened. The PC then asks to
enroll and waits for an admin to approve it, which the script watches for
locally for two minutes before exiting.
- New endpointbootstraptokens table and migration
endpoints0024bootstraptokens: hash only, expiry, use count, revoke.
- Admin routes to generate, list and revoke tokens; the secret is shown once.
- Two token-only routes: the script, and the package (which spends a use).
Anything wrong with the token answers 404 rather than confirming it exists,
both are rate limited per address and both are audited with the caller.
- The package builder is now shared rather than duplicated, and a bootstrap
package can never carry a collector key.
|
||
|
cproudlock
|
408cf08628 |
Stop the USB settings page from replacing the kiosk token with "true"
The page saved every field as 'true'/'false', the kiosk device token included, and loaded the token as a boolean. One save of Settings > USB Devices therefore stored the literal word "true" (or "false") as the token, and every kiosk provisioned with the real token got 404 on each request - reported as the kiosk no longer accepting an SSO. Generate + Save never stored the generated value either. The token now loads and saves as text. The server already keeps the stored token when it receives the mask back unchanged. The two switches still save as booleans. The parts kiosk page was not affected. |
||
|
cproudlock
|
3d9fe793b2 |
Keep every server pin in the upgrade script, and stop promising enrollment to an enrolled PC
Found in the Windows VM check. - The agent upgrade script wrapped Get-ShopDbServerPins in @(), which turned several registry pins into one joined string. During a certificate rotation the upgrade download would have refused the rotated server. It also merged appsettings pins with the registry; it now follows the agent's rule: when the ServerPins value exists, even empty, the registry owns the list. The shared pin block in the upgrade script and both reporters gains Test-ShopDbServerPinsValue. - A key-less package installed over an enrolled PC printed both "existing enrolled credential found; kept as is" and "This PC will request enrollment". The enrollment notices now print only when no credential exists. |
||
|
cproudlock
|
8f10c09a76 |
Always pin the server, let the registry own pins, and survive a revoke mid-upgrade
Found in the Windows VM re-test. - A CA-flagged server certificate (common with a plain openssl self-signed cert) produced an installer package with no pin at all, so agent 1.20 refused the server. The package now always carries the SPKI pin of the certificate PCs receive; the CA flag only skips the trusted-root import. When an uploaded CA-flagged certificate differs from the server's own, both pins are included. The build response carries X-ShopDB-Build-Warning and the installer logs what older agents need. - Pins written into appsettings.json by the installer were merged with the registry, so removing a pin from HKLM never took effect. When the ServerPins value exists in the registry it is now authoritative, even when empty; appsettings pins apply only when it is absent. Set-ShopDBServerPins.ps1 -Clear deletes the value; new -None writes an empty value (no pins). - A script or package download sent with a credential revoked moments earlier got 401 CREDENTIAL_INVALID and failed the command. The agent now retries that download once with the shared key and records the fallback in the result. |
||
|
cproudlock
|
1c2ba677c3 |
Let key-less PCs ask to enroll, and let an admin retry a trust-blocked PC
Found in the Windows VM re-test. - A PC installed without the shared key got 401 from POST /api/endpoints/enroll while the shared key was allowed, so no request was recorded and no admin could approve it. Enroll now never refuses: a valid shared key only decides auto-issue on a clean match; no key or an invalid key creates or updates the one pending request for that hostname and serial. An approved request still issues only to its own nonce. - A PC blocked with "server certificate not trusted by this PC" could never be queued again after trust was fixed, because the blocker comes from its last failed upgrade. Naming the PC in a wave now queues it when trust is its only blocker, and the response lists it under "retried"; "next n" still skips it. The Migration table lets such a PC be selected. - counts.pending on GET /enrollment counts pending requests, including ones for unknown hostnames; counts.pendingcomputers keeps the per-computer figure. |
||
|
cproudlock
|
22d06abb70 |
Show per-PC credentials across the admin interface
- Endpoints (shared key) page: the key is presented as what PCs use until they enroll, with enrollment progress, a link to Enrollment, rewritten guidance, and rotate/revoke confirmations that say how many PCs are not enrolled yet. - Settings cards describe the server certificate as what PCs actually receive, on IIS or Docker; Enrollment has its own icon (the icon set lacked the one the Server Certificate card already asked for). - Server Certificate page checks the certificate the site presents at its public address and compares its pin with the certificate file, warning when they differ. The host comes from site settings only; the check is a TLS handshake, never an HTTP request. - Fleet list: Credential column and filter (enrolled, pending, re-enroll blocked, shared key, not reporting), computed in the list query. - PC detail: credential state, prefix and dates for fleet readers, state only for others, with admin Revoke (optionally blocking re-enrollment) and Allow re-enrollment. - Dashboard: pending enrollment requests, and PCs blocked from re-enrolling for more than seven days. - Runbook: where to find each of these in the app. |
||
|
cproudlock
|
70d5531b4d |
Make rollback, first upgrades and pin changes work as tested on Windows
Found in an end-to-end test on a Windows 11 VM against a self-signed server.
- Rollback: a 1.20 agent re-enrolls within a second of an un-enroll, so an
older agent pushed afterwards arrived on an enrolled PC and was refused.
Revoke now takes {"blockreenroll": true, "reason": ...}; a blocked PC is
never issued a credential and keeps working on the shared key until
POST .../assets/<assetid>/allow-reenroll. The Enrollment page offers the
block when un-enrolling and shows blocked PCs with an allow button.
- First upgrade of old agents: agents before 1.20 check the server
certificate when downloading the upgrade, so a store wave failed on PCs that
do not trust a self-signed server while the Migration tab said ready. A
failed download now marks the PC blocked with "server certificate not
trusted by this PC". New Install-ShopDBServerCertificate.ps1 (GE-Enforce,
SYSTEM) adds a non-CA server certificate to the trusted roots after checking
its thumbprint; the server certificate page offers a .cer download and the
SHA-1 thumbprint.
- The generated installer writes ServerPins (with an optional next pin for a
planned rotation), can leave the shared key out so the PC waits for
approval, never touches an existing credential, and skips the trusted-root
import for a CA certificate.
- The agent re-reads server pins about every 60 seconds and recycles pooled
connections within 2 minutes, so a pin change needs no restart.
- An alert can only be acknowledged by a PC it targets.
- A wave no longer counts a PC as failed after a later wave upgraded it.
- The scripts report a pin mismatch only when the pin really is missing, and
otherwise log the real error.
- Runbook: trust step before the first wave, rollback order, pin re-read,
verifying the pin against the live URL; agent contract updated.
|
||
|
cproudlock
|
8eee6f19d8 |
Move the security audit and scan reports out of the product tree
The 2026-09-12 audit reports, the unvalidated findings and the Darkwing scan output described weaknesses in detail on the production branch. They are kept outside the repository with the site's other working records; git history still holds them. |
||
|
cproudlock
|
1164cfb287 |
Add the enrollment page, move scripts to per-PC credentials, and document the rollout
- Settings > Endpoints > Enrollment. Migration tab: readiness counts and progress, each PC's state with the reasons it is blocked, "Upgrade selected" and "Upgrade next 25", and live wave progress. Credentials tab: enrolled PCs with revoke, pending requests with approve and reject, and the shared key and auto-enroll switches. Turning the shared key off asks for confirmation and shows how many PCs are not enrolled. - The server certificate page shows the SPKI pin to put in the GE-Enforce pin entry. - The agent upgrade script and the asset and printer reporters send the PC's credential when it has one (over https, only to the issuing server), fall back to the shared key, check server pins, and log refusals clearly. - New Set-ShopDBServerPins.ps1 (GE-Enforce entry) and Get-ShopDBServerPin.ps1. - AGENT-CONTRACT.md documents per-PC credentials; the rollout runbook covers phases, gates, rollback, pin and wave entries, and every place the shared key lives and how to clear it at cutover. - Removed the 09-14 certificate design notes and dated session reports, and two VM notes that carried a live collector key and advised plain http. - Corrected the GE-Enforce collector note, which claimed a plain registry token is protected by ACL. |
||
|
cproudlock
|
35c17d2ea2 |
Authenticate the agent with its own credential, pin the server, and take secrets out of the tray
Agent 1.20.0, tray 1.7.0. Ship together. Agent: - Every HTTP client, including script and package downloads, is built by one AgentHttpFactory. Server pins (SPKI SHA-256, current and next) come from HKLM ServerPins or appsettings serverpins; with a pin set only a matching key is trusted, otherwise the normal chain check applies. - EnrollmentService enrolls once using the shared key the PC already has, stores the credential DPAPI-protected in a SYSTEM-only directory, and swaps to it without a restart. Pending requests retry every 15 minutes; an old server without the enroll route leaves the agent on the shared key. A revoked credential is deleted and the PC re-enrolls. - A credential is only ever sent to the server that issued it. - Telemetry posts to /api/endpoints/collect when using a credential. - A named pipe lets the interactive user ask the agent to send status, send a report, list alerts and acknowledge one. The report sidecar and screenshot must sit in the caller's own profile and not be a link. - --once and --report run as a user become pipe clients. - appsettings.json is restricted to SYSTEM and Administrators once the pipe is up. - Removed client certificate renewal and expiry monitoring. Tray: - Holds no key and reads no agent configuration. Alerts, acknowledgements, Send Status and Report an issue all go through the agent's pipe. - An acknowledgement is remembered only when the agent confirms it. - When the agent is not running, the menu says so and alert polling retries quietly. |
||
|
cproudlock
|
7480e4fe3a |
Give each endpoint its own credential and fix the review findings on the server
Per-PC credentials (replaces the 09-14 client-certificate attempt): - POST /api/endpoints/enroll issues a credential to a PC that matches exactly one computer by hostname and BIOS serial and has none yet. Anything else (unknown or duplicate hostname, serial mismatch, already enrolled) waits as a request an admin approves or rejects. The agent's persisted nonce binds an approved request to the PC that asked. - One auth helper serves every agent route. With X-Endpoint-Credential the asset comes from the credential and a request naming another PC gets 403 ASSET_MISMATCH. The shared key still works for PCs that have not enrolled, is refused (401 SHARED_KEY_REFUSED) for any PC that has, and can be switched off. A revoked credential gets 401 CREDENTIAL_INVALID so the agent re-enrolls. - The core collector accepts the credential for the computers, printers and endpoints collectors through two new plugin hooks exposed on shopdb.api (contract 0.33.0), and refuses the shared key for enrolled PCs. - Admin API: enrollment list, approve, reject, revoke (un-enroll), settings. Migration API: per-PC readiness with blockers, and upgrade waves that package the agent once and queue the existing agent upgrade per PC. - Migration 0023 creates endpointcredentials, endpointenrollrequests and endpointmigrationwaves, drops the certificate columns from assets where test databases have them, and renames app_fingerprint to appfingerprint. 0021 and 0022 keep their ids but no longer alter the core assets table. - Removed: client_cert.py, certificate renewal, the unauthenticated /test/client-cert route, nginx client certificate forwarding and the agent certificate script. The client-cert header could be forged on IIS, the key was readable by users, and the shared key remained a fallback. - GET /settings/server-cert returns the SPKI SHA-256 pin for GE-Enforce. Review fixes: - report_command_result used g without importing it; every result was a 500. - collect_log refuses setup answer files, appsettings, agent directories and key files at enqueue. - Fleet alerts: alerttimeout and alertfleet agree with the migration, the scope is validated and a malformed scope no longer reaches the whole fleet, and the ack race no longer raises NameError. - A refused break-glass override leaves no hold and no audit row. - Agent upgrade checks locks and permissions before repackaging, and writes the package atomically. - Server certificate upload validates the pair before replacing files; generation no longer builds an openssl config from user input; audit rows are actually written. - Health evaluation runs in a savepoint so a caught error cannot lose the check-in. - Required-app policies are found by appid, survive renames, and deleting an app deactivates policies that still name it. The UI and API now say such a policy only takes effect inside a business policy. - Backfill handles duplicate fingerprints and rolls back a failed batch. - Plugin settings routes skip a plugin whose hook raises. - The setup token is no longer printed to the log. - The naming gate checks top-level files again and handles odd file names. - Removed root scratch scripts, one of which carried a token. - Rewrote the fleet alert scheduling and app delta tests on real fixtures. |
||
|
cproudlock
|
f484be8a33 |
Make fleet alerts readable, deliberate to acknowledge and remembered
Tray 1.6.11. - One alert with a null or malformed field no longer throws away the whole alert list; nullable server fields are tolerated. - Keystrokes can no longer acknowledge an alert the operator has not seen. There is no default button, focus starts on the message, and acknowledging takes a mouse click after the alert has been on screen for 1.5 seconds. - Long messages scroll instead of being cut off, and an ampersand is shown as typed. - A timeout closes the alert without acknowledging it. - Missing type names, titles and colours fall back to safe defaults. - Shutdown waits on cancellation instead of sleeping, and closes an open alert before the client is disposed. - The acknowledged-alerts store moved from ProgramData, which the operator cannot write, to the user's local app data, guarded by a mutex. Acked alerts used to reappear at every logon. |
||
|
cproudlock
|
1cf30843f3 |
Harden the agent's command execution and restore server certificate checks
Agent 1.19.0. - With no pinned thumbprint the collector, command and fleet alert clients accepted any server certificate. They now require a valid chain again. Production uses a self-signed certificate, so this build must not reach bays before server pinning ships with it. - SYSTEM script runs read stdout and stderr while the script runs, cap what is kept, and kill the whole process tree after five minutes. A chatty script used to fill the pipe and hang the command channel until a service restart. - Scripts and update_app packages are staged in a directory only SYSTEM and Administrators can touch, under a random name, with the hash checked in memory and again on disk. A standard user could previously swap the file between the hash check and execution. - The user-context transcript lives in a SYSTEM-created directory, and SYSTEM refuses to follow a link when reading or deleting it. - Strict mode is inserted after any param block, #Requires or using lines, keeps line numbers and writes UTF-8 with a BOM. - A user-context run that reports no exit code now fails. - collect_log refuses setup answer files, appsettings files, the agent's own directories, key files and links. - The startup renewal check is capped at 15 seconds and exits non-zero when it wants a restart, so service recovery restarts the agent. |
||
Proudlock
|
1ed0c4d215 |
HOTFIX: Remove hot-path logging and unnecessary flush from collector
Removed performance bottlenecks in _replace_applications(): - 5 logger.info/warning calls executing on every collector check-in - Explicit db.session.flush() in legacy path forcing immediate SQL - With 800 check-ins/hour, this was 800 log writes + 800 premature flushes Changes: - Removed all hot-path logging (moved critical info to warnings list) - Removed flush() in legacy fallback path (let transaction commit batch SQL) - Kept conditional flush in delta mode (only when changes detected) Impact: Eliminates 800+ disk I/O operations per hour in collector path Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|
Proudlock
|
affe455c55 |
HOTFIX: Remove row-level locks from collector check-in path
Removed .with_for_update() from _replace_applications() which was acquiring row-level locks on every collector check-in (800/hour). These locks were holding during fingerprint comparison, blocking concurrent reads and causing severe performance degradation. The UNIQUE constraint on (assetid, app_fingerprint) already provides sufficient concurrency protection without explicit row locks. Impact: Eliminates 800 lock acquisitions per hour, unblocks reads during check-in processing. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|
Proudlock
|
5cd5332397 |
Add agent fingerprinting, client certificate auth, and security audit documentation
This commit adds: - Database migrations for app_delta fingerprints and client certificates - Scripts for certificate generation and fingerprint backfilling - Comprehensive security audit reports and findings tracking - Implementation documentation for certificate authentication - Test utilities for certificate renewal and fingerprinting - GE-ENFORCE collector fix documentation Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|
Proudlock
|
1a7bb9bd34 |
Fix code block still showing white text in dark mode (!important)
UX Fix: Code block contrast in dark mode Problem: - Code block with JSON example still white-on-white in dark mode - User reported: 'still white text on white bg' - Previous dark mode fix didn't override parent styles Root Cause: - Parent .field-hint and .info-list classes override .code-block - Need !important to force dark mode colors - <pre> tag selector needed more specificity Solution: - Added !important to dark mode color rules - Added explicit selectors for pre.code-block - Added nested selectors: .info-list .code-block, .field-hint .code-block - Ensures dark mode applies regardless of parent context Dark Mode Colors (enforced): - background: #2b2b2b (dark gray) - color: #e0e0e0 (light gray) - border: #444 (medium gray) Testing: - Run: node scripts/stage-frontend.mjs - Check: JSON code block in 'Agent certificate pinning' section - Verify: Dark background, light text, readable in dark mode Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|
Proudlock
|
812d1ee4d1 |
Fix dark mode color contrast on server cert settings page
UX Fix: Dark mode readability Problem: - Warning banner: light yellow background with dark text unreadable in dark mode - Code blocks: light gray background with dark text unreadable in dark mode - User reported: 'warning on darkmode not visible, light yellow background on white text' Solution: - Added @media (prefers-color-scheme: dark) styles - Warning banner: dark background (#664d03), light text (#ffecb5) - Code blocks: dark background (#2b2b2b), light text (#e0e0e0) - Maintains good contrast in both light and dark themes Colors: - Light mode warning: #fff3cd background, #856404 text (WCAG AA compliant) - Dark mode warning: #664d03 background, #ffecb5 text (high contrast) - Light mode code: #f8f9fa background, #212529 text - Dark mode code: #2b2b2b background, #e0e0e0 text Deployment Note: After editing plugins/*/frontend/ files, run: node scripts/stage-frontend.mjs This copies plugin frontends to frontend/src/.plugins-staged/ where Vite reads them (per ADR-013 plugin pruning). Context: - Missed running staging script initially (my error) - Manual copy to .plugins-staged worked but wrong workflow - .plugins-staged/ is .gitignored (generated, not committed) - Proper workflow: edit source -> stage -> Vite hot-reloads Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|
Proudlock
|
b7e454b5ef |
Fix blank screen on server cert settings page (missing authentication)
Critical Bug Fix: Server cert settings page blank screen
Problem:
- Navigating to /settings/endpoints/server-cert showed blank page
- Both localhost:5173 and localhost:5001 affected
- Browser console showed authentication errors
Root Cause:
- ServerCertSettings.vue used raw fetch() without authentication
- API endpoint requires admin role (@require_role('admin'))
- fetch() didn't include JWT bearer token from cookies
- API returned 401 Unauthorized, component failed silently
Solution:
- Converted from Options API to Composition API (Vue 3 standard)
- Changed from raw fetch() to authenticated api client
- Used 'import api from @/api' (same as AgentKeySettings.vue)
Changes:
Before (broken):
export default {
methods: {
async loadCertificate() {
const response = await fetch('/api/endpoints/settings/server-cert')
// No auth headers = 401 error
}
}
}
After (working):
import { ref, onMounted } from 'vue'
import api from '@/api'
async function loadCertificate() {
const response = await api.get('/endpoints/settings/server-cert')
// api.get() includes JWT from cookie automatically
}
API Client Features:
- Automatically includes Authorization header with JWT
- Handles response parsing
- Consistent error handling
- Used by all other admin settings pages
Result:
- Page loads successfully
- Certificate details display
- Generate/upload forms work
- All admin-only endpoints authenticated
Tested:
- Hard refresh clears browser cache
- Page renders certificate details
- Generate/upload buttons functional
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
||
|
Proudlock
|
87582d0851 |
Fix endpoints plugin load failure (audit_log decorator incorrect usage)
Critical Bug Fix: Plugin load failure
Problem:
- Endpoints plugin failed to load after adding server cert management
- Error: audit_log() missing 1 required positional argument: 'entitytype'
- Plugin completely broken - no endpoints routes available
Root Cause:
- Used @audit_log('action') as decorator on routes.py:4394, 4521
- audit_log() is NOT a decorator - it's a function requiring (action, entitytype)
- Decorator pattern doesn't exist in ShopDB API
Solution:
- Removed @audit_log decorators from both endpoints:
- /settings/server-cert/generate
- /settings/server-cert/upload
- Added manual AuditLog.log() calls inside functions after success
- Wrapped in try/except to prevent audit logging from breaking endpoint
Implementation:
try:
AuditLog.log(action='endpoints.server_cert.generated',
userid=current_user.userid if current_user else None,
details='Generated self-signed certificate: CN=...')
except Exception:
pass
Result:
- Endpoints plugin loads successfully
- Blueprint registered: /api/endpoints
- All endpoints routes working (tested: /api/endpoints/alerts)
- Settings menu now visible in sidebar
- Audit logging still captures cert operations
Verification:
- docker-compose logs: 'Endpoints plugin initialized (v0.1.0)'
- GET /api/endpoints/alerts returns 200
- GET /api/dashboard/health returns healthy
Impact:
- Entire endpoints plugin was broken since commit
|
||
|
Proudlock
|
967589e585 |
Fix agent build errors for .NET 8 (File.GetAccessControl moved to FileInfo)
Build Fix: .NET 8 API changes
Problem:
- Agent failed to compile with .NET 8.0.424
- Error: 'File' does not contain a definition for 'GetAccessControl'
- Error: 'File' does not contain a definition for 'SetAccessControl'
Root Cause:
- File.GetAccessControl() and File.SetAccessControl() removed in .NET 5+
- Moved to System.IO.FileSystem.AccessControl package as extension methods
- Methods now on FileInfo/DirectoryInfo classes, not File static class
Solution:
1. Added package reference:
- System.IO.FileSystem.AccessControl v5.0.0 (ShopDbEndpointAgent.Core.csproj)
2. Added using directive:
- using System.Security.AccessControl; (CertificateRenewal.cs)
3. Changed API usage (CertificateRenewal.cs:290-298):
Before:
var acl = File.GetAccessControl(keyFile);
File.SetAccessControl(keyFile, acl);
After:
var fileInfo = new FileInfo(keyFile);
var acl = fileInfo.GetAccessControl();
fileInfo.SetAccessControl(acl);
Build Results:
- ShopDbEndpointAgent.Core: SUCCESS (1 warning, 0 errors)
- ShopDbEndpointAgent: SUCCESS (0 warnings, 0 errors)
- ShopDbEndpointAgent.Tray: SUCCESS (0 warnings, 0 errors)
- Published to agent/publish/agent/ and agent/publish/tray/
Context:
- Private key ACL grants required for tray app access (CertificateRenewal.cs:290)
- BUILTIN\Users group needs Read+ReadAndExecute on private key file
- Enables certificate renewal from both service and tray contexts
Related Changes:
- CertificateExpiryMonitor.cs (periodic monitoring, commit
|
||
|
Proudlock
|
ca90969eb4 |
Fix fleet alerts sent immediately instead of respecting scheduled starttime
Bug Fix: Fleet alert scheduling regression
Problem:
- User sets notification to run at 3:30 PM (future time)
- Fleet alert sent to agents immediately at 3:10 PM instead of waiting
- Agents see notification 20 minutes early
Root Cause:
- GET /api/endpoints/alerts query filtered for:
- alertfleet = True
- isactive = True
- endtime is NULL OR endtime > now (not expired)
- Not acknowledged
- MISSING CHECK: starttime is NULL OR starttime <= now (has started)
- Result: Any notification with future starttime appeared immediately
Solution:
- Added starttime filter to fleet alert query (routes.py:4151-4153)
- Query now checks:
db.or_(
Notification.starttime.is_(None),
Notification.starttime <= now
)
- Notifications with future starttime excluded until that time
Behavior After Fix:
- Notification with no starttime: appears immediately (existing behavior)
- Notification with past starttime: appears immediately (existing behavior)
- Notification with future starttime: waits until that time (FIXED)
- Notification with past endtime: never appears (existing behavior)
Testing:
- tests/test_plugins/test_endpoints_fleet_alert_scheduling.py
- test_fleet_alert_respects_future_starttime (regression test)
- test_fleet_alert_with_no_starttime_appears_immediately
- test_fleet_alert_past_starttime_appears
- test_fleet_alert_expired_does_not_appear
Timeline Example:
- 3:10 PM: User creates notification with starttime=3:30 PM, alertfleet=True
- 3:15 PM: Agent polls /api/endpoints/alerts -> empty (starttime not reached)
- 3:35 PM: Agent polls again -> alert appears (starttime has passed)
Impact:
- Agents poll every 60 seconds, so scheduled alerts appear within 1 minute of starttime
- No database migration needed (starttime column already exists)
- Backwards compatible (NULL starttime still means immediate)
Related:
- Notification.is_current property already checks starttime (models/notification.py:162-169)
- Shopfloor board respects starttime correctly
- Only fleet alert endpoint had this regression
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
||
|
Proudlock
|
d11ba5c5d6 |
Add server certificate management UI (Settings > Endpoints > Server Certificate)
Operational Feature: Web UI for TLS certificate management
Problem:
- Server certificate (nginx HTTPS) managed manually via certs/ directory
- No visibility into cert expiry from UI
- Regeneration required command-line OpenSSL knowledge
- No warning when cert approaching expiration
- Agents using thumbprint pinning need new thumbprint after renewal
Solution:
- New admin-only settings page: Settings > Endpoints > Server Certificate
- View current certificate details (CN, SANs, expiry, thumbprint)
- Warning banner when cert expires within 30 days
- Generate self-signed certificate (configure CN, SANs, validity)
- Upload CA-signed certificate (PEM format)
- Automatic nginx restart after certificate changes
- Copy-to-clipboard for agent thumbprint pinning
Frontend:
- plugins/endpoints/frontend/views/ServerCertSettings.vue
- Certificate status display with expiry countdown
- Self-signed generation form (CN, SANs, days)
- CA certificate upload (cert + key files)
- Documentation collapsibles (requirements, pinning, air-gapped)
- plugins/endpoints/frontend/routes.js
- Route: /settings/endpoints/server-cert (admin-only)
Backend API:
- plugins/endpoints/api/routes.py
- GET /api/endpoints/settings/server-cert (view cert details)
- Uses openssl to extract CN, issuer, dates, thumbprint, SANs
- Calculates days until expiry (warns if < 30)
- Returns self-signed status
- POST /api/endpoints/settings/server-cert/generate
- Generates self-signed cert with custom CN/SANs/validity
- Backs up old certificate with timestamp
- Writes new cert/key to certs/ directory
- Restarts nginx (docker-compose restart nginx)
- Audit logged: endpoints.server_cert.generated
- POST /api/endpoints/settings/server-cert/upload
- Validates PEM format (cert + unencrypted key)
- Backs up old certificate
- Writes uploaded files to certs/
- Restarts nginx automatically
- Audit logged: endpoints.server_cert.uploaded
- All endpoints admin-only (@require_role('admin'))
Navigation:
- plugins/endpoints/plugin.py
- Added menu item: Settings > Endpoints > Server Certificate
- Icon: shield-check, Position: 29
Security:
- Admin-only access (certificate management is privileged operation)
- Private key validation (rejects encrypted keys)
- Certificate format validation (PEM-encoded)
- Audit logging for all cert operations
- Automatic backup before replacement (timestamped .backup files)
Operational Notes:
- Certificate changes restart nginx (brief HTTPS interruption)
- Agents using ServerCertThumbprint pinning must be updated after renewal
- Self-signed certs valid 1 year by default (configurable)
- Air-gapped sites: cert files in certs/ directory bundled automatically
Related:
- Complements certs/README.md and regeneration scripts (commit
|
||
|
Proudlock
|
fd6240772e |
Add server certificate management documentation and regeneration scripts
Documentation: - certs/README.md: Complete guide for managing nginx TLS certificates - Server cert generation (self-signed, CA-signed, Let's Encrypt) - Certificate expiry monitoring and renewal procedures - Agent certificate pinning configuration - Air-gapped deployment notes Tooling: - certs/regenerate-server-cert.sh: Bash script for cert regeneration - certs/regenerate-server-cert.ps1: PowerShell script for Windows - Both scripts backup old certs and display new thumbprint Config: - Updated .gitignore to allow cert management files (docs/scripts/config) - Private keys and actual certificates still excluded (security) Context: - User asked where to manage server-side certificate - Current cert: self-signed, CN=localhost, valid 1 year - Agents validate server cert via pinning or Windows trusted roots - Critical for mTLS client cert authentication (ADR-006) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> |