Files
shopdb-flask/docs/adr/ADR-019-per-pc-endpoint-credentials.md
cproudlock f9d6b57f0f Move the documentation to the line that ships, and correct it against the code
The docs lived on one lineage and the fleet code on another. That is why six
pages described a shared fleet key as how an agent authenticates, why
COLLECTOR-AUTH had to warn that the per-PC credential code was not in the
repository it sat in, and why ADR-019 recorded a decision whose implementation
was somewhere else. Documentation that cannot see the code it documents goes
stale by construction, and no amount of editing fixes that.

So the corpus moves here: 84 files, 56 pages, 20 ADRs, the proposals, INDEX.md,
llms.txt and both gates. Filenames are unchanged, because a wiki page name is
derived from the filename and a rename breaks every link anybody saved.

THE POINT OF THE MOVE IS THE CORRECTION IT ENABLES. Every caveat that said "not
in this tree" was false the moment the files landed beside the code, so each was
rewritten from what is actually here:

- COLLECTOR-AUTH gained the real enroll request body, the two bounds on the
  unauthenticated route (per-IP rate limit, pending cap with stale rows dropped
  before a real PC is refused), and the rejected pending reason that binds to
  the endpoint identity rather than the nonce.
- ADR-019 now describes an implementation that is present, including its tests,
  and says plainly that what remains is the FLEET migration, not the code.
- proposals/device-identity claimed phase 1 was unbuilt. plugins/computers
  declares deviceid on the collector schema and prefers it over the older
  fallbacks. Corrected in the page, its summary and the index.

WINDOWS/IIS IS NOW DOCUMENTED AS THE LEGACY PATH, per the product decision, and
carefully: it is what production runs today. Every Windows page opens by saying
it is supported, not for a new site, and where a new site should go instead. No
procedure was cut - an operator on an IIS box at 2am needs these pages to work
exactly as well tomorrow. MIGRATE-TOPOLOGY stops being a footnote and becomes
the off-ramp, which is why it nearly doubled: what was there did not match how
the Docker stack really works here.

The Docker material that already lived on this line was kept and given
ownership rather than buried under the ported Windows-centric prose: DEPLOY now
describes the four services, the migrate service that runs the upgrade chain
before api starts, TLS terminated by nginx, and the plugin count corrected from
thirteen to seventeen - a number both lineages had wrong.

ROADMAP is rewritten from evidence rather than memory: what has shipped, read
out of the changelog, the ADRs and the manifests, and ahead of it exactly two
items - finalize a distributed release and polish the setup wizard.

PROJECT-MAP is generated and was stale; the GENERATOR was what was wrong.
--check passes now.

API-REFERENCE is retired to a pointer. /api/docs serves 573 operations with
derived auth, query parameters and harvested examples, and a hand-written page
competing with that is a liability. The file stays so its wiki page and its
inbound links survive.

AND THE MOVE TURNED ON A SCRUB THAT HAD NEVER RUN HERE. test_docs_publishable
keys on docs/*.md, which this line did not have, so it skipped - its own comment
predicted what would surface. Porting the corpus activated it and it found 48
lines that must not reach a published wiki: branch names abbreviating the site,
assistant-tooling paths, the site name in two deployment pages, and an employee
ID in a Windows path that the scrub does not even look for.
2026-09-19 09:10:15 -04:00

11 KiB

ADR-019: Per-PC endpoint credentials

  • Status: ACCEPTED
  • Date: 2026-09-18
  • Deciders: ShopDB maintainers
  • Relates to: ADR-006 (collector contract), ADR-012 (GE-Enforce manifest ownership), ADR-015 (site-specific configuration)
  • Supersedes: ADR-016 (credential delivery to the fleet)

Context

The endpoints plugin authenticates every agent-facing call with ONE key. The same X-API-Key value is on every PC in the fleet, and the server cannot tell one caller from another: a request names its own hostname and is believed.

That key is not only an inventory-ingest key. It is also the key on the command channel (AGENT-CONTRACT.md section 8), where the server hands an agent running as LocalSystem an allowlisted action to execute - including run_script, which fetches a reviewed PowerShell body and runs it. Anyone holding the key can claim commands as any PC and post results as any PC.

And the key is readable. The agent installer writes it to HKLM\SOFTWARE\GE\ShopDB CollectorKey, and the DACL that subkey inherits by default grants BUILTIN\Users read, so any logged-on user on any bay could read the fleet key (the reason is recorded in plugins/endpoints/api/siteagent.py beside the code that now locks the key down). The same value also lived in appsettings.json in the agent directory, in a text file on the GE-Enforce share, and in the Args of the manifest entry that wrote it.

One fleet-wide secret, readable by a standard user on any bay, authenticating a SYSTEM command channel. Rotating it meant touching every one of those places at once, and nothing could say which PCs had picked the new value up.

ADR-016 answered a different question about credentials - which controller password a given bay should receive, decided by server-side targeting rules - and was never built. The question underneath it is "which PC is this", and it needs no targeting at all.

Decision

Each endpoint holds its OWN secret, obtained by enrolling, and the server derives the calling asset from the secret.

The credential

  • X-Endpoint-Credential: sdbe_ + 43 characters of base64url (32 random bytes).
  • The server stores only the SHA-256 of the secret plus its first 12 characters as a display prefix (EndpointCredential in plugins/endpoints/models/credential.py). The full value exists once, in the enroll response.
  • One row per asset, so an asset never carries two credentials. Re-issuing after a revoke updates the same row.
  • A request whose credential resolves to one asset while naming another is refused 403 ASSET_MISMATCH. A PC can no longer report as another PC.

Enrollment

POST /api/endpoints/enroll never refuses a call for its authentication. The shared key, if present and still allowed, only decides whether the credential is issued automatically. Auto-issue requires all of: the shared key authenticated the call, plugin.endpoints.autoenroll is on, the hostname matches exactly one computer record, the serial on record matches, and the asset has no active credential. Anything else records a PENDING request an admin approves in the UI.

The agent generates an enrollment nonce once and persists it. The nonce is recorded on first sight and an approval is bound to it, so a second machine claiming the same hostname and serial cannot collect the approved credential. A rejected request binds to the identity instead, so a reinstall cannot argue its way back into the queue.

Storage on the endpoint

%ProgramData%\ShopDB\EndpointAgent\credential\credential.bin, DPAPI LocalMachine, in a directory ACLed to SYSTEM and Administrators only. The tray holds no secret at all: it runs as the interactive operator and asks the SYSTEM service over a local named pipe. HKLM\SOFTWARE\GE\ShopDB is locked to SYSTEM and Administrators in both registry views.

Server pinning

The agent pins the server's public key: the lowercase hex SHA-256 of the leaf certificate's SubjectPublicKeyInfo, in ServerPins (REG_MULTI_SZ, both views), authoritative whenever the value exists even when empty. Locking the registry key is what makes the registry safe to treat as authoritative, since a standard user can no longer write a pin.

The shared key stays, deliberately

plugin.endpoints.sharedkeyallowed keeps the shared key working for PCs that have NOT enrolled. It is always refused for a PC that holds an active credential (401 SHARED_KEY_REFUSED). A PC whose credential was REVOKED falls back to the shared key, which is what makes a rollback to an older agent possible; plugin.endpoints.revokeddeniessharedkey turns that off and is off by default, because it is only safe once the fleet is off the shared key.

Alternatives rejected

Client certificates, 2026-09-14. This was not a thought experiment: it was built and then removed three days later. client_cert.py, certificate renewal, expiry monitoring, an unauthenticated /test/client-cert route, nginx client-certificate forwarding and an agent certificate script all shipped on a branch and were all deleted.

Three reasons, and the first is fatal on its own:

  1. The header was forgeable. Behind IIS the client certificate reaches the app as a request header, and nothing stopped a caller from simply sending that header itself.
  2. The private key was readable by users on the endpoint, so the certificate was no better protected than the key it replaced.
  3. The shared key remained underneath it as a fallback, so the weakest path was still open and the certificate bought nothing.

It also cost more than it looked. The attempt put five underscore-spelled certificate columns on the CORE Asset model for a plugin's feature and retracted them again, with no contract bump in either direction - an ADR-001 and ADR-002 breach inside an unreleased branch. Its migrations (endpoints0021clientcerts, 0022) survive as retired no-ops so a database that ran them still upgrades; 0023 drops the columns where a database has them.

ShopDB-resolved credential targeting (ADR-016). Decided 2026-08-11 and never implemented. It gave a PC a credential chosen by rules over eight targeting axes with an explicit priority order and an explain endpoint to debug the result. The targeting was the whole design, and the exposure that needed closing was not a targeting problem: the fleet needed identity, not a rule engine. ADR-016 is superseded rather than deleted because the problem it opens with - a cleartext controller password on a share, with no rotation story - is still unsolved. This ADR does not solve it.

Keep one key and just lock down where it lives. The registry ACL is part of this decision anyway, and it is worth having on its own. It does not help: one secret shared by every PC still means the server cannot identify a caller, a revoked bay cannot be evicted without rotating the whole fleet, and any administrator or any process running as SYSTEM on any bay still reads the key that drives every other bay's command channel.

Consequences

The shared key remains until every consumer is migrated. That is the main cost of this decision and it is not short. The key lives in a GE-Enforce manifest entry's Args, in CollectorKey in both registry views, in a text file in the share's common/configs, in appsettings.json in the agent directory, in every generated install.ps1 that was kept, and on display kiosks. Each one has to be cleared in a specific order - retire the entry that rewrites the value before deleting the value - and the agent never deletes the shared key itself, because other SYSTEM scripts may still read it. The runbook (plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md) carries the table.

Turning the shared key off is wider than the agent. It is also refused on /api/collector/computers and /api/collector/printers, so a PC with reporter scripts but no enrolled agent goes silent at cutover.

Enrollment is a new admin queue. Anything that cannot be auto-issued waits for a person: unknown or duplicate hostname, missing or mismatched serial, a one-off PC nobody wrote a manifest entry for. A dashboard row counts PCs waiting and PCs blocked from re-enrolling.

Pins have to arrive before the agent does. An agent that pins with no pin set falls back to plain Windows chain validation, which a self-signed or internal-CA server may fail, and a pin set after the fact is a per-bay debugging job. Rotating the server key becomes a two-phase fleet operation: publish old and new pins everywhere, switch the certificate, then retire the old pin. Switching first cuts off every pinned PC, including its route to the corrected pin list.

GE-Enforce can undo an upgrade. Agents installed from the share are held to a file hash by a manifest entry. A store upgrade that lands a newer agent while the entry still carries the old hash is reverted on the next cycle, so the entries have to be re-stamped or removed before the waves reach those PCs.

Revoke means two different things depending on one setting. With revokeddeniessharedkey off, revoking drops a PC back to the shared key, which is how rollback and credential rotation work. With it on, revoking is a real eviction and the PC reports nothing until it enrolls again.

A PC that cannot reach ShopDB keeps its credential. Unlike ADR-016's fetch model, there is nothing to re-fetch per cycle: the secret is local and long-lived until an admin revokes it.

Implementation status

Built, and in this tree. plugins/endpoints/api/agentauth.py, api/enrollment.py and models/credential.py are all present, shopdb/core/api/collector.py reads X-Endpoint-Credential on the generic collector route, AGENT-CONTRACT.md carries section 9, and tests/test_plugins/test_endpoints_credentials.py covers the auth decisions. The C# agent enrolls and sends the header, and both shipped PowerShell reporters send it when a credential is present and fall back to the shared key when it is not.

Proven on a Windows test PC against a test server: self-enrollment, the shared key refused for an enrolled PC, a request naming another asset refused, pins enforced, the tray pipe working for a non-admin user, both SYSTEM reporters sending with the per-PC credential, revoke followed by re-enrollment, rollback to the previous agent, and enrolled PCs still working with the shared key switched off.

What is NOT finished is the fleet migration, which is the consequence this ADR opens with: every PC that has not enrolled is still on the shared key by design, and the key still lives in the places listed above. The runbook is plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md.

References

  • plugins/endpoints/api/agentauth.py - credential and shared-key authentication
  • plugins/endpoints/api/enrollment.py - enroll, approve, revoke, migration waves
  • plugins/endpoints/models/credential.py - credential, enroll request, wave
  • plugins/endpoints/AGENT-CONTRACT.md section 9 - the wire contract
  • plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md - the rollout runbook
  • ADR-016 - the superseded targeting model