Add the enrollment page, move scripts to per-PC credentials, and document the rollout
- Settings > Endpoints > Enrollment. Migration tab: readiness counts and progress, each PC's state with the reasons it is blocked, "Upgrade selected" and "Upgrade next 25", and live wave progress. Credentials tab: enrolled PCs with revoke, pending requests with approve and reject, and the shared key and auto-enroll switches. Turning the shared key off asks for confirmation and shows how many PCs are not enrolled. - The server certificate page shows the SPKI pin to put in the GE-Enforce pin entry. - The agent upgrade script and the asset and printer reporters send the PC's credential when it has one (over https, only to the issuing server), fall back to the shared key, check server pins, and log refusals clearly. - New Set-ShopDBServerPins.ps1 (GE-Enforce entry) and Get-ShopDBServerPin.ps1. - AGENT-CONTRACT.md documents per-PC credentials; the rollout runbook covers phases, gates, rollback, pin and wave entries, and every place the shared key lives and how to clear it at cutover. - Removed the 09-14 certificate design notes and dated session reports, and two VM notes that carried a live collector key and advised plain http. - Corrected the GE-Enforce collector note, which claimed a plain registry token is protected by ACL.
This commit is contained in:
cproudlock
238
plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md
Normal file
238
plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md
Normal file
@@ -0,0 +1,238 @@
|
||||
# Per-PC credentials rollout
|
||||
|
||||
Runbook for moving a site's endpoint fleet from the shared collector key to
|
||||
per-PC credentials. The wire contract is `AGENT-CONTRACT.md` section 9.
|
||||
|
||||
Placeholders: `<server>` is the ShopDB base URL, `<share>` the GE-Enforce share
|
||||
root, `<pin>` a server key pin, `<pilotbay>` / `<pilothost>` the pilot PC.
|
||||
|
||||
## What changes
|
||||
|
||||
- Every endpoint gets its own secret (`sdbe_...`), stored in
|
||||
`%ProgramData%\ShopDB\EndpointAgent\credential\credential.bin`, DPAPI
|
||||
LocalMachine, readable by SYSTEM and Administrators only.
|
||||
- The server knows which asset is calling from the secret. A PC can no longer
|
||||
report as another PC.
|
||||
- The tray holds no secret. It talks to the SYSTEM agent over a local pipe.
|
||||
- The agent, the upgrade script and the SYSTEM reporters
|
||||
(`Report-AssetToShopDB.ps1`, `Report-PrintersToShopDB.ps1`) pin the server's
|
||||
public key when `ServerPins` is set.
|
||||
- The shared key keeps working for PCs that have NOT enrolled, until an admin
|
||||
turns it off. An enrolled PC is always refused the shared key
|
||||
(`401 SHARED_KEY_REFUSED`).
|
||||
|
||||
## Before you start
|
||||
|
||||
1. Server deployed with the endpoints migration `endpoints0023credentials`
|
||||
applied (`flask plugin upgrade-all`).
|
||||
2. Settings -> Endpoints -> Server Certificate shows a **Public key pin (SPKI
|
||||
SHA-256)**. Cross-check it from a PC:
|
||||
`Get-ShopDBServerPin.ps1 -Url https://<server>`. The two must match.
|
||||
3. Agent 1.20 (and tray 1.7) uploaded as the current release under Settings ->
|
||||
Endpoints -> Upload New/Legacy Agents. Keep the previous release staged as a
|
||||
legacy build: it is the rollback.
|
||||
4. Settings -> Endpoints -> Enrollment -> Credentials: **Shared key allowed**
|
||||
on, **Auto-enroll** on (both are the defaults).
|
||||
5. The GE-Enforce manifest scope that will carry the entries below is at
|
||||
`Version` 2.2 or later, and the fleet's GE-Enforce client library
|
||||
understands `TargetHostnames` / `TargetMachineNumbers`. An older manifest or
|
||||
library ignores those lists and applies the entry to EVERY PC in scope. The
|
||||
same happens with an empty list: never publish a targeted entry with a blank
|
||||
target.
|
||||
|
||||
## Phase 1: pins on the PCs
|
||||
|
||||
Pins must be on a PC BEFORE agent 1.20 arrives there. A 1.20 agent with no pin
|
||||
falls back to plain Windows validation, which a self-signed or internal-CA
|
||||
server may fail; a pin set after the fact is not a rollout order you want to
|
||||
debug per bay.
|
||||
|
||||
Stage `plugins/endpoints/client/Set-ShopDBServerPins.ps1` on the share and add
|
||||
a PS1 entry. Pilot first:
|
||||
|
||||
```json
|
||||
{
|
||||
"_comment": "Server key pin for per-PC credentials. Pilot only.",
|
||||
"Name": "ShopDB server pins",
|
||||
"Type": "PS1",
|
||||
"Script": "scripts/Set-ShopDBServerPins.ps1",
|
||||
"Args": "-Pins <pin>",
|
||||
"DetectionMethod": "Always",
|
||||
"TargetMachineNumbers": ["<pilotbay>"]
|
||||
}
|
||||
```
|
||||
|
||||
Then widen by hostname list (or machine number list) in batches:
|
||||
|
||||
```json
|
||||
{
|
||||
"Name": "ShopDB server pins",
|
||||
"Type": "PS1",
|
||||
"Script": "scripts/Set-ShopDBServerPins.ps1",
|
||||
"Args": "-Pins <pin>",
|
||||
"DetectionMethod": "Always",
|
||||
"TargetHostnames": ["<host1>", "<host2>", "<host3>"]
|
||||
}
|
||||
```
|
||||
|
||||
The script compares before it writes, so `Always` costs nothing per cycle.
|
||||
|
||||
**Gate:** on the pilot, `HKLM\SOFTWARE\GE\ShopDB` and
|
||||
`HKLM\SOFTWARE\WOW6432Node\GE\ShopDB` both hold `ServerPins` (REG_MULTI_SZ)
|
||||
equal to the pin on the settings page, and the next `report-asset-*.log` under
|
||||
`C:\Logs\Shopfloor` says `server pin check on (1 pin(s))` followed by a
|
||||
RESPONSE line.
|
||||
|
||||
**Rotating the server key later:** put the old AND the new pin in `-Pins`
|
||||
(`"<oldpin>,<newpin>"`), wait until every PC has both, switch the certificate,
|
||||
then drop the old pin.
|
||||
|
||||
## Phase 2: re-stamp the share's agent entries
|
||||
|
||||
PCs that got the agent from the share (collections manifest entries with a
|
||||
`Hash` detection on `shopdbagent.exe`) are held to that hash by GE-Enforce. If
|
||||
a store upgrade wave puts 1.20 on such a PC while the entry still carries the
|
||||
1.19 hash, GE-Enforce reinstalls 1.19 on its next cycle and the PC flips back.
|
||||
|
||||
Before any wave reaches a share-installed PC, either:
|
||||
|
||||
- re-stamp those entries to the 1.20 payload and its `DetectionValue` SHA-256
|
||||
(and keep the `ProductCode`/version guards the entries already have), or
|
||||
- remove the entries for the PCs the wave will cover.
|
||||
|
||||
Agent entries stay targeted. Illustrative shape for the pilot (keep your
|
||||
existing entry's type, installer and guards; what matters is the hash and a
|
||||
non-empty target):
|
||||
|
||||
```json
|
||||
{
|
||||
"Name": "ShopDB endpoint agent 1.20",
|
||||
"Type": "PS1",
|
||||
"Script": "apps/shopdbagent/1.20.0/install-shopdbagent.ps1",
|
||||
"DetectionMethod": "Hash",
|
||||
"DetectionPath": "C:\\Program Files\\ShopDB\\EndpointAgent\\shopdbagent.exe",
|
||||
"DetectionValue": "<sha256 of the 1.20 shopdbagent.exe>",
|
||||
"TargetHostnames": ["<pilothost>"]
|
||||
}
|
||||
```
|
||||
|
||||
**Gate:** the manifest preview for the pilot shows the 1.20 hash, and no entry
|
||||
for the same file with a blank target list exists in any scope that reaches it.
|
||||
|
||||
## Phase 3: pilot one PC
|
||||
|
||||
1. Enrollment -> Migration: find `<pilothost>`. It should be **Ready**. If it is
|
||||
**Blocked**, the Blockers column lists every reason; fix those first.
|
||||
2. Tick it, **Upgrade selected**.
|
||||
3. Watch the row: **Upgrading**, then **Enrolled** within a few minutes of the
|
||||
new agent starting. The agent enrolls on its own with the shared key it
|
||||
already holds.
|
||||
|
||||
**Gate before any wave:**
|
||||
|
||||
- Credentials tab shows the PC enrolled, `enrolledvia` = `auto`, and a
|
||||
`lastusedat` that keeps moving.
|
||||
- The PC's Fleet row keeps reporting; commands (Collect now) still work.
|
||||
- The tray shows alerts and Send Status works (it now goes through the pipe).
|
||||
- `report-asset-*.log` and `report-printers-*.log` show `RESPONSE (credential)`.
|
||||
- Nothing reinstalled the old agent after two GE-Enforce cycles.
|
||||
|
||||
## Phase 4: waves
|
||||
|
||||
Migration tab -> **Upgrade next 25**. The confirm dialog states how many PCs get
|
||||
agent 1.20. The server re-checks readiness per PC and packages the agent once
|
||||
per wave; PCs that stopped being ready are returned as skipped.
|
||||
|
||||
Between waves, in the Waves list:
|
||||
|
||||
- wait until the wave shows **Done** (every PC enrolled or failed),
|
||||
- investigate every **failed** PC before the next wave,
|
||||
- check nothing moved from Enrolled back to Upgrading or Ready (a share entry
|
||||
reinstalling the old agent, see phase 2).
|
||||
|
||||
Repeat until **Ready** is 0.
|
||||
|
||||
## Phase 5: stragglers and new PCs
|
||||
|
||||
- **Blocked:** each blocker names its fix. `offline`: bring the PC back or wait.
|
||||
`no serial on record` / `duplicate hostname`: fix the computer record.
|
||||
`commands locked`: release the endpoint lock or policy hold. `no agent package
|
||||
in store`: upload 1.20. `agent too old for remote upgrade`: move it via the
|
||||
share (targeted agent entry as in phase 2).
|
||||
- **Not reporting:** the PC has no working agent. It can only move via the
|
||||
share channel: add its hostname to a targeted 1.20 agent entry. It shows as
|
||||
Pending or Enrolled once the agent runs.
|
||||
- **Pending:** the agent asked but auto-enroll could not decide (serial
|
||||
mismatch, duplicate or unknown hostname, already enrolled, or auto-enroll
|
||||
off). Credentials tab -> Requests -> **Approve**; the picker defaults to the
|
||||
matched computer, search to choose another. The PC collects its credential on
|
||||
its next retry (within 15 minutes). **Reject** anything you do not recognise.
|
||||
- **Newly imaged PCs:** imaging installs 1.20 with the shared key, and the agent
|
||||
auto-enrolls on first start. After the shared key is turned off (phase 6) a
|
||||
new PC cannot auto-enroll: its request lands in Pending and an admin approves
|
||||
it in the UI. That is the steady-state path for new PCs.
|
||||
|
||||
## Phase 6: cutover
|
||||
|
||||
**Gate:** Credentials tab **Not enrolled** is 0, or every remaining PC is one
|
||||
you have decided may stop reporting. Remember that turning the shared key off
|
||||
also refuses it on `/api/collector/computers` and `/api/collector/printers`, so
|
||||
a PC with reporter scripts but no enrolled agent goes silent.
|
||||
|
||||
1. Turn **Shared key allowed** off. The confirm shows how many PCs are not
|
||||
enrolled; type `OFF`.
|
||||
2. Watch Fleet for an hour. A PC that went quiet is either unenrolled (turn the
|
||||
shared key back on, fix it, try again) or failed pinning.
|
||||
3. Remove the shared key from every place it lives (below).
|
||||
4. Rotate the old key so any copy you missed is dead: Settings -> Endpoints ->
|
||||
**Regenerate key** (and rotate the collector token the reporters used, if it
|
||||
was a separate token). With the shared key off nothing on the floor needs
|
||||
the new key: approved enroll requests are collected without it. Store it
|
||||
the way the key page describes and keep it out of every file below. A
|
||||
fleet rollback after this point (see Rollback) needs the new key pushed to
|
||||
the PCs again.
|
||||
|
||||
### Every place the shared key lives
|
||||
|
||||
| Where | Put there by | How to clear at cutover |
|
||||
|---|---|---|
|
||||
| GE-Enforce manifest entry `Args` (`-ApiKey ...`) | whoever authored the entry | Remove the argument from the entry and publish. Search every scope for the key prefix. |
|
||||
| `HKLM\SOFTWARE\GE\ShopDB` `CollectorKey`, both registry views | agent `install.ps1`; `Set-ShopdbCollectorKey.ps1` from `<share>\common\configs\shopdb-collector.txt` | First retire the `Set-ShopdbCollectorKey.ps1` entry (or it rewrites the value every cycle), then a targeted PS1 entry that deletes `CollectorKey` in both views. |
|
||||
| `<share>\common\configs\shopdb-collector.txt` | share staging | Delete the file after the entry above is retired. |
|
||||
| `appsettings.json` `apikey` in the agent install folder | agent package / hand edits | Targeted PS1 entry that sets `apikey` to an empty string. The agent never needs it once enrolled. |
|
||||
| Generated `install.ps1` default `-CollectorKey` (Build site agent downloads) | Settings -> Endpoints -> Build site agent | Delete every kept copy (share, USB, work folders). Rebuild after rotation only if you still need the package. |
|
||||
| Display kiosk `shopdb-key.txt` | display imaging | Confirm what the kiosk uses it for before clearing; a kiosk has no endpoint agent, so anything it reports with the shared key stops at cutover. Delete the file once that is settled. |
|
||||
|
||||
The agent never deletes the shared key itself: other SYSTEM scripts may still
|
||||
read it until this table is done.
|
||||
|
||||
## Rollback
|
||||
|
||||
**One PC:**
|
||||
|
||||
1. Make sure **Shared key allowed** is on.
|
||||
2. Credentials tab -> **Un-enroll** the PC. Its credential stops working
|
||||
immediately; the server admits the shared key for it again.
|
||||
3. Push the previous agent: Upload New/Legacy Agents -> make the legacy build
|
||||
current and upgrade that PC, or re-stamp its share entry to the old hash.
|
||||
4. Remove the PC from the pin entry's targets only if pinning itself is the
|
||||
problem (`Set-ShopDBServerPins.ps1 -Clear` removes the value).
|
||||
|
||||
A 1.20 agent that is un-enrolled but NOT downgraded simply enrolls again on its
|
||||
next cycle; that is expected.
|
||||
|
||||
**Whole fleet:** turn **Shared key allowed** back on first. Every PC that still
|
||||
holds the shared key reports again immediately. Then un-enroll and downgrade
|
||||
PCs as above, in waves.
|
||||
|
||||
## Quick reference
|
||||
|
||||
- Pin of the live server: Server Certificate settings page, or
|
||||
`Get-ShopDBServerPin.ps1 -Url https://<server>`.
|
||||
- Pin of a certificate file: `Get-ShopDBServerPin.ps1 -Path <file>`.
|
||||
- Agent log of enrollment: `%ProgramData%\ShopDB\EndpointAgent\logs`.
|
||||
- Reporter logs: `C:\Logs\Shopfloor\report-asset-*.log`,
|
||||
`report-printers-*.log`. `SHARED_KEY_REFUSED` there means the PC is enrolled
|
||||
or the shared key is off and the reporter could not read the credential;
|
||||
`CREDENTIAL_INVALID` means the PC was un-enrolled and the agent is
|
||||
re-enrolling.
|
||||
Reference in New Issue
Block a user