shopdb-flask/docs/geenforce-api-cutover.md

# GE-Enforce over HTTPS API: cutover reference

This is the operator/developer reference for moving a shopfloor PC type off the
SMB-share GE-Enforce delivery and onto the shopdb HTTPS API. The first cohort
cut over was the displays/kiosks (`gea-shopfloor-display`): share-less,
Intune/Entra-joined PCs with no SFLD share credentials. This doc captures
everything learned doing that, so extending the cutover to the other pc-types
(`gea-shopfloor-cmm`, `-collections`, `-keyence`, `-genspect`, `-heattreat`,
`-partmarker`, `-nocollections`, `common`) does not require re-learning it.

It pairs with the existing docs (which describe the pieces; this one describes
the CUTOVER):

- `docs/GE-ENFORCE.md` - concepts and the plugin
- `docs/GE-ENFORCE-CLIENT.md` - client fetch/report contract
- `docs/GE-ENFORCE-DEPLOY.md` - what must land on a PC
- `docs/GE-ENFORCE-DISPLAY.md` - the display scope specifics
- `/home/camp/projects/pxe/docs/ge-enforce-v2-architecture.md` - the SMB world
  being cut away from

Contents:

1. [Overview and why](#1-overview-and-why)
2. [Server architecture](#2-server-architecture)
3. [Delivery models: smb vs http/inline payloads](#3-delivery-models-smb-vs-httpinline-payloads)
4. [Authoring a scope](#4-authoring-a-scope)
5. [The on-PC client and engine](#5-the-on-pc-client-and-engine)
6. [Bootstrap for share-less PCs](#6-bootstrap-for-share-less-pcs)
7. [Asset reporting via the collector](#7-asset-reporting-via-the-collector)
8. [HARD-WON GOTCHAS](#8-hard-won-gotchas)
9. [How this was verified](#9-how-this-was-verified)
10. [Deploy](#10-deploy)
11. [PLAYBOOK: extending to a new pc-type](#11-playbook-extending-to-a-new-pc-type)
12. [Open items / TODO](#12-open-items--todo)

---

## 1. Overview and why

GE-Enforce v2 delivers desired-state manifests and installer payloads from the
SFLD SMB share (`\\tsgwp00525.wjs.geaerospace.net\shared\dt\shopfloor\`).
Every PC mounts the share with Azure-DSC-provisioned SFLD credentials, reads
`<scope>\manifest.json`, and runs the engine
(`Install-FromManifest.ps1`). That works for the domain fleet but is a hard
dead end for share-less PCs.

The HTTPS API path replaces the share as the transport, keeping the engine and
its detection/self-heal behavior untouched:

| Concern | SMB share | HTTPS API |
|---------|-----------|-----------|
| Manifest source | `<scope>\manifest.json` on the share | `GET /api/geenforce/manifest?pctype=<scope>` (published snapshot) |
| Payload source | share paths (`apps\...`) | `GET /api/geenforce/payload/<sha256>` (content-addressed) |
| Auth | SFLD share credential (DSC) | `geenforce.fetch` service token OR source-IP allowlist |
| Result visibility | log files on the PC | `POST /api/geenforce/report` -> Enforcement Reports UI |
| Versioning | file overwrite + `_meta/history` | immutable published versions, rollback, ETag |

Which PCs MUST use the API: the share-less ones. Displays/kiosks are
Intune/Entra-joined with no SFLD credentials and no domain trust, so SMB is not
an option at all. The rest of the fleet CAN stay on the share (and currently
does); for them the API is an opt-in migration, not a forced one.

West Jefferson facts used throughout this doc:

| Fact | Value |
|------|-------|
| Prod host | `tsgwp00525.wjs.geaerospace.net` |
| App mount | `/shopdb` (IIS, app dir `C:\inetpub\wwwroot\shopdb`, pool `shopdbflask-prod`) |
| BaseUrl clients use | `https://tsgwp00525.wjs.geaerospace.net/shopdb` |
| Prod DB | `shopdb_flask` (MySQL) |
| Client allowlist CIDRs | `10.134.48.0/23,10.48.249.0/26` (the WJ corp/AESFMA shopfloor subnets) |
| Dev/staging instance | `/ops` mount, DB `shopdb_flask_dev`, pool `shopdbflask` |

---

## 2. Server architecture

All server code is in `plugins/geenforce/` (routes: `plugins/geenforce/api/routes.py`).

### Client-facing endpoints (three)

| Endpoint | Method | Auth | What it does |
|----------|--------|------|--------------|
| `/api/geenforce/manifest?pctype=<scope>&phase=runtime` | GET | `geenforce.fetch` token OR IP allowlist | Serves the CURRENT PUBLISHED manifest snapshot for a scope (never the draft). `ETag: "<scopeid>-v<version>"`, `X-Manifest-Version` header, 304 on `If-None-Match`. |
| `/api/geenforce/payload/<sha256>` | GET | `geenforce.fetch` token OR IP allowlist | Streams a payload blob by content hash: blob store first (`service.blob_path`), then inline `ManifestPayload`. ETag = the hash. Per-IP rate limited (120/min default) and size-capped (512 MB default, 413 above). |
| `/api/geenforce/report` | POST | `geenforce.report` token OR IP allowlist | Records one enforcement cycle via `service.record_enforcement_report`: hostname, scopename, appliedversion, enforcerversion, counts, per-entry results. Upserts the current report per (hostname, scopename, phase); older reports kept as history. |

Plus the collector for asset reporting (section 7): `POST
/api/collector/computers` in `shopdb/core/api/collector.py` - a DIFFERENT auth
domain (`collector.ingest`), NOT covered by the geenforce allowlist.

### Auth model

`_require_service_token(scope)` in `routes.py` is the decorator factory. Two
paths, fail-closed (neither -> 401):

1. A managed service token with the scope (`geenforce.fetch` for
   manifest/payload, `geenforce.report` for report), sent as `X-API-Key` or a
   Bearer PAT (`authorized_service_token`). A token may carry
   `resourcescopelist` bindings: a bound token can only fetch its own scope's
   manifest (403 otherwise) and only blobs those scopes' published manifests
   reference (`service.blob_referenced_by_scopes`, 404 so hashes cannot be
   probed). Displays get a token bound to `gea-shopfloor-display` (see
   GE-ENFORCE-DISPLAY.md).
2. The IP allowlist: `_ip_allowlisted()` checks the caller against the setting
   `geenforce_allowed_cidrs` (comma-separated CIDRs/IPs, empty = disabled).
   Network trust replaces the shared secret for a vaulted fleet. The
   allowlisted path has no resource-scope binding (unrestricted).

The token paths use RBAC permissions the plugin registers in
`GeEnforcePlugin.get_permissions()`: `geenforce.manage` (edit),
`geenforce.publish` (ship), `geenforce.fetch` and `geenforce.report`
(client service tokens). Admin CRUD/publish/simulate/compliance routes are JWT
plus `geenforce.manage`/`geenforce.publish`.

### Why the allowlist uses remote_addr (the IIS XFF dependency)

`_trusted_client_ip()` returns `request.remote_addr`, NOT the raw
`X-Forwarded-For` header. Proxies APPEND to X-Forwarded-For, so its first hop
is attacker-controlled: parsing it (as `_client_ip()` does, acceptably, for
rate limiting only) would let any caller send `X-Forwarded-For:
<allowlisted-ip>` and bypass the token entirely.

`remote_addr` is trustworthy only because of a two-piece chain that MUST stay
in place:

1. The IIS URL-Rewrite rule in the `/shopdb` web.config OVERWRITES (not
   appends) the inbound `X-Forwarded-For` with `REMOTE_ADDR`, the real TCP
   peer.
2. waitress runs with `--trusted-proxy=127.0.0.1
   --trusted-proxy-headers=x-forwarded-for`, so it derives `remote_addr` from
   that overwritten header only when the request comes from IIS on localhost.
   A client that somehow hits waitress directly is not a trusted proxy, so its
   `remote_addr` is its own real peer address. Either way, real client IP.

If the IIS rule is ever removed, the allowlist stops matching (fails closed to
401 for token-less clients) - it does NOT become spoofable, but the fleet
falls back to cached manifests. Keep the rule on.

---

## 3. Delivery models: smb vs http/inline payloads

Every `ManifestEntry` carries a `payloadsource` (`plugins/geenforce/serializer.py`
and `importer.py`):

| PayloadSource | Meaning | Manifest emission |
|---------------|---------|-------------------|
| `smb` (default) | Entry installs from the share exactly as v2 does; the entry's `Installer`/`Script`/`Source` is a share-relative path. | Nothing emitted - share manifests round-trip byte-identical, parity preserved. |
| `http` | Payload lives in the server's content-addressed blob store (`instance/geenforce/payloads/<sha256>`, registry row `ManifestBlob`). For big files (MSIs, EXEs). Upload via `flask geenforce add-payload <file>` or `service.store_blob`. | `PayloadSource`, `PayloadSha256`, `PayloadRef` keys on the entry. |
| `inline` | Payload bytes live IN the DB (`ManifestPayload`, <= 1 MB) - small scripts and configs. Attach via `service.store_inline_payload(entry, filename, contenttype, rawbytes)` or `POST /api/geenforce/entries/<id>/payload`. | Same three keys. |

Both `http` and `inline` are served from the same client URL:
`GET /api/geenforce/payload/<sha256>` (blob store checked first, then inline).
The sha256 IS the integrity contract - the client re-hashes after download.

### How the client stages payloads (Resolve-ShopdbPayloads)

`Resolve-ShopdbPayloads` in `plugins/geenforce/client/ShopdbEnforceClient.psm1`
is the bridge that lets the UNCHANGED engine install share-less:

1. For each entry with `PayloadSha256` and `PayloadSource` http/inline, call
   `Get-ShopdbPayload`: download to
   `C:\ProgramData\ShopDB\geenforce\payloads\<sha><ext>` (ext from
   `PayloadRef`), verify the sha256, keep it as a content-addressed
   last-known-good cache (a cache hit only counts if the bytes still hash
   right).
2. Rewrite the entry's path field to the LEAF filename of the staged file
   (`Split-Path -Leaf`) - NOT the absolute path. Field by Type:
   `Installer` for MSI/EXE/CMD/BAT/INF, `Script` for PS1, `Source` for File.
3. Write a sibling `<scope>.resolved.json` manifest and return its path (or
   the original path if nothing needed resolving). A payload that cannot be
   fetched/verified THROWS - the runner's fail-safe catch decides what happens.

The runner (`Invoke-ShopdbEnforce.ps1`) then sets the engine's
`-InstallerRoot` to that same payloads directory, so the engine's
`Join-Path $InstallerRoot <leaf>` resolves to the staged file. `smb` entries in
a mixed manifest are left untouched and still resolve against the share (a PC
that has it).

---

## 4. Authoring a scope

Two authoring paths, both ending in `service.replace_scope_draft(scopename,
phase, manifest_dict)` (idempotent draft rebuild - published versions are never
touched by a re-import):

### A. import-share: adopt an existing SMB manifest

```
flask geenforce import-share --shareroot <path> [--scope <name>] [--preinstall <path>]
flask geenforce publish <scopename> [--phase runtime] [--notes "..."]
```

`importer.discover_share` walks the share root and ingests
`common/manifest.json`, `display/manifest.json`, and every
`gea-shopfloor-*/manifest.json` (skipping `.bak` variants). Entries come in as
`smb` payloads. Run `flask geenforce parity --shareroot <path>` first (Gate A):
proves import+re-export is behaviorally lossless before anything ships.

### B. authoring in code: seed_display_scope as the template

`plugins/geenforce/seed_display_scope.py` is the reference for a scope that
never existed on the share. Pattern:

- Build the manifest dict in Python (`build_display_manifest()`): four
  `Type=Registry` drift-heal entries re-asserting the Edge kiosk relaunch
  policies from imaging (`09-Setup-Display.ps1`), one inline PS1 dispatcher,
  one inline PS1 always-on script. Registry heals use
  `DetectionMethod=ValueMatches` against the same path/name they write, so
  drift self-heals; the PS1s use `DetectionMethod=Always` and are idempotent.
- The dispatcher (`Invoke-DisplayKioskDispatch.ps1`, generated by
  `build_dispatcher_script()`) reads `C:\Enrollment\display-type.txt`, maps
  the subtype through the data-driven `DISPLAY_TYPE_TARGETS` table
  (Dashboard -> `/shopfloor`, Lobby -> `/tv`, 3DPrintRoom -> `/parts-kiosk`),
  and writes an all-users Startup shortcut (`ShopDB Kiosk.lnk`) launching Edge
  `--kiosk` fullscreen at `{BaseUrl}{route}`. It does NOT Start-Process Edge
  (see gotchas). Base URL comes from HKLM `BaseUrl`, falling back to the WJ
  host.
- `seed_display_scope(publish=False)`: `replace_scope_draft`, flush (entries
  need entryids), then `service.store_inline_payload(...)` for each script
  entry (sets `payloadsource='inline'`, `payloadsha256`, `payloadref`),
  optionally `service.publish_scope(...)`, commit. Draft rebuild is
  idempotent; publish always creates a NEW version.

Run it on the server:

```
cd C:\inetpub\wwwroot\shopdb
$env:FLASK_APP = 'shopdb'
'from plugins.geenforce.seed_display_scope import seed_display_scope; print(seed_display_scope(publish=True))' | venv\Scripts\python -m flask shell
```

Expected: `{scopeid, entrycount: 6, entrytypes: [Registry x4, PS1, PS1],
dispatchersha256, alwaysonsha256, publishedversion: N}`.

### Publishing

`service.publish_scope` freezes the draft (rendered by
`serializer.scope_to_json`) into an immutable `ManifestPublishedVersion` and
flips `iscurrent`. Clients only ever see published versions.
`rollback_scope` re-currents an older version. Also available over the API
(`POST /scopes/<id>/publish`, permission `geenforce.publish`) and the
GE-Enforce UI.

### Attaching payloads

- Inline (<= 1 MB): `service.store_inline_payload` in code, or
  `POST /api/geenforce/entries/<entryid>/payload` (multipart file).
- Blob (`http`): `flask geenforce add-payload <filepath>` prints the sha256;
  set `PayloadSource=http` + `PayloadSha256` (+ `PayloadRef` for the
  filename/extension) on the entry.

Publish AFTER attaching - the published JSON is what carries the
`PayloadSha256` the client fetches, and blob access for resource-bound tokens
is checked against the CURRENT published manifest.

---

## 5. The on-PC client and engine

### Config: HKLM:\SOFTWARE\GE\ShopDB

| Value | Used by | Notes |
|-------|---------|-------|
| `BaseUrl` | enforce client + kiosk dispatcher | e.g. `https://tsgwp00525.wjs.geaerospace.net/shopdb`. Required. |
| `ApiToken` | enforce client | `geenforce.fetch` (+ report) PAT. OPTIONAL - a token-less client relies on the IP allowlist (`Get-ShopdbConfig` treats BaseUrl-only as valid). |
| `CollectorKey` | `Report-AssetToShopDB.ps1` | `collector.ingest` PAT. REQUIRED for asset reporting (allowlist does not cover the collector). |

The key's ACL is restricted to SYSTEM + Administrators (the bootstrap does
this) so the kiosk auto-login user cannot read the PATs.

### The pieces on disk (kiosk layout, `C:\ProgramData\GE-Enforce`)

- `Invoke-ShopdbEnforce.ps1` - the runner
- `ShopdbEnforceClient.psm1` - the client module
- `lib\Install-FromManifest.ps1` - the engine (>= 2.6)
- `Report-AssetToShopDB.ps1` - the asset collector
- Cache: `C:\ProgramData\ShopDB\geenforce\` (`<scope>.json`, `.etag`,
  `.version`, `payloads\`), logs `C:\Logs\Shopfloor\`

### Scheduled tasks (SYSTEM, RunLevel Highest)

| Task | Runs | Interval |
|------|------|----------|
| `ShopDB GE-Enforce` | `Invoke-ShopdbEnforce.ps1 -Scope <scope> -EnginePath <engine> -BaseUrl <url>` | AtStartup + every 15 min |
| `ShopDB Asset Report` | `Report-AssetToShopDB.ps1` | AtStartup + every 60 min |

Tokens are NOT in the task arguments (visible in task XML) - the scripts read
them from HKLM.

### The runner flow (Invoke-ShopdbEnforce.ps1)

1. `Get-ShopdbConfig` (params override registry). No BaseUrl -> exit 0, retry
   next cycle.
2. `Sync-ShopdbManifest -Scope <scope>`: ETag-conditional GET; 200 validates
   the JSON before overwriting the cache (a proxy error page served as 200
   must not clobber last-known-good); 304 or any network failure -> cached
   copy. Nothing at all -> Windows event log entry (source `ShopdbEnforce`,
   id 1001) plus a best-effort failure report so it is visible server-side,
   then exit 0.
3. `-ShadowMode` (with `-ShareManifestPath`): `Compare-ShopdbShadow` logs
   name/order diffs, engine runs against the SHARE (zero behavior change).
   This is the first step of every cutover.
4. Cutover mode: optional `-IncludeCommon` merges the fleet `common` scope
   (`Merge-ShopdbManifests`: common's unique entries first, pctype wins on
   Name conflict). OFF by default - a scope is enforced ALONE and displays are
   self-sufficient. Then `Resolve-ShopdbPayloads` stages http/inline payloads
   (section 3).
5. Engine call (the integration point):
   `& $EnginePath -ManifestPath <resolved> -PCType $Scope -InstallerRoot <payloads dir> -LogFile <log>`.
6. `ConvertTo-ShopdbSummary` normalizes whatever came back (summary object,
   array of emitted objects, bare int, $null) into
   `@{Installed;Skipped;Failed;Filtered;Results;EnforcerVersion}`, then
   `New-ShopdbReport` maps to the lowercase wire contract and
   `Send-ShopdbReport` POSTs it. All best-effort; the whole script exits 0 no
   matter what (fail-safe: a broken web app never breaks a PC).

### The engine contract (Install-FromManifest.ps1, lib 2.6)

Mandatory params: `-ManifestPath`, `-InstallerRoot`, `-LogFile`; optional
`-PCType`, `-PCSubType`. Entry Types: MSI, EXE, CMD/BAT, PS1, INF, File,
Registry. Detection: Registry, File, FileVersion, Hash, MarkerFile,
ValueMatches, pnputil, Always. Filters: PCTypes (with old/new-name alias
groups), TargetHostnames, TargetMachineNumbers, `_CmmVersion`. Exit 0/1/2
unchanged for the SMB path; NEW in the API cutover: the engine ends with
`Write-Output` of a summary pscustomobject
(`Installed/Skipped/Failed/Filtered/EnforcerVersion/Results`), which is the
only thing on the success stream (logs go via Write-Host), so
`& $EnginePath ...` captures it cleanly.

`SelfHealed` on a result means a REAL drift correction (a detected-missing
entry that got reinstalled). `Always`/no-detection entries install every cycle
by design and are not flagged, so the server-side status derivation
(`service.record_enforcement_report`: failed > selfhealed > ok) stays honest.

### The display-type.txt dispatcher pattern

For kiosks, per-subtype behavior does not fork the scope: ONE scope
(`gea-shopfloor-display`), one inline dispatcher entry that reads
`C:\Enrollment\display-type.txt` at enforce time and acts on it. Subtype
changes are a one-file edit plus the next 15-min cycle, and retargeting a
subtype is a table edit in `seed_display_scope.py` + republish. The dispatcher
writes an all-users Startup shortcut (never Start-Process - see gotchas), and
sweeps stale kiosk launchers first (its own `ShopDB Kiosk*.lnk`, the imaging
installers' `GE Aerospace Dashboard*`/`GE Aerospace Lobby*` shortcuts, any
browser .lnk with `-kiosk`-ish args or shopdb URLs, and .url files pointing at
the kiosk routes).

---

## 6. Bootstrap for share-less PCs

A share-less PC cannot pull its first files from the share, so the bootstrap
itself is downloadable from the web app.

`Install-ShopdbKiosk.ps1` (source of truth:
`/home/camp/pxe-images/shopdb-migration/kiosk-installer/`) is hosted at
`C:\inetpub\wwwroot\shopdb\installers\kiosk\` and downloadable at
`{BaseUrl}/installers/kiosk/Install-ShopdbKiosk.ps1`. Run elevated on the PC:

```
Set-ExecutionPolicy Bypass -Scope Process -Force
$u = 'https://tsgwp00525.wjs.geaerospace.net/shopdb/installers/kiosk/Install-ShopdbKiosk.ps1'
Invoke-RestMethod $u -OutFile "$env:TEMP\Install-ShopdbKiosk.ps1"
& "$env:TEMP\Install-ShopdbKiosk.ps1" -DisplayType Lobby -CollectorKey 'shopdb_pat_...'
# add -ShopdbToken 'shopdb_pat_...' only if the subnet is NOT allowlisted
```

What it does (idempotent; re-running is also the manual update path):

1. Writes `C:\Enrollment\display-type.txt` (the subtype) and
   `C:\Enrollment\pc-type.txt` (the scope, default `gea-shopfloor-display`).
2. Writes HKLM `BaseUrl` [+ `ApiToken`] + `CollectorKey`, then locks the key
   ACL to SYSTEM + Administrators.
3. Downloads runner + module + engine + collector from
   `{BaseUrl}/installers/kiosk/` over HTTPS (TLS 1.2 forced).
4. Registers the two SYSTEM tasks (section 5).
5. Starts both once so the PC is live immediately.

IIS prerequisite: the `installers\kiosk` web.config MUST carry
`<staticContent>` MIME maps for `.ps1`/`.psm1` (`text/plain`) or IIS 404.3s
the downloads (see gotchas).

Delivery options for the bootstrap itself:

- Imaging-baked: the display image runs it (or lays down the same state) at
  imaging time - see `project-display-self-contained`.
- Installer push: Intune/hand-run the one-liner above on an already-deployed
  PC. This is how the pilot kiosks were done
  (`shopdb-migration/run-on-kiosk-F.txt`).

---

## 7. Asset reporting via the collector

`Report-AssetToShopDB.ps1` (in the kiosk bundle; also deployed in the SMB
`common\` scope for the share fleet) POSTs the PC's identity to:

```
POST {BaseUrl}/api/collector/computers
X-API-Key: <collector.ingest PAT or COLLECTOR_API_KEY env key>
```

Auth (`shopdb/core/api/collector.py`, `_check_collector_auth`): a managed
token scoped `collector.ingest` (Bearer or X-API-Key) OR the
`COLLECTOR_API_KEY`/`COLLECTOR_API_KEY_COMPUTERS` env key. The GE-Enforce IP
allowlist does NOT apply here - the collector always needs a key, read from
HKLM `CollectorKey` (or the manifest entry's `Args -ApiKey`).

Schema (`plugins/computers/plugin.py`, `get_collector_schema`): identity field
`hostname` (required); optional `machinenumber`, `pctype`, `pcsubtype`,
`serialnumber`, `loggedinuser`, `lastboottime`, `lastcheckin`, `ipaddress`,
`vendorname`, `modelnumber`, `osname`, `installedsoftware`, `defaultprinter`,
`printers`. All lowercase concatenated (the project naming convention).

`apply_collector_payload` upserts idempotently by hostname (falls back to
`Asset.assetnumber`), creates the Asset+Computer when missing, maps
`machinenumber` -> `Asset.assetnumber` (imaging placeholder `9999` skipped both
client- and server-side), `pctype` -> ComputerType via the settings mapping,
and creates Vendor/Model/OS rows as needed. Fields not posted are not touched -
a partial read never blanks a good value, so the script only includes fields it
actually resolved.

Client details worth keeping: machine number resolution order is eDNC registry
`MachineNo` (WOW6432Node then native) -> `C:\Enrollment\cmm\cmmid.txt` ->
`C:\Enrollment\machine-number.txt`; the reported `ipaddress` is filtered to
the corp ranges (same two CIDRs as the allowlist) so a machine-LAN controller
NIC never lands in shopdb.

---

## 8. HARD-WON GOTCHAS

Read this section before touching ANY of the moving parts. Every bullet cost
real debugging time. Format: symptom -> cause -> fix.

- **PS crash "property 'X' cannot be found" under Set-StrictMode** ->
  the module runs `Set-StrictMode -Version Latest`, and engine
  results/summaries arrive as EITHER hashtables or PSCustomObjects with
  varying key casing; direct `$obj.Key` access on an absent key throws ->
  route every dynamic property read through `Get-ShopdbProperty` (handles
  both shapes, case-insensitive, returns $null when absent). Never dot into
  parsed JSON or engine output directly.
- **Register-ScheduledTask rejects the repeating trigger** -> passing
  `-RepetitionDuration [TimeSpan]::MaxValue` serializes to an out-of-range
  Duration the Task Scheduler XML schema rejects -> use
  `-RepetitionInterval` ALONE; it defaults to indefinite repetition
  (verified Win11 / PS 5.1). See `Register-SystemTask` in
  `Install-ShopdbKiosk.ps1`.
- **Engine exits 2 / "InstallerRoot not found"** -> `-InstallerRoot` and
  `-LogFile` are MANDATORY engine params and InstallerRoot must EXIST ->
  the runner always passes both and pre-creates the payloads dir before the
  engine call. Any new caller must do the same.
- **http payload "not found: C:\...\C:\..." (path doubling)** -> the engine
  resolves entry paths as `Join-Path $InstallerRoot <field>`; writing the
  staged payload's ABSOLUTE path into the entry made the engine double it ->
  `Resolve-ShopdbPayloads` writes the LEAF filename only, and the runner sets
  `-InstallerRoot` to the payloads cache dir. Keep those two in lockstep.
- **Manifest fetch "works" but parsing fails / cache garbage** -> if the
  manifest is served with a non-JSON content type, PowerShell 5.1
  `Invoke-WebRequest` `.Content` comes back as a `byte[]` instead of a string
  -> the server route returns `mimetype='application/json'` (see
  `get_manifest`); any mock server or proxy in the chain must do the same.
  The client also validates JSON before overwriting last-known-good.
- **Bootstrap download 404 (HTTP 404.3)** -> IIS refuses to serve unknown
  static extensions; `.ps1`/`.psm1` have no default MIME map -> add
  `<staticContent><mimeMap fileExtension=".ps1" mimeType="text/plain" />`
  (and `.psm1`) in the `installers\kiosk` web.config, with `<remove>` first
  if inherited.
- **IP allowlist spoofable / mysteriously not matching** -> raw
  `X-Forwarded-For` is attacker-controlled (proxies append; first hop is the
  caller's to write) -> `_ip_allowlisted` uses `request.remote_addr` via
  `_trusted_client_ip`, which is only correct because the IIS URL-Rewrite
  rule OVERWRITES X-Forwarded-For with REMOTE_ADDR and waitress trusts only
  127.0.0.1 as proxy. The rule is a hard dependency: never remove it, and
  verify the spoof is closed after server changes
  (`curl -H "X-Forwarded-For: 10.134.48.10"` from a non-allowlisted host
  must get 401).
- **Kiosk browser never appears though the dispatcher "ran fine"** -> the
  enforce task runs as SYSTEM in session 0, which has no interactive
  desktop; `Start-Process msedge.exe` opens INVISIBLY there -> write an
  all-users Startup shortcut
  (`C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup\ShopDB Kiosk.lnk`)
  and let the auto-login user launch it in a visible session. This is why
  the dispatcher is shortcut-based.
- **TLS/transport errors on older images ("could not create SSL/TLS secure
  channel")** -> Windows PowerShell 5.1 does not reliably negotiate TLS 1.2
  by default -> every network helper calls `Set-ShopdbTls`
  (`[Net.ServicePointManager]::SecurityProtocol = Tls12`) first; the
  bootstrap and collector force it too. Any new script that touches the API
  must do the same.
- **Secrets readable by the kiosk user / visible in Task Scheduler** ->
  default HKLM\SOFTWARE ACL grants BUILTIN\Users read, and task arguments
  are world-readable in the task XML -> restrict the
  `HKLM:\SOFTWARE\GE\ShopDB` key ACL to SYSTEM + Administrators (the
  bootstrap does), and NEVER put a token in a task's `-Argument` string -
  scripts read `ApiToken`/`CollectorKey` from HKLM at run time.
- **Enforcement Reports show 0/0/0 with no per-entry detail** -> the engine
  historically returned nothing on the success stream, so the runner had no
  counts to report -> the 2.6 engine emits the summary object as its ONLY
  `Write-Output` (all logging is Write-Host), and
  `ConvertTo-ShopdbSummary` tolerates non-compliant engines by zero-filling.
  If reports go 0/0/0 again, the engine on that PC is pre-summary - update
  it.
- **Two kiosk browsers fighting / stale kiosk launch after retarget** -> old
  installs left their own Startup launchers behind, in several flavors ->
  the dispatcher's sweep must match ALL of: single- AND double-dash `-kiosk`
  arguments (the regex `-kiosk` matches both), shortcuts whose args carry
  shopdb URLs (`tsgwp00525`, `/shopdb/`, the dead `shopfloor-dashboard`
  route), the imaging installers' `GE Aerospace Dashboard*` / `GE Aerospace
  Lobby*` shortcut names, and `.url` files pointing at the kiosk routes.
  Extend the sweep whenever a new launcher naming appears.

---

## 9. How this was verified

Two complementary verification passes; keep BOTH for future cohorts because
they catch different bug classes:

- **Code review** finds logic bugs: the XFF spoof hole, the StrictMode
  absent-key crashes, the status-derivation trap (`installed>0` is not
  self-heal), the report casing mismatch.
- **VM smoke test** finds integration/OS-version bugs: the
  RepetitionDuration serialization rejection, the .ps1 MIME 404, session-0
  invisibility, TLS negotiation, byte[] vs string response content - none of
  which a read of the code surfaces.

The VM rig:

- The win11 virt-manager VM (see `project-win11-vm` /
  `reference-vm-qga-as-system` memory), driven by
  `/home/camp/pxe-images/ednc-bins/qga-run.py` - qemu guest agent
  `guest-exec`, which runs PowerShell AS SYSTEM. That matters: the scheduled
  tasks run as SYSTEM, so testing as SYSTEM reproduced the session-0 and
  profile-less behaviors an interactive test would have masked.
- A mock HTTP server on the host exposing `/api/geenforce/manifest` (served
  `application/json`), `/api/geenforce/payload/<sha>`, and capturing the
  `/api/geenforce/report` POST body. This exercised the full client loop -
  ETag/304, cache fallback (kill the server mid-test), payload hash
  verification, resolved-manifest rewrite, engine run, summary -> report
  mapping - without touching prod.
- Full pass = bootstrap installer run end to end in the VM, then assert: both
  tasks registered, HKLM written + ACLed, manifest cached, payloads staged by
  sha, Startup shortcut written, report captured with real counts.

---

## 10. Deploy

TWO independent channels. Confusing them is the classic mistake: the client/
engine/collector/bootstrap are pxe-images SHARE artifacts, NOT deployed by the
git pipeline.

### Channel 1: backend + frontend (the git .cmd pipeline)

Prod is air-gapped from dev; code moves via a git bundle on the share
(`\\172.16.9.9\pxe-images\github\shopdb-flask-pub.bundle`) and the .cmd
scripts in `/home/camp/pxe-images/github/`, run on the work PC:

1. `pull-shopdb-bundle.cmd` - fetch the bundle into the local clone
   (ff-only).
2. `update-shopdb-github.cmd` - push the clone to GitHub.
3. `update-dev-server.cmd` - robocopy the clone to `X:` (dev `/ops` tree) +
   the `/ops`-base frontend dist. Validate on `/ops` FIRST.
4. `update-prod-server.cmd` - robocopy to `Y:` (`C:\inetpub\wwwroot\shopdb`)
   + the `/shopdb`-base dist (`frontend-dist-subpath-shopdb`). Both scripts
   carry instance guards (web.config MOUNT_PATH check) - do not bypass them.

Then RDP: `Restart-WebAppPool shopdbflask-prod`, and if migrations/deps
changed: `flask db upgrade`, `flask plugin upgrade-all`, `flask seed
permissions`, `flask seed settings`.

(The fast path used during the pilot - robocopy just the changed plugin files
from `shopdb-migration\prod-patch-geenforce\` + pool restart, per
`deploy-server-patch.txt` - works, but the same commits must ALSO go through
the bundle pipeline or prod drifts from git.)

### Channel 2: client + engine + collector + bootstrap (the kiosk bundle)

These live at `/home/camp/pxe-images/shopdb-migration/kiosk-installer/` and
deploy by robocopy from the work PC (Z: = share, Y: = prod app dir):

```
robocopy Z:\shopdb-migration\kiosk-installer Y:\installers\kiosk /E
```

That directory (bundle contents: `Install-ShopdbKiosk.ps1`,
`Invoke-ShopdbEnforce.ps1`, `ShopdbEnforceClient.psm1`,
`lib\Install-FromManifest.ps1`, `Report-AssetToShopDB.ps1`, `web.config` with
the MIME maps) IS the distribution point - PCs download from
`{BaseUrl}/installers/kiosk/`. Reference copies of the client kit also live in
the repo at `plugins/geenforce/client/` and the engine's source of truth is
`/home/camp/pxe-images/common/lib/Install-FromManifest.ps1`; when the engine
or client changes, update the kiosk bundle copy too (nothing syncs it
automatically). PCs pick up new bytes by re-running the bootstrap one-liner.

### Server prerequisites (once per site, all three or token-less clients 401)

1. Publish the scope(s) - `seed_display_scope(publish=True)` or
   `flask geenforce publish <scope>`.
2. Seed `geenforce_allowed_cidrs` = `10.134.48.0/23,10.48.249.0/26`
   (Settings rail > GE-Enforce Settings, or SQL upsert into `settings`).
3. Mint tokens (Settings > API Tokens, Restrict permissions ON):
   `collector.ingest` (required, the kiosk `-CollectorKey`) and
   `geenforce.fetch` (fallback for non-allowlisted subnets; resource-bind it
   to the scope).
4. Keep the IIS URL-Rewrite XFF-overwrite rule enabled (section 2).

---

## 11. PLAYBOOK: extending to a new pc-type

Checklist for cutting any of the remaining scopes (`gea-shopfloor-cmm`,
`-collections`, `-keyence`, `-genspect`, `-heattreat`, `-partmarker`,
`-nocollections`, `common`) over to the API.

1. **Decide the delivery model.** Does this pc-type keep SMB access? If yes,
   the cheap cutover is manifest-over-API + payloads-still-smb (entries stay
   `smb`, nothing to upload, the engine resolves share paths as today). Only
   a genuinely share-less PC needs http/inline payload conversion. Note the
   payload endpoint's 512 MB default ceiling
   (`GEENFORCE_PAYLOAD_MAX_BYTES`) before promising huge installers over
   HTTPS.
2. **Get the scope into shopdb.** Existing share manifest:
   `flask geenforce parity` then `flask geenforce import-share --scope
   <name>`. New/reworked scope: author in code following
   `seed_display_scope.py` (registry heals with ValueMatches detection,
   idempotent Always PS1s, data-driven tables for anything per-subtype).
3. **Convert payloads (share-less only).** Small scripts/configs ->
   `store_inline_payload` / the entry payload upload endpoint. Installers ->
   `flask geenforce add-payload <file>`, set
   `PayloadSource=http` + `PayloadSha256` + `PayloadRef` on the entry.
   Remember: `PayloadRef`'s extension decides the staged filename's
   extension.
4. **Publish.** New version every publish; clients converge within one
   enforce cycle. Verify with
   `curl "{BaseUrl}/api/geenforce/manifest?pctype=<scope>"` from an
   allowlisted host.
5. **Auth for the PCs.** Subnet already inside
   `10.134.48.0/23,10.48.249.0/26` -> token-less, nothing to do. New subnet
   -> add its CIDR to `geenforce_allowed_cidrs` (Settings rail validates).
   Not network-trustable -> mint a `geenforce.fetch` token resource-bound to
   the scope and deliver it to HKLM `ApiToken`.
6. **Bootstrap the client.** Share-attached fleet: adapt the dispatcher /
   `Install-GEEnforce.ps1` path (the pilot flow in
   `shopdb-migration/kiosk-api-pilot.txt`: shadow first, then flip, then
   DISABLE the old share enforce task so the two do not fight). Share-less:
   the `Install-ShopdbKiosk.ps1` pattern - generalize `-Scope` and skip the
   display-only pieces. Decide `-IncludeCommon`: displays run without it;
   a non-display share-less PC that needs the fleet-wide common entries over
   HTTPS turns it on AND requires common's entries to be payload-converted
   first (an SMB-payload common entry will fail on a share-less PC).
7. **Run SHADOW mode first** on one pilot PC
   (`-ShadowMode -ShareManifestPath <share manifest>`): fetch + compare +
   report with zero behavior change. Watch the shadow diff log lines and the
   Enforcement Reports row before flipping.
8. **Re-read section 8 (gotchas).** Especially: LEAF filenames, StrictMode
   property access, SYSTEM/session-0, task trigger serialization, MIME maps
   if you host new downloadables.
9. **Verify on the VM** (section 9) before the pilot PC: bootstrap +
   enforce cycle against a mock or the dev `/ops` instance, as SYSTEM via
   qga-run.py.
10. **Pilot one PC, then the cohort.** Keep the rollback in your pocket:
    disable/remove the new task, re-enable the share task, remove HKLM
    `BaseUrl` - the share path is untouched by all of this.
11. **pc-type mapping.** Make sure the collector's pctype mapping
    (computers plugin settings, `pctypemap`) covers the scope name so asset
    reports do not warn `no ComputerType mapping`.

---

## 12. Open items / TODO

- **Name resolution for reported users.** `loggedinuser` lands as a bare
  username; resolving it to a display name depends on either the
  `wjf_employees` `First_Name`/`Last_Name` data or shopdb User accounts
  existing for shopfloor users. Not wired up; reports show raw usernames
  until it is.
- **Old imaging-installer registry cleanup.** Displays imaged before the API
  cutover carry leftover state from the old imaging-time kiosk installers
  (superseded shortcuts are already swept by the dispatcher; stale registry
  values are not yet cleaned). A cleanup entry in the display scope is the
  natural vehicle.
- **The SMB fleet is still on the share.** Only displays/kiosks are on the
  API. cmm/collections/keyence/genspect/heattreat/partmarker/nocollections/
  common still enforce from SFLD; section 11 is the path. Shadow mode makes
  each migration observable before it changes anything.
- **3DPrintRoom kiosk target is a placeholder.** `DISPLAY_TYPE_TARGETS`
  points it at `/parts-kiosk`; confirm the real route with the floor team
  before publishing to production 3D-print-room displays (flagged in
  `seed_display_scope.py`).
- **Fast-path prod patches vs git.** The `prod-patch-geenforce` robocopy
  path can leave prod ahead of the repo; reconcile by pushing the same
  changes through the bundle pipeline (section 10, channel 1).