diff --git a/docs/proposals/ge-enforce-plugin.md b/docs/proposals/ge-enforce-plugin.md index 49f5046..0a3ca09 100644 --- a/docs/proposals/ge-enforce-plugin.md +++ b/docs/proposals/ge-enforce-plugin.md @@ -111,15 +111,25 @@ convention): - `manifestscopes` - one row per imaging PC type / scope. - `scopeid` PK - - `scopename` (e.g. `gea-shopfloor-cmm`; unique) - - `computertypeid` FK -> `computertypes` (this REPLACES the thin - `pctypemap_` setting; the mapping becomes a column here) + - `scopename` (e.g. `gea-shopfloor-cmm`) - `phase` enum (`runtime` | `preinstall`) + - UNIQUE (`scopename`, `phase`), NOT `scopename` alone: `common` exists in + runtime, and a scope name can appear in both phases. Note the phases are + shaped differently - runtime is many per-pctype scopes (one manifest file + each), preinstall is ONE flat manifest gated internally by `PCTypes`, so + preinstall is modeled as a single `phase=preinstall` scope, not per-pctype + scopes. + - `computertypeid` FK -> `computertypes` (this REPLACES the thin + `pctypemap_` setting; the mapping becomes a column here). + Runtime-scope only; null for the preinstall scope. + - `measuringtooltypeid` FK -> `measuringtooltypes`, nullable (metrology + scopes: what device this scope implies; keeps imaging + collector agreed, + see section 11). - `manifestversion` (string, mirrors manifest `Version`) - `description`, `isactive` - `iscommon` bool (the `common/` fleet-wide scope) -- `manifestentries` - one row per Applications[] entry. +- `manifestentries` - one row per Applications[] entry (the working/draft copy). - `entryid` PK, `scopeid` FK - `sortorder` int (preserves array order; the ordering contract) - `name`, `entrytype` (MSI/EXE/.../Registry), `comment` @@ -128,6 +138,12 @@ convention): `regpath`, `regname`, `regvalue`, `regtype` - `payloadsource` enum (`smb` | `http` | `inline`) + `payloadref` (see section 5) + - `payloadsha256` - integrity hash of the payload bytes, INDEPENDENT of the + detection method. Mandatory for `http`/`inline` payloads; optional for + `smb`. Do NOT reuse `detectionvalue` for this - `detectionvalue` is a + SHA256 only when `detectionmethod = Hash`; an MSI with `Registry`/ + `FileVersion` detection has no payload hash, so an HTTP fetch would + otherwise run unverified bytes (see section 5). - detection columns: `detectionmethod`, `detectionpath`, `detectionname`, `detectionvalue`, `detectionpattern` - gates: `cmmversion`, plus child tables for the multi-value filters @@ -135,6 +151,15 @@ convention): - preinstall flags: `preenrollment`, `killafterdetection`, `pctypesstrict` - `isactive` +- `manifestpublishedversions` + `manifestpublishedentries` - immutable + published snapshots. Editing `manifestentries` never affects the fleet; a + "publish" action freezes the current draft into a new numbered snapshot. The + client is ALWAYS served the latest published snapshot for a scope, never the + live draft, so a half-finished edit can never reach a PC. Rollback = mark an + older snapshot current (this is the post-cutover safety net that replaces the + "revert the dispatcher" rollback once the on-share JSON is retired). Mirrors + the current `_meta/history/-.json` backups, but authoritative. + - `manifestentrypctypes`, `manifestentryhostnames`, `manifestentrymachinenumbers` - child rows for the ANDed multi-value filters (one value per row, wildcards stored verbatim as patterns) @@ -142,23 +167,27 @@ convention): - `manifestinusechecks` + `manifestinusecheckprocesses` - the nested InUseCheck object and its Processes[] child list -- `pctypealiases` - the old<->new name alias graph (seed from - `Install-FromManifest.ps1:463-475`), so the server can resolve/validate - PCTypes the same way the engine does. +- `pctypealiases` - a MIRROR of the old<->new name alias graph from + `Install-FromManifest.ps1:463-475`, for server-side resolve/validate only. + The engine lib stays the single source of truth (see section 10); shopdb + never becomes the authority the client depends on for aliases. -The JSON the client receives is REBUILT from these rows in exact array order. -It must byte-for-byte match what the current engine expects (round-trip tested -against the live manifests). +The JSON the client receives is REBUILT from a published snapshot in exact +array order. Parity with the current engine is proven by BEHAVIORAL equivalence, +not byte-identity (see section 9): re-serialized JSON will differ in key order +and whitespace, so the test is that both manifests parse to the same ordered +entry set with the same detection/targeting/action semantics. ## 5. Payloads: SMB and/or HTTP (both supported) The user asked whether payloads can be SMB and/or HTTP. Yes - per entry: - `payloadsource = smb`: `payloadref` is the current relative path - (`apps/eDNC_6-4-5.msi`); the client resolves it against the mounted scope - root exactly as today. Zero client behavior change for these rows. This is - the default and the migration target for large binaries (MSIs are hundreds of - MB; SMB streaming beats HTTP for those). + (`apps/eDNC_6-4-5.msi`); the client still mounts W: and resolves it against + the scope root exactly as today. The engine is unchanged for these rows (the + mount + scope-root resolution still happen; an HTTP-only site skips the mount + because it has no `smb` rows). This is the default and the migration target + for large binaries (MSIs are hundreds of MB; SMB streaming beats HTTP). - `payloadsource = http`: `payloadref` is a URL (absolute, or relative to a configured payload base). The client downloads to a local temp dir, verifies the Hash/FileVersion detection value, then runs it. Good for small @@ -171,10 +200,22 @@ Manifest generation emits, per entry, whatever the client needs to fetch the bytes. The engine's existing "stage network EXE to local temp first" logic (SYSTEM access-denied workaround) generalizes cleanly to HTTP download. -Hashing stays the integrity mechanism regardless of transport: `DetectionValue` -carries the SHA256, the client verifies after fetch. This is the security- -critical bit - shopdb serves a manifest that says "install X with hash H"; a -tampered payload from any transport fails the hash and does not run. +Payload integrity uses the dedicated `payloadsha256` column, NOT `DetectionValue`. +This is the correction to a subtle trap: `DetectionValue` is a SHA256 only when +`DetectionMethod = Hash`. Most binaries detect by `Registry` or `FileVersion` +and carry no payload hash at all, so relying on `DetectionValue` would let an +HTTP/inline-fetched MSI run unverified. Instead, publishing an `http`/`inline` +payload computes and stores `payloadsha256`, and the client verifies the fetched +bytes against it BEFORE running, independent of how the entry detects install +state. `smb` payloads may set it too (defense in depth) but the share ACL is +their primary trust boundary. Detection stays a separate concern: it decides +whether to act; the payload hash decides whether the bytes are trustworthy. + +Transport security: the client fetches as SYSTEM, so the shopdb TLS cert must be +trusted machine-wide. Sites with a self-signed or air-gapped shopdb need the CA +in the machine trust store (provisioned by the same Azure DSC step that writes +the token). Plain HTTP is acceptable only inside a trusted segment, and even +then the `payloadsha256` check is what actually guarantees payload integrity. ## 6. API surface (`/api/geenforce/...`) @@ -184,19 +225,25 @@ Admin CRUD (gated by a new `geenforce.manage` permission via the plugin's - `GET/POST /scopes//entries`, `PUT/DELETE /entries/` - manifest entries - `PUT /scopes//entries/reorder` - the ordering contract, drag-to-reorder - `POST /entries//payload` - upload an inline/http payload (multipart), - compute + store its SHA256 into `detectionvalue` -- `GET /scopes//preview` - the exact JSON a client would receive (for review - before publish) + compute + store its `payloadsha256` (the integrity hash; NOT `detectionvalue`) +- `POST /scopes//publish` - freeze the current draft into a new immutable + `manifestpublishedversions` snapshot (this is what the fleet gets) +- `POST /scopes//rollback/` - mark an older snapshot current +- `GET /scopes//preview` - the draft JSON a client WOULD receive on next + publish (review before publish); `GET /scopes//published` shows the + currently-served snapshot Client-facing (gated by a collector-style service token, `geenforce.fetch` scope, reusing the PAT + `X-API-Key` machinery already built for the collector): - `GET /manifest?pctype=&subtype=&hostname=&machinenumber=` - Returns the assembled manifest JSON for that machine. The server can pre-apply - the PCTypes/hostname/machinenumber/cmmversion filters (thin client) OR return - the full scope and let the engine filter (fat client, matches today). Start - fat: return the scope manifest unchanged so the engine logic is untouched. + Returns the latest PUBLISHED snapshot for that scope (never the live draft). + The server can pre-apply the PCTypes/hostname/machinenumber/cmmversion filters + (thin client) OR return the full scope and let the engine filter (fat client, + matches today). Start fat: return the scope manifest unchanged so the engine + logic is untouched. Include the snapshot version + an ETag so the client can + cache and no-op when unchanged. - Payload fetch for `http`/`inline` rows: `GET /payload/` streaming the - bytes with the right content type. + bytes; the client verifies them against `payloadsha256` from the manifest. ## 7. Frontend: expand `/settings/pctypemapping` @@ -215,12 +262,17 @@ PC-type manager: relevant Detection* fields. Filter chips for PCTypes/hostnames/machine numbers. InUseCheck sub-editor. Payload source selector (smb/http/inline) with upload for the latter two. -- **Preview + publish**: show the generated JSON; "publish" bumps - `manifestversion` and writes a history snapshot (mirrors the current - `_meta/history/-.json` backups). -- **Desired vs observed**: because the collector already reports - `installedversions`, the scope page can show, per entry, how many fleet PCs - match the expected detection value. This is the payoff of unifying the two. +- **Draft, preview, publish**: editing changes only the draft; "publish" freezes + an immutable snapshot (see section 4) and is what the fleet then gets. Show the + draft-vs-published diff before publishing. Rollback republishes a prior + snapshot. +- **Desired vs observed**: the scope page can show, per entry, how many fleet + PCs match the expected detection value. CAVEAT: this is NOT free with today's + collector - it reports `installedsoftware[]`, not the per-entry manifest + status map (`installedVersions` keyed `/` that GE-Enforce already + computes for status.json). Delivering this feature needs a new collector + payload field carrying that map. Worth it (it is the payoff of unifying + desired + observed state) but it is a dependency, not existing data. This is an ADR-010 settings card contributed by the geenforce plugin, so it only appears when the plugin is enabled. @@ -253,11 +305,18 @@ The manifest is desired-state that runs as SYSTEM and installs software fleet- wide. A bad cutover = a fleet-wide mis-install. Stage it: 1. **Import + parity.** Write a one-shot importer that reads the current - on-share manifests (common + every `gea-shopfloor-*` + preinstall.json) into - the new tables. Then generate JSON back out and `diff` against the originals - until byte-identical for every scope. This proves the model is lossless - before anything depends on it. (This is the same discipline as the ADR-001 - data migration.) + on-share manifests (common + every `gea-shopfloor-*` + preinstall.json; + skip `.bak` / `.pre-mtconnect.bak` variants) into the new tables. Then + generate JSON back out and prove BEHAVIORAL equivalence for every scope - do + NOT chase byte-identity. Re-serialized JSON will differ in key order, + whitespace, and `_comment` formatting, so a raw `diff` would never converge. + The correct test: parse both the original and the regenerated manifest, + normalize, and assert the same ordered entry list with identical + detection/targeting/action fields per entry (ideally a small harness that + mimics the engine's filter+detect decisions and confirms the same entries + would fire in the same order on representative machine profiles). That, not + byte equality, is what proves the model is lossless. (Same discipline as the + ADR-001 data migration.) 2. **Shadow mode.** shopdb serves the manifest at a new endpoint; a canary PC fetches from shopdb but ALSO reads the share, and logs any diff. No install behavior changes. Run across one of each PC type for a few cycles. @@ -272,8 +331,11 @@ wide. A bad cutover = a fleet-wide mis-install. Stage it: in the shopdb UI and the on-share JSON is retired (or auto-exported as a backup for break-glass). -Rollback at every stage is a one-line dispatcher revert, because the engine and -payload layout never stop working from the share. +Rollback during cutover (stages 2-4) is a one-line dispatcher revert, because +the engine and payload layout never stop working from the share. AFTER the share +JSON is retired (stage 5), that escape hatch is gone - post-cutover rollback is +republishing a prior `manifestpublishedversions` snapshot (section 4). Both +mechanisms must exist before stage 5, not just the dispatcher revert. ## 10. Risks / open questions @@ -281,14 +343,22 @@ payload layout never stop working from the share. what `Install-FromManifest.ps1` expects is a fleet-wide install bug. The byte-identical round-trip test (step 1) is non-negotiable, and the plugin must pin which engine lib version it targets (>= 2.6 for `_CmmVersion`). -- **PCTypes alias graph** must be seeded and kept in sync with - `Install-FromManifest.ps1:463-475`. If the engine's alias map changes, ours - must too. Consider having the engine fetch the alias map from shopdb as well, - so there is one source. +- **PCTypes alias graph** must be kept in sync with + `Install-FromManifest.ps1:463-475`. The engine lib stays the single source of + truth; shopdb only MIRRORS the map for server-side validation. Do NOT invert + this to have the engine fetch aliases from shopdb - that would add exactly the + availability coupling the next bullet warns against. When the lib's alias map + changes, update shopdb's mirror as part of shipping that lib version. - **Availability coupling.** GE-Enforce currently depends only on SMB. Adding an HTTP dependency on shopdb means shopdb downtime could stall enforcement - hence the last-known-good local cache in section 8. Must be built in from day - one, not bolted on. + one, not bolted on. This is also why alias resolution and payloads stay + independent of a live shopdb wherever possible. +- **Transport trust.** The client runs as SYSTEM, so shopdb's TLS cert must be + in the machine trust store (self-signed/air-gapped sites need the CA + provisioned via the same DSC step as the token). `payloadsha256` verification + is the real integrity guarantee and holds even over plain HTTP inside a + trusted segment (section 5). - **Secrets in payloads.** Some config drops (site-config, credentials) may contain secrets. `inline` payloads live in the shopdb DB - those must respect the existing "secrets stay in .env, not the settings table" rule. Likely keep @@ -297,10 +367,13 @@ payload layout never stop working from the share. before enrollment). It may not have a shopdb token yet at that point in the imaging sequence. Preinstall may need to stay share-sourced longer than runtime, or fetch a bootstrap manifest anonymously over HTTP. -- **This is a big build.** Realistically phased: (P1) model + importer + parity - test; (P2) admin API + CRUD; (P3) frontend editor on /settings/pctypemapping; - (P4) client fetch + shadow mode; (P5) read cutover; (P6) payload migration. - P1 is the gating de-risk - if the round-trip is not lossless, stop. +- **This is a big build.** Realistically phased: (P1) model + importer + + behavioral-parity test; (P2) admin API + CRUD + publish/snapshot/rollback; + (P3) frontend editor on /settings/pctypemapping; (P4) client fetch + shadow + mode; (P5) read cutover; (P6) payload migration. P1 is the gating de-risk - if + behavioral parity does not hold, stop. Snapshots (P2) must land before any + client points at shopdb (P4), since serving the live draft to the fleet is + unacceptable. ## 11. Relationship to existing work @@ -325,7 +398,12 @@ payload layout never stop working from the share. Feasible and a strong architectural fit, but it is a multi-phase build with a fleet-wide blast radius. The single most important gate is P1: import the real -manifests and prove byte-identical round-trip. Do not build the UI or touch a -client until that parity test is green. If and when we proceed, this warrants a -new ADR (ADR-012: GE-Enforce manifest ownership) capturing the desired-state -model, the SMB/HTTP/inline payload contract, and the fail-safe cache. +manifests and prove BEHAVIORAL parity (same entries fire in the same order with +the same detection/targeting), not byte-identity. Do not build the UI or touch a +client until that parity holds. Three things separate a safe build from a +dangerous one and must not be cut: behavioral-parity import (P1), immutable +published snapshots with rollback before any client points at shopdb (P2/P4), +and a dedicated `payloadsha256` for every HTTP/inline payload (section 5). If and +when we proceed, this warrants a new ADR (ADR-012: GE-Enforce manifest +ownership) capturing the desired-state model, the published-snapshot contract, +the SMB/HTTP/inline payload + integrity model, and the fail-safe cache.