shopdb-flask/docs/proposals/ge-enforce-plugin.md

# Proposal: GE-Enforce as a shopdb plugin

Status: DRAFT / planning only. Not accepted, not built.
Author: planning session 2026-07-12.

## 1. What this is

Today GE-Enforce is a PowerShell manifest engine that reads per-PC-type
`manifest.json` files off an SMB share (`\\tsgwp00525.wjs.geaerospace.net\
shared\dt\shopfloor\`). Each logon, a scheduled task running as SYSTEM mounts
the share, reads the manifest for the machine's PC type, and installs or
self-heals apps, files, drivers, registry values, and scripts. A parallel
`preinstall.json` runs the same schema once at imaging.

This proposal turns the *manifest* into shopdb data: the authoritative manifest
lives in the shopdb database, is edited through the shopdb UI (an expansion of
`/settings/pctypemapping`), and is served to clients over HTTP as JSON. The
*payloads* (MSI/EXE/PS1/config bytes) stay on SMB, on HTTP, or both, referenced
by URL/path from the manifest rows. GE-Enforce.ps1 changes from "read a file on
W:" to "GET a manifest from shopdb, then fetch each payload from wherever the
row says."

The result: managing imaging PC types, their apps, scripts, files, registry
rules, and version gates becomes a first-class shopdb feature instead of hand-
edited JSON on a file share.

## 2. Why it fits shopdb

- shopdb already models the fleet (the collector ingests every PC's hostname,
  pctype, installed software, versions). Making shopdb *also* own what SHOULD be
  installed closes the loop: desired-state (manifest) and observed-state
  (collector) live in one system and can be diffed.
- `/settings/pctypemapping` already maps `gea-shopfloor-*` PC types to
  `ComputerType`. That page becomes the entry point for full imaging-PC-type
  management.
- The plugin contract (per-plugin models, migrations, API prefix, settings
  cards, collector hooks) is exactly the shape this needs.
- ADR-004 (per-site instances) matches: each site's shopdb owns each site's
  manifest. No multi-tenant complication.

## 3. Grounding: the real manifest schema

Source of truth for these field names (do not invent others):

- Schema: `pxe-images/tsgwp00525-v2/shared/dt/shopfloor/_meta/manifest-schema.json`
- Engine: `pxe-images/common/lib/Install-FromManifest.ps1`
- Dispatcher: `.../shopfloor/common/GE-Enforce.ps1`
- Architecture: `pxe/docs/ge-enforce-v2-architecture.md`

A manifest is `{ "Version": str, "_comment": str, "Applications": [entry, ...] }`.
Only `Name` and `Type` are required per entry.

### Per-entry fields (complete set)

Identity / action:
- `Name` (required, unique, also the status-key `<scope>/<Name>`)
- `Type` (required): one of `MSI EXE CMD BAT PS1 INF File Registry`
- `_comment` (documentation, heavily used in practice)

Type-specific payload references (sparse; depends on Type):
- MSI/EXE/CMD/BAT/INF: `Installer` (relative path) + `InstallArgs`
- PS1: `Script` (relative path, falls back to `Installer`) + `Args`
- File: `Source` (relative) + `Destination` (absolute on-PC path)
- Registry: `RegPath` + `RegName` + `RegValue` + `RegType`
  (`RegType` in `String DWord QWord MultiString ExpandString Binary`)
- Optional `LogFile`, `WaitTimeoutSec` (EXE hang kill), `InUseCheck`

Detection (decides whether the action fires / self-heals):
- `DetectionMethod`: one of
  `Registry File FileVersion Hash MarkerFile ValueMatches pnputil Always`
- `DetectionPath`, `DetectionName`, `DetectionValue`, `DetectionPattern`
- Note: `DetectionValue` is method-dependent - SHA256 for Hash, a 4-part
  version for FileVersion, a registry value for Registry, ignored for
  Always/File. Same column, different meaning per method.
- No `DetectionMethod` = always installs.

Targeting filters (all ANDed; each is multi-value):
- `PCTypes` (array; `"*"` = all; alias graph expands old<->new names)
- `PCSubTypes` / subtype via `<pctype>-<subtype>` values
- `TargetHostnames` (array; exact + `-like WJS-*` wildcards)
- `TargetMachineNumbers` (array; per-bay)
- `_CmmVersion` (scalar; per-entry PC-DMIS version gate, needs lib >= 2.6)

Nested:
- `InUseCheck`: `{ Behavior, Processes: [{Name, ExePath, GracefulCloseTimeoutSec}] }`
  Behavior in `Defer CloseAndReopen ForceClose ScheduleForReboot`

Parsed-but-inert today (model them, mark inert):
- `ApplyMode` (`Nightly Immediate ImmediateReboot`), `UpdateWindow` (`HH:MM-HH:MM`)

Preinstall-only extras (phase discriminator):
- `PreEnrollment`, `KillAfterDetection`, `PCTypesStrict`, `_pcTypesNote`

### Load-bearing behaviors the model must preserve

1. **Array order IS execution order.** Config-restore entries are deliberately
   placed AFTER their vendor installer so a mid-cycle overwrite heals the same
   cycle (eMxInfo.txt after eDNC; udc_webserver_settings after UDC). We MUST
   store an explicit per-scope `sortorder`, not a set.
2. **PCTypes alias graph** is many-to-many old<->new names resolved by set
   intersection, with a `PCTypesStrict` escape hatch. Not a simple FK.
3. **Polymorphic entry by Type** - sparse column set per type. Model as
   single-table with nullable columns, or a typed-payload child. STI is simpler.
4. **Two manifest phases** - runtime (self-heal, per logon) and preinstall
   (once at imaging) share the schema. One table with a `phase` discriminator.

## 4. Data model (new `geenforce` plugin)

Per-plugin Alembic chain (ADR-008). Tables (lowercase concatenated per naming
convention):

- `manifestscopes` - one row per imaging PC type / scope.
  - `scopeid` PK
  - `scopename` (e.g. `gea-shopfloor-cmm`)
  - `phase` enum (`runtime` | `preinstall`)
  - UNIQUE (`scopename`, `phase`), NOT `scopename` alone: `common` exists in
    runtime, and a scope name can appear in both phases. Note the phases are
    shaped differently - runtime is many per-pctype scopes (one manifest file
    each), preinstall is ONE flat manifest gated internally by `PCTypes`, so
    preinstall is modeled as a single `phase=preinstall` scope, not per-pctype
    scopes.
  - `computertypeid` FK -> `computertypes` (this REPLACES the thin
    `pctypemap_<pxetype>` setting; the mapping becomes a column here).
    Runtime-scope only; null for the preinstall scope.
  - `measuringtooltypeid` FK -> `measuringtooltypes`, nullable (metrology
    scopes: what device this scope implies; keeps imaging + collector agreed,
    see section 11).
  - `manifestversion` (string, mirrors manifest `Version`)
  - `description`, `isactive`
  - `iscommon` bool (the `common/` fleet-wide scope)

- `manifestentries` - one row per Applications[] entry (the working/draft copy).
  - `entryid` PK, `scopeid` FK
  - `sortorder` int (preserves array order; the ordering contract)
  - `name`, `entrytype` (MSI/EXE/.../Registry), `comment`
  - payload columns (nullable, per type): `installer`, `installargs`,
    `scriptpath`, `scriptargs`, `sourcepath`, `destination`,
    `regpath`, `regname`, `regvalue`, `regtype`
  - `payloadsource` enum (`smb` | `http` | `inline`) + `payloadref`
    (see section 5)
  - `payloadsha256` - integrity hash of the payload bytes, INDEPENDENT of the
    detection method. Mandatory for `http`/`inline` payloads; optional for
    `smb`. Do NOT reuse `detectionvalue` for this - `detectionvalue` is a
    SHA256 only when `detectionmethod = Hash`; an MSI with `Registry`/
    `FileVersion` detection has no payload hash, so an HTTP fetch would
    otherwise run unverified bytes (see section 5).
  - detection columns: `detectionmethod`, `detectionpath`, `detectionname`,
    `detectionvalue`, `detectionpattern`
  - gates: `cmmversion`, plus child tables for the multi-value filters
  - control: `logfile`, `waittimeoutsec`, `applymode`, `updatewindow`
  - preinstall flags: `preenrollment`, `killafterdetection`, `pctypesstrict`
  - `isactive`

- `manifestpublishedversions` + `manifestpublishedentries` - immutable
  published snapshots. Editing `manifestentries` never affects the fleet; a
  "publish" action freezes the current draft into a new numbered snapshot. The
  client is ALWAYS served the latest published snapshot for a scope, never the
  live draft, so a half-finished edit can never reach a PC. Rollback = mark an
  older snapshot current (this is the post-cutover safety net that replaces the
  "revert the dispatcher" rollback once the on-share JSON is retired). Mirrors
  the current `_meta/history/<date>-<scope>.json` backups, but authoritative.

- `manifestentrypctypes`, `manifestentryhostnames`, `manifestentrymachinenumbers`
  - child rows for the ANDed multi-value filters (one value per row, wildcards
    stored verbatim as patterns)

- `manifestinusechecks` + `manifestinusecheckprocesses`
  - the nested InUseCheck object and its Processes[] child list

- `pctypealiases` - a MIRROR of the old<->new name alias graph from
  `Install-FromManifest.ps1:463-475`, for server-side resolve/validate only.
  The engine lib stays the single source of truth (see section 10); shopdb
  never becomes the authority the client depends on for aliases.

The JSON the client receives is REBUILT from a published snapshot in exact
array order. Parity with the current engine is proven by BEHAVIORAL equivalence,
not byte-identity (see section 9): re-serialized JSON will differ in key order
and whitespace, so the test is that both manifests parse to the same ordered
entry set with the same detection/targeting/action semantics.

## 5. Payloads: SMB and/or HTTP (both supported)

The user asked whether payloads can be SMB and/or HTTP. Yes - per entry:

- `payloadsource = smb`: `payloadref` is the current relative path
  (`apps/eDNC_6-4-5.msi`); the client still mounts W: and resolves it against
  the scope root exactly as today. The engine is unchanged for these rows (the
  mount + scope-root resolution still happen; an HTTP-only site skips the mount
  because it has no `smb` rows). This is the default and the migration target
  for large binaries (MSIs are hundreds of MB; SMB streaming beats HTTP).
- `payloadsource = http`: `payloadref` is a URL (absolute, or relative to a
  configured payload base). The client downloads to a local temp dir, verifies
  the Hash/FileVersion detection value, then runs it. Good for small
  config/script payloads and for sites with no SMB share.
- `payloadsource = inline`: for small text payloads (a `.ps1`, a config file, a
  registry value), the bytes live in shopdb itself and are served in-band. No
  external store at all. Best for scripts and File-type config drops.

Manifest generation emits, per entry, whatever the client needs to fetch the
bytes. The engine's existing "stage network EXE to local temp first" logic
(SYSTEM access-denied workaround) generalizes cleanly to HTTP download.

Payload integrity uses the dedicated `payloadsha256` column, NOT `DetectionValue`.
This is the correction to a subtle trap: `DetectionValue` is a SHA256 only when
`DetectionMethod = Hash`. Most binaries detect by `Registry` or `FileVersion`
and carry no payload hash at all, so relying on `DetectionValue` would let an
HTTP/inline-fetched MSI run unverified. Instead, publishing an `http`/`inline`
payload computes and stores `payloadsha256`, and the client verifies the fetched
bytes against it BEFORE running, independent of how the entry detects install
state. `smb` payloads may set it too (defense in depth) but the share ACL is
their primary trust boundary. Detection stays a separate concern: it decides
whether to act; the payload hash decides whether the bytes are trustworthy.

Transport security: the client fetches as SYSTEM, so the shopdb TLS cert must be
trusted machine-wide. Sites with a self-signed or air-gapped shopdb need the CA
in the machine trust store (provisioned by the same Azure DSC step that writes
the token). Plain HTTP is acceptable only inside a trusted segment, and even
then the `payloadsha256` check is what actually guarantees payload integrity.

## 6. API surface (`/api/geenforce/...`)

Admin CRUD (gated by a new `geenforce.manage` permission via the plugin's
`get_permissions()` hook):
- `GET/POST /scopes`, `GET/PUT/DELETE /scopes/<id>` - imaging PC types
- `GET/POST /scopes/<id>/entries`, `PUT/DELETE /entries/<id>` - manifest entries
- `PUT /scopes/<id>/entries/reorder` - the ordering contract, drag-to-reorder
- `POST /entries/<id>/payload` - upload an inline/http payload (multipart),
  compute + store its `payloadsha256` (the integrity hash; NOT `detectionvalue`)
- `POST /scopes/<id>/publish` - freeze the current draft into a new immutable
  `manifestpublishedversions` snapshot (this is what the fleet gets)
- `POST /scopes/<id>/rollback/<version>` - mark an older snapshot current
- `GET /scopes/<id>/preview` - the draft JSON a client WOULD receive on next
  publish (review before publish); `GET /scopes/<id>/published` shows the
  currently-served snapshot

Client-facing (gated by a collector-style service token, `geenforce.fetch`
scope, reusing the PAT + `X-API-Key` machinery already built for the collector):
- `GET /manifest?pctype=<scope>&subtype=<s>&hostname=<h>&machinenumber=<n>`
  Returns the latest PUBLISHED snapshot for that scope (never the live draft).
  The server can pre-apply the PCTypes/hostname/machinenumber/cmmversion filters
  (thin client) OR return the full scope and let the engine filter (fat client,
  matches today). Start fat: return the scope manifest unchanged so the engine
  logic is untouched. Include the snapshot version + an ETag so the client can
  cache and no-op when unchanged.
- Payload fetch for `http`/`inline` rows: `GET /payload/<entryid>` streaming the
  bytes; the client verifies them against `payloadsha256` from the manifest.

## 7. Frontend: expand `/settings/pctypemapping`

The current page (`PCTypeMappingSettings.vue`, "Collector PC Types") is a read-
only-ish table of `pxetype -> ComputerType` dropdowns. It grows into the imaging-
PC-type manager:

- **Scopes list**: add/rename/delete imaging PC types; each still carries its
  `ComputerType` mapping (that column moves from a setting into `manifestscopes`).
  A `phase` toggle (runtime vs preinstall). Common scope flagged.
- **Scope detail / manifest editor**: an ordered, drag-reorderable list of
  entries (the ordering contract made visible). Each entry is a typed form -
  the visible fields switch on `entrytype` (MSI shows Installer+InstallArgs;
  PS1 shows Script+Args; File shows Source+Destination; Registry shows the Reg*
  quartet). Detection block with a method dropdown that reveals only the
  relevant Detection* fields. Filter chips for PCTypes/hostnames/machine numbers.
  InUseCheck sub-editor. Payload source selector (smb/http/inline) with upload
  for the latter two.
- **Draft, preview, publish**: editing changes only the draft; "publish" freezes
  an immutable snapshot (see section 4) and is what the fleet then gets. Show the
  draft-vs-published diff before publishing. Rollback republishes a prior
  snapshot.
- **Desired vs observed**: the scope page can show, per entry, how many fleet
  PCs match the expected detection value. CAVEAT: this is NOT free with today's
  collector - it reports `installedsoftware[]`, not the per-entry manifest
  status map (`installedVersions` keyed `<scope>/<Name>` that GE-Enforce already
  computes for status.json). Delivering this feature needs a new collector
  payload field carrying that map. Worth it (it is the payoff of unifying
  desired + observed state) but it is a dependency, not existing data.

This is an ADR-010 settings card contributed by the geenforce plugin, so it only
appears when the plugin is enabled.

## 8. Client change (minimal, staged)

`GE-Enforce.ps1` today: mount W:, read `<scope>\manifest.json`, hand to
`Install-FromManifest`. New path: GET the manifest from shopdb, write it to the
same local location the engine reads, then run the engine unchanged. That is the
smallest possible client delta - the engine, detection logic, self-heal, and
SMB payload resolution all stay identical. Only the *source of the JSON* moves
from file to HTTP.

Payloads: `smb` rows need no client change. `http`/`inline` rows need a small
fetch-and-verify helper (download to temp, check SHA256, then the existing
installer action runs against the local copy). The engine already stages network
EXEs to temp, so this is an extension, not a rewrite.

Auth: the client already has SFLD credentials in
`HKLM:\SOFTWARE\GE\SFLD\Credentials`. Add a shopdb service token (a
`geenforce.fetch` PAT) provisioned the same way (Azure DSC writes it to
registry), sent as `X-API-Key`. If shopdb is unreachable, the client falls back
to the last-known-good manifest cached locally (fail-safe: never leave a PC
unmanaged because the web app is down). This mirrors today's "creds missing =
exit 0, retry next cycle" resilience.

## 9. Cutover strategy

The manifest is desired-state that runs as SYSTEM and installs software fleet-
wide. A bad cutover = a fleet-wide mis-install. Stage it:

1. **Import + parity.** Write a one-shot importer that reads the current
   on-share manifests (common + every `gea-shopfloor-*` + preinstall.json;
   skip `.bak` / `.pre-mtconnect.bak` variants) into the new tables. Then
   generate JSON back out and prove BEHAVIORAL equivalence for every scope - do
   NOT chase byte-identity. Re-serialized JSON will differ in key order,
   whitespace, and `_comment` formatting, so a raw `diff` would never converge.
   The correct test: parse both the original and the regenerated manifest,
   normalize, and assert the same ordered entry list with identical
   detection/targeting/action fields per entry (ideally a small harness that
   mimics the engine's filter+detect decisions and confirms the same entries
   would fire in the same order on representative machine profiles). That, not
   byte equality, is what proves the model is lossless. (Same discipline as the
   ADR-001 data migration.)
2. **Shadow mode.** shopdb serves the manifest at a new endpoint; a canary PC
   fetches from shopdb but ALSO reads the share, and logs any diff. No install
   behavior changes. Run across one of each PC type for a few cycles.
3. **Read cutover, payloads still SMB.** Flip GE-Enforce to source the JSON from
   shopdb (payloads stay `smb`). The blast radius is only "where the JSON comes
   from"; the bytes and engine are unchanged. Keep the share manifests as the
   rollback (revert the dispatcher one-liner).
4. **Payload migration (optional, per entry).** Move small scripts/configs to
   `inline`/`http` opportunistically. Leave big MSIs on SMB indefinitely - SMB
   is the right transport for them.
5. **Author in shopdb.** Once read-cutover is stable, new manifest edits happen
   in the shopdb UI and the on-share JSON is retired (or auto-exported as a
   backup for break-glass).

Rollback during cutover (stages 2-4) is a one-line dispatcher revert, because
the engine and payload layout never stop working from the share. AFTER the share
JSON is retired (stage 5), that escape hatch is gone - post-cutover rollback is
republishing a prior `manifestpublishedversions` snapshot (section 4). Both
mechanisms must exist before stage 5, not just the dispatcher revert.

## 10. Risks / open questions

- **The engine is the contract.** Any drift between shopdb's generated JSON and
  what `Install-FromManifest.ps1` expects is a fleet-wide install bug. The
  byte-identical round-trip test (step 1) is non-negotiable, and the plugin must
  pin which engine lib version it targets (>= 2.6 for `_CmmVersion`).
- **PCTypes alias graph** must be kept in sync with
  `Install-FromManifest.ps1:463-475`. The engine lib stays the single source of
  truth; shopdb only MIRRORS the map for server-side validation. Do NOT invert
  this to have the engine fetch aliases from shopdb - that would add exactly the
  availability coupling the next bullet warns against. When the lib's alias map
  changes, update shopdb's mirror as part of shipping that lib version.
- **Availability coupling.** GE-Enforce currently depends only on SMB. Adding an
  HTTP dependency on shopdb means shopdb downtime could stall enforcement -
  hence the last-known-good local cache in section 8. Must be built in from day
  one, not bolted on. This is also why alias resolution and payloads stay
  independent of a live shopdb wherever possible.
- **Transport trust.** The client runs as SYSTEM, so shopdb's TLS cert must be
  in the machine trust store (self-signed/air-gapped sites need the CA
  provisioned via the same DSC step as the token). `payloadsha256` verification
  is the real integrity guarantee and holds even over plain HTTP inside a
  trusted segment (section 5).
- **Secrets in payloads.** Some config drops (site-config, credentials) may
  contain secrets. `inline` payloads live in the shopdb DB - those must respect
  the existing "secrets stay in .env, not the settings table" rule. Likely keep
  any secret-bearing payload on SMB with ACLs, never inline.
- **Preinstall runner** is a separate consumer (`00-PreInstall-*` at imaging,
  before enrollment). It may not have a shopdb token yet at that point in the
  imaging sequence. Preinstall may need to stay share-sourced longer than
  runtime, or fetch a bootstrap manifest anonymously over HTTP.
- **This is a big build.** Realistically phased: (P1) model + importer +
  behavioral-parity test; (P2) admin API + CRUD + publish/snapshot/rollback;
  (P3) frontend editor on /settings/pctypemapping; (P4) client fetch + shadow
  mode; (P5) read cutover; (P6) payload migration. P1 is the gating de-risk - if
  behavioral parity does not hold, stop. Snapshots (P2) must land before any
  client points at shopdb (P4), since serving the live draft to the fleet is
  unacceptable.

## 11. Relationship to existing work

- Replaces `plugins/computers/pctypemap.py` (the thin `pctypemap_<pxetype>`
  settings) - the pctype -> ComputerType mapping becomes the `computertypeid`
  column on `manifestscopes`. Migrate those settings in, then retire them.
- Also folds in the metrology mapping now living in `pctypemap.py`
  (`METROLOGY_TOOL_MAP`). The collector already auto-creates a MeasuringTool
  asset and a directional PC->tool `controls` relationship when it sees a
  metrology pctype (CMM / Keyence / Genspect / wax-and-trace); the PC stays a
  shopfloor PC. A metrology scope in the manifest model should carry the
  attached-measuring-tool type alongside its ComputerType so imaging and
  collector agree on what device the scope implies.
- Reuses the collector's token machinery (PAT + `X-API-Key` + scopes) for the
  client-facing endpoints.
- Reuses `get_permissions()` (contract 0.10.0) for `geenforce.manage` /
  `geenforce.fetch`.
- Pairs with the collector: desired-state (this plugin) + observed-state
  (collector) enable a fleet compliance view.

## 12. Recommendation

Feasible and a strong architectural fit, but it is a multi-phase build with a
fleet-wide blast radius. The single most important gate is P1: import the real
manifests and prove BEHAVIORAL parity (same entries fire in the same order with
the same detection/targeting), not byte-identity. Do not build the UI or touch a
client until that parity holds. Three things separate a safe build from a
dangerous one and must not be cut: behavioral-parity import (P1), immutable
published snapshots with rollback before any client points at shopdb (P2/P4),
and a dedicated `payloadsha256` for every HTTP/inline payload (section 5). If and
when we proceed, this warrants a new ADR (ADR-012: GE-Enforce manifest
ownership) capturing the desired-state model, the published-snapshot contract,
the SMB/HTTP/inline payload + integrity model, and the fail-safe cache.