ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

notifications: correct timezone handling + configurable site timezone

Browse Source 
Notification start/end times displayed and stored wrong by the tz offset
(a 2:34 PM entry showed 6:34 PM). Two stacked bugs: to_dict emitted stored
UTC as naive ISO (no offset) so the browser read it as local, and the form
filled the datetime-local input from toISOString() (UTC).

Fix and generalize to a configurable site timezone (multi-site):
- New setting site_timezone (default America/New_York), public, editable in
  Settings > Site > Localization (common-zone dropdown).
- Backend tags datetimes UTC (_utc_iso); parse normalizes to naive UTC
  (_parse_utc); daily-reset expiry uses the site zone (_next_site_time);
  calendar allDay events key off the site-local day (_site_date).
- Shared frontend util datetime.js (Intl-based, DST-safe) converts between a
  UTC instant and a site-zone wall clock. Notification form, list, and
  calendar all render/enter in the site zone.
This commit is contained in:
cproudlock
2026-07-30 15:08:59 -04:00
parent 3ad26ba010
commit ea6fae91c3
9 changed files with 920 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files

697
docs/geenforce-api-cutover.md Normal file
View File

@@ -0,0 +1,697 @@
# GE-Enforce over HTTPS API: cutover reference
This is the operator/developer reference for moving a shopfloor PC type off the
SMB-share GE-Enforce delivery and onto the shopdb HTTPS API. The first cohort
cut over was the displays/kiosks (`gea-shopfloor-display`): share-less,
Intune/Entra-joined PCs with no SFLD share credentials. This doc captures
everything learned doing that, so extending the cutover to the other pc-types
(`gea-shopfloor-cmm`, `-collections`, `-keyence`, `-genspect`, `-heattreat`,
`-partmarker`, `-nocollections`, `common`) does not require re-learning it.
It pairs with the existing docs (which describe the pieces; this one describes
the CUTOVER):
- `docs/GE-ENFORCE.md` - concepts and the plugin
- `docs/GE-ENFORCE-CLIENT.md` - client fetch/report contract
- `docs/GE-ENFORCE-DEPLOY.md` - what must land on a PC
- `docs/GE-ENFORCE-DISPLAY.md` - the display scope specifics
- `/home/camp/projects/pxe/docs/ge-enforce-v2-architecture.md` - the SMB world
being cut away from
Contents:
1. [Overview and why](#1-overview-and-why)
2. [Server architecture](#2-server-architecture)
3. [Delivery models: smb vs http/inline payloads](#3-delivery-models-smb-vs-httpinline-payloads)
4. [Authoring a scope](#4-authoring-a-scope)
5. [The on-PC client and engine](#5-the-on-pc-client-and-engine)
6. [Bootstrap for share-less PCs](#6-bootstrap-for-share-less-pcs)
7. [Asset reporting via the collector](#7-asset-reporting-via-the-collector)
8. [HARD-WON GOTCHAS](#8-hard-won-gotchas)
9. [How this was verified](#9-how-this-was-verified)
10. [Deploy](#10-deploy)
11. [PLAYBOOK: extending to a new pc-type](#11-playbook-extending-to-a-new-pc-type)
12. [Open items / TODO](#12-open-items--todo)
---
## 1. Overview and why
GE-Enforce v2 delivers desired-state manifests and installer payloads from the
SFLD SMB share (`\\tsgwp00525.wjs.geaerospace.net\shared\dt\shopfloor\`).
Every PC mounts the share with Azure-DSC-provisioned SFLD credentials, reads
`<scope>\manifest.json`, and runs the engine
(`Install-FromManifest.ps1`). That works for the domain fleet but is a hard
dead end for share-less PCs.
The HTTPS API path replaces the share as the transport, keeping the engine and
its detection/self-heal behavior untouched:
| Concern | SMB share | HTTPS API |
|---------|-----------|-----------|
| Manifest source | `<scope>\manifest.json` on the share | `GET /api/geenforce/manifest?pctype=<scope>` (published snapshot) |
| Payload source | share paths (`apps\...`) | `GET /api/geenforce/payload/<sha256>` (content-addressed) |
| Auth | SFLD share credential (DSC) | `geenforce.fetch` service token OR source-IP allowlist |
| Result visibility | log files on the PC | `POST /api/geenforce/report` -> Enforcement Reports UI |
| Versioning | file overwrite + `_meta/history` | immutable published versions, rollback, ETag |
Which PCs MUST use the API: the share-less ones. Displays/kiosks are
Intune/Entra-joined with no SFLD credentials and no domain trust, so SMB is not
an option at all. The rest of the fleet CAN stay on the share (and currently
does); for them the API is an opt-in migration, not a forced one.
West Jefferson facts used throughout this doc:
| Fact | Value |
|------|-------|
| Prod host | `tsgwp00525.wjs.geaerospace.net` |
| App mount | `/shopdb` (IIS, app dir `C:\inetpub\wwwroot\shopdb`, pool `shopdbflask-prod`) |
| BaseUrl clients use | `https://tsgwp00525.wjs.geaerospace.net/shopdb` |
| Prod DB | `shopdb_flask` (MySQL) |
| Client allowlist CIDRs | `10.134.48.0/23,10.48.249.0/26` (the WJ corp/AESFMA shopfloor subnets) |
| Dev/staging instance | `/ops` mount, DB `shopdb_flask_dev`, pool `shopdbflask` |
---
## 2. Server architecture
All server code is in `plugins/geenforce/` (routes: `plugins/geenforce/api/routes.py`).
### Client-facing endpoints (three)
| Endpoint | Method | Auth | What it does |
|----------|--------|------|--------------|
| `/api/geenforce/manifest?pctype=<scope>&phase=runtime` | GET | `geenforce.fetch` token OR IP allowlist | Serves the CURRENT PUBLISHED manifest snapshot for a scope (never the draft). `ETag: "<scopeid>-v<version>"`, `X-Manifest-Version` header, 304 on `If-None-Match`. |
| `/api/geenforce/payload/<sha256>` | GET | `geenforce.fetch` token OR IP allowlist | Streams a payload blob by content hash: blob store first (`service.blob_path`), then inline `ManifestPayload`. ETag = the hash. Per-IP rate limited (120/min default) and size-capped (512 MB default, 413 above). |
| `/api/geenforce/report` | POST | `geenforce.report` token OR IP allowlist | Records one enforcement cycle via `service.record_enforcement_report`: hostname, scopename, appliedversion, enforcerversion, counts, per-entry results. Upserts the current report per (hostname, scopename, phase); older reports kept as history. |
Plus the collector for asset reporting (section 7): `POST
/api/collector/computers` in `shopdb/core/api/collector.py` - a DIFFERENT auth
domain (`collector.ingest`), NOT covered by the geenforce allowlist.
### Auth model
`_require_service_token(scope)` in `routes.py` is the decorator factory. Two
paths, fail-closed (neither -> 401):
1. A managed service token with the scope (`geenforce.fetch` for
manifest/payload, `geenforce.report` for report), sent as `X-API-Key` or a
Bearer PAT (`authorized_service_token`). A token may carry
`resourcescopelist` bindings: a bound token can only fetch its own scope's
manifest (403 otherwise) and only blobs those scopes' published manifests
reference (`service.blob_referenced_by_scopes`, 404 so hashes cannot be
probed). Displays get a token bound to `gea-shopfloor-display` (see
GE-ENFORCE-DISPLAY.md).
2. The IP allowlist: `_ip_allowlisted()` checks the caller against the setting
`geenforce_allowed_cidrs` (comma-separated CIDRs/IPs, empty = disabled).
Network trust replaces the shared secret for a vaulted fleet. The
allowlisted path has no resource-scope binding (unrestricted).
The token paths use RBAC permissions the plugin registers in
`GeEnforcePlugin.get_permissions()`: `geenforce.manage` (edit),
`geenforce.publish` (ship), `geenforce.fetch` and `geenforce.report`
(client service tokens). Admin CRUD/publish/simulate/compliance routes are JWT
plus `geenforce.manage`/`geenforce.publish`.
### Why the allowlist uses remote_addr (the IIS XFF dependency)
`_trusted_client_ip()` returns `request.remote_addr`, NOT the raw
`X-Forwarded-For` header. Proxies APPEND to X-Forwarded-For, so its first hop
is attacker-controlled: parsing it (as `_client_ip()` does, acceptably, for
rate limiting only) would let any caller send `X-Forwarded-For:
<allowlisted-ip>` and bypass the token entirely.
`remote_addr` is trustworthy only because of a two-piece chain that MUST stay
in place:
1. The IIS URL-Rewrite rule in the `/shopdb` web.config OVERWRITES (not
appends) the inbound `X-Forwarded-For` with `REMOTE_ADDR`, the real TCP
peer.
2. waitress runs with `--trusted-proxy=127.0.0.1
--trusted-proxy-headers=x-forwarded-for`, so it derives `remote_addr` from
that overwritten header only when the request comes from IIS on localhost.
A client that somehow hits waitress directly is not a trusted proxy, so its
`remote_addr` is its own real peer address. Either way, real client IP.
If the IIS rule is ever removed, the allowlist stops matching (fails closed to
401 for token-less clients) - it does NOT become spoofable, but the fleet
falls back to cached manifests. Keep the rule on.
---
## 3. Delivery models: smb vs http/inline payloads
Every `ManifestEntry` carries a `payloadsource` (`plugins/geenforce/serializer.py`
and `importer.py`):
| PayloadSource | Meaning | Manifest emission |
|---------------|---------|-------------------|
| `smb` (default) | Entry installs from the share exactly as v2 does; the entry's `Installer`/`Script`/`Source` is a share-relative path. | Nothing emitted - share manifests round-trip byte-identical, parity preserved. |
| `http` | Payload lives in the server's content-addressed blob store (`instance/geenforce/payloads/<sha256>`, registry row `ManifestBlob`). For big files (MSIs, EXEs). Upload via `flask geenforce add-payload <file>` or `service.store_blob`. | `PayloadSource`, `PayloadSha256`, `PayloadRef` keys on the entry. |
| `inline` | Payload bytes live IN the DB (`ManifestPayload`, <= 1 MB) - small scripts and configs. Attach via `service.store_inline_payload(entry, filename, contenttype, rawbytes)` or `POST /api/geenforce/entries/<id>/payload`. | Same three keys. |
Both `http` and `inline` are served from the same client URL:
`GET /api/geenforce/payload/<sha256>` (blob store checked first, then inline).
The sha256 IS the integrity contract - the client re-hashes after download.
### How the client stages payloads (Resolve-ShopdbPayloads)
`Resolve-ShopdbPayloads` in `plugins/geenforce/client/ShopdbEnforceClient.psm1`
is the bridge that lets the UNCHANGED engine install share-less:
1. For each entry with `PayloadSha256` and `PayloadSource` http/inline, call
`Get-ShopdbPayload`: download to
`C:\ProgramData\ShopDB\geenforce\payloads\<sha><ext>` (ext from
`PayloadRef`), verify the sha256, keep it as a content-addressed
last-known-good cache (a cache hit only counts if the bytes still hash
right).
2. Rewrite the entry's path field to the LEAF filename of the staged file
(`Split-Path -Leaf`) - NOT the absolute path. Field by Type:
`Installer` for MSI/EXE/CMD/BAT/INF, `Script` for PS1, `Source` for File.
3. Write a sibling `<scope>.resolved.json` manifest and return its path (or
the original path if nothing needed resolving). A payload that cannot be
fetched/verified THROWS - the runner's fail-safe catch decides what happens.
The runner (`Invoke-ShopdbEnforce.ps1`) then sets the engine's
`-InstallerRoot` to that same payloads directory, so the engine's
`Join-Path $InstallerRoot <leaf>` resolves to the staged file. `smb` entries in
a mixed manifest are left untouched and still resolve against the share (a PC
that has it).
---
## 4. Authoring a scope
Two authoring paths, both ending in `service.replace_scope_draft(scopename,
phase, manifest_dict)` (idempotent draft rebuild - published versions are never
touched by a re-import):
### A. import-share: adopt an existing SMB manifest
```
flask geenforce import-share --shareroot <path> [--scope <name>] [--preinstall <path>]
flask geenforce publish <scopename> [--phase runtime] [--notes "..."]
```
`importer.discover_share` walks the share root and ingests
`common/manifest.json`, `display/manifest.json`, and every
`gea-shopfloor-*/manifest.json` (skipping `.bak` variants). Entries come in as
`smb` payloads. Run `flask geenforce parity --shareroot <path>` first (Gate A):
proves import+re-export is behaviorally lossless before anything ships.
### B. authoring in code: seed_display_scope as the template
`plugins/geenforce/seed_display_scope.py` is the reference for a scope that
never existed on the share. Pattern:
- Build the manifest dict in Python (`build_display_manifest()`): four
`Type=Registry` drift-heal entries re-asserting the Edge kiosk relaunch
policies from imaging (`09-Setup-Display.ps1`), one inline PS1 dispatcher,
one inline PS1 always-on script. Registry heals use
`DetectionMethod=ValueMatches` against the same path/name they write, so
drift self-heals; the PS1s use `DetectionMethod=Always` and are idempotent.
- The dispatcher (`Invoke-DisplayKioskDispatch.ps1`, generated by
`build_dispatcher_script()`) reads `C:\Enrollment\display-type.txt`, maps
the subtype through the data-driven `DISPLAY_TYPE_TARGETS` table
(Dashboard -> `/shopfloor`, Lobby -> `/tv`, 3DPrintRoom -> `/parts-kiosk`),
and writes an all-users Startup shortcut (`ShopDB Kiosk.lnk`) launching Edge
`--kiosk` fullscreen at `{BaseUrl}{route}`. It does NOT Start-Process Edge
(see gotchas). Base URL comes from HKLM `BaseUrl`, falling back to the WJ
host.
- `seed_display_scope(publish=False)`: `replace_scope_draft`, flush (entries
need entryids), then `service.store_inline_payload(...)` for each script
entry (sets `payloadsource='inline'`, `payloadsha256`, `payloadref`),
optionally `service.publish_scope(...)`, commit. Draft rebuild is
idempotent; publish always creates a NEW version.
Run it on the server:
```
cd C:\inetpub\wwwroot\shopdb
$env:FLASK_APP = 'shopdb'
'from plugins.geenforce.seed_display_scope import seed_display_scope; print(seed_display_scope(publish=True))' | venv\Scripts\python -m flask shell
```
Expected: `{scopeid, entrycount: 6, entrytypes: [Registry x4, PS1, PS1],
dispatchersha256, alwaysonsha256, publishedversion: N}`.
### Publishing
`service.publish_scope` freezes the draft (rendered by
`serializer.scope_to_json`) into an immutable `ManifestPublishedVersion` and
flips `iscurrent`. Clients only ever see published versions.
`rollback_scope` re-currents an older version. Also available over the API
(`POST /scopes/<id>/publish`, permission `geenforce.publish`) and the
GE-Enforce UI.
### Attaching payloads
- Inline (<= 1 MB): `service.store_inline_payload` in code, or
`POST /api/geenforce/entries/<entryid>/payload` (multipart file).
- Blob (`http`): `flask geenforce add-payload <filepath>` prints the sha256;
set `PayloadSource=http` + `PayloadSha256` (+ `PayloadRef` for the
filename/extension) on the entry.
Publish AFTER attaching - the published JSON is what carries the
`PayloadSha256` the client fetches, and blob access for resource-bound tokens
is checked against the CURRENT published manifest.
---
## 5. The on-PC client and engine
### Config: HKLM:\SOFTWARE\GE\ShopDB
| Value | Used by | Notes |
|-------|---------|-------|
| `BaseUrl` | enforce client + kiosk dispatcher | e.g. `https://tsgwp00525.wjs.geaerospace.net/shopdb`. Required. |
| `ApiToken` | enforce client | `geenforce.fetch` (+ report) PAT. OPTIONAL - a token-less client relies on the IP allowlist (`Get-ShopdbConfig` treats BaseUrl-only as valid). |
| `CollectorKey` | `Report-AssetToShopDB.ps1` | `collector.ingest` PAT. REQUIRED for asset reporting (allowlist does not cover the collector). |
The key's ACL is restricted to SYSTEM + Administrators (the bootstrap does
this) so the kiosk auto-login user cannot read the PATs.
### The pieces on disk (kiosk layout, `C:\ProgramData\GE-Enforce`)
- `Invoke-ShopdbEnforce.ps1` - the runner
- `ShopdbEnforceClient.psm1` - the client module
- `lib\Install-FromManifest.ps1` - the engine (>= 2.6)
- `Report-AssetToShopDB.ps1` - the asset collector
- Cache: `C:\ProgramData\ShopDB\geenforce\` (`<scope>.json`, `.etag`,
`.version`, `payloads\`), logs `C:\Logs\Shopfloor\`
### Scheduled tasks (SYSTEM, RunLevel Highest)
| Task | Runs | Interval |
|------|------|----------|
| `ShopDB GE-Enforce` | `Invoke-ShopdbEnforce.ps1 -Scope <scope> -EnginePath <engine> -BaseUrl <url>` | AtStartup + every 15 min |
| `ShopDB Asset Report` | `Report-AssetToShopDB.ps1` | AtStartup + every 60 min |
Tokens are NOT in the task arguments (visible in task XML) - the scripts read
them from HKLM.
### The runner flow (Invoke-ShopdbEnforce.ps1)
1. `Get-ShopdbConfig` (params override registry). No BaseUrl -> exit 0, retry
next cycle.
2. `Sync-ShopdbManifest -Scope <scope>`: ETag-conditional GET; 200 validates
the JSON before overwriting the cache (a proxy error page served as 200
must not clobber last-known-good); 304 or any network failure -> cached
copy. Nothing at all -> Windows event log entry (source `ShopdbEnforce`,
id 1001) plus a best-effort failure report so it is visible server-side,
then exit 0.
3. `-ShadowMode` (with `-ShareManifestPath`): `Compare-ShopdbShadow` logs
name/order diffs, engine runs against the SHARE (zero behavior change).
This is the first step of every cutover.
4. Cutover mode: optional `-IncludeCommon` merges the fleet `common` scope
(`Merge-ShopdbManifests`: common's unique entries first, pctype wins on
Name conflict). OFF by default - a scope is enforced ALONE and displays are
self-sufficient. Then `Resolve-ShopdbPayloads` stages http/inline payloads
(section 3).
5. Engine call (the integration point):
`& $EnginePath -ManifestPath <resolved> -PCType $Scope -InstallerRoot <payloads dir> -LogFile <log>`.
6. `ConvertTo-ShopdbSummary` normalizes whatever came back (summary object,
array of emitted objects, bare int, $null) into
`@{Installed;Skipped;Failed;Filtered;Results;EnforcerVersion}`, then
`New-ShopdbReport` maps to the lowercase wire contract and
`Send-ShopdbReport` POSTs it. All best-effort; the whole script exits 0 no
matter what (fail-safe: a broken web app never breaks a PC).
### The engine contract (Install-FromManifest.ps1, lib 2.6)
Mandatory params: `-ManifestPath`, `-InstallerRoot`, `-LogFile`; optional
`-PCType`, `-PCSubType`. Entry Types: MSI, EXE, CMD/BAT, PS1, INF, File,
Registry. Detection: Registry, File, FileVersion, Hash, MarkerFile,
ValueMatches, pnputil, Always. Filters: PCTypes (with old/new-name alias
groups), TargetHostnames, TargetMachineNumbers, `_CmmVersion`. Exit 0/1/2
unchanged for the SMB path; NEW in the API cutover: the engine ends with
`Write-Output` of a summary pscustomobject
(`Installed/Skipped/Failed/Filtered/EnforcerVersion/Results`), which is the
only thing on the success stream (logs go via Write-Host), so
`& $EnginePath ...` captures it cleanly.
`SelfHealed` on a result means a REAL drift correction (a detected-missing
entry that got reinstalled). `Always`/no-detection entries install every cycle
by design and are not flagged, so the server-side status derivation
(`service.record_enforcement_report`: failed > selfhealed > ok) stays honest.
### The display-type.txt dispatcher pattern
For kiosks, per-subtype behavior does not fork the scope: ONE scope
(`gea-shopfloor-display`), one inline dispatcher entry that reads
`C:\Enrollment\display-type.txt` at enforce time and acts on it. Subtype
changes are a one-file edit plus the next 15-min cycle, and retargeting a
subtype is a table edit in `seed_display_scope.py` + republish. The dispatcher
writes an all-users Startup shortcut (never Start-Process - see gotchas), and
sweeps stale kiosk launchers first (its own `ShopDB Kiosk*.lnk`, the imaging
installers' `GE Aerospace Dashboard*`/`GE Aerospace Lobby*` shortcuts, any
browser .lnk with `-kiosk`-ish args or shopdb URLs, and .url files pointing at
the kiosk routes).
---
## 6. Bootstrap for share-less PCs
A share-less PC cannot pull its first files from the share, so the bootstrap
itself is downloadable from the web app.
`Install-ShopdbKiosk.ps1` (source of truth:
`/home/camp/pxe-images/shopdb-migration/kiosk-installer/`) is hosted at
`C:\inetpub\wwwroot\shopdb\installers\kiosk\` and downloadable at
`{BaseUrl}/installers/kiosk/Install-ShopdbKiosk.ps1`. Run elevated on the PC:
```
Set-ExecutionPolicy Bypass -Scope Process -Force
$u = 'https://tsgwp00525.wjs.geaerospace.net/shopdb/installers/kiosk/Install-ShopdbKiosk.ps1'
Invoke-RestMethod $u -OutFile "$env:TEMP\Install-ShopdbKiosk.ps1"
& "$env:TEMP\Install-ShopdbKiosk.ps1" -DisplayType Lobby -CollectorKey 'shopdb_pat_...'
# add -ShopdbToken 'shopdb_pat_...' only if the subnet is NOT allowlisted
```
What it does (idempotent; re-running is also the manual update path):
1. Writes `C:\Enrollment\display-type.txt` (the subtype) and
`C:\Enrollment\pc-type.txt` (the scope, default `gea-shopfloor-display`).
2. Writes HKLM `BaseUrl` [+ `ApiToken`] + `CollectorKey`, then locks the key
ACL to SYSTEM + Administrators.
3. Downloads runner + module + engine + collector from
`{BaseUrl}/installers/kiosk/` over HTTPS (TLS 1.2 forced).
4. Registers the two SYSTEM tasks (section 5).
5. Starts both once so the PC is live immediately.
IIS prerequisite: the `installers\kiosk` web.config MUST carry
`<staticContent>` MIME maps for `.ps1`/`.psm1` (`text/plain`) or IIS 404.3s
the downloads (see gotchas).
Delivery options for the bootstrap itself:
- Imaging-baked: the display image runs it (or lays down the same state) at
imaging time - see `project-display-self-contained`.
- Installer push: Intune/hand-run the one-liner above on an already-deployed
PC. This is how the pilot kiosks were done
(`shopdb-migration/run-on-kiosk-F.txt`).
---
## 7. Asset reporting via the collector
`Report-AssetToShopDB.ps1` (in the kiosk bundle; also deployed in the SMB
`common\` scope for the share fleet) POSTs the PC's identity to:
```
POST {BaseUrl}/api/collector/computers
X-API-Key: <collector.ingest PAT or COLLECTOR_API_KEY env key>
```
Auth (`shopdb/core/api/collector.py`, `_check_collector_auth`): a managed
token scoped `collector.ingest` (Bearer or X-API-Key) OR the
`COLLECTOR_API_KEY`/`COLLECTOR_API_KEY_COMPUTERS` env key. The GE-Enforce IP
allowlist does NOT apply here - the collector always needs a key, read from
HKLM `CollectorKey` (or the manifest entry's `Args -ApiKey`).
Schema (`plugins/computers/plugin.py`, `get_collector_schema`): identity field
`hostname` (required); optional `machinenumber`, `pctype`, `pcsubtype`,
`serialnumber`, `loggedinuser`, `lastboottime`, `lastcheckin`, `ipaddress`,
`vendorname`, `modelnumber`, `osname`, `installedsoftware`, `defaultprinter`,
`printers`. All lowercase concatenated (the project naming convention).
`apply_collector_payload` upserts idempotently by hostname (falls back to
`Asset.assetnumber`), creates the Asset+Computer when missing, maps
`machinenumber` -> `Asset.assetnumber` (imaging placeholder `9999` skipped both
client- and server-side), `pctype` -> ComputerType via the settings mapping,
and creates Vendor/Model/OS rows as needed. Fields not posted are not touched -
a partial read never blanks a good value, so the script only includes fields it
actually resolved.
Client details worth keeping: machine number resolution order is eDNC registry
`MachineNo` (WOW6432Node then native) -> `C:\Enrollment\cmm\cmmid.txt` ->
`C:\Enrollment\machine-number.txt`; the reported `ipaddress` is filtered to
the corp ranges (same two CIDRs as the allowlist) so a machine-LAN controller
NIC never lands in shopdb.
---
## 8. HARD-WON GOTCHAS
Read this section before touching ANY of the moving parts. Every bullet cost
real debugging time. Format: symptom -> cause -> fix.
- **PS crash "property 'X' cannot be found" under Set-StrictMode** ->
the module runs `Set-StrictMode -Version Latest`, and engine
results/summaries arrive as EITHER hashtables or PSCustomObjects with
varying key casing; direct `$obj.Key` access on an absent key throws ->
route every dynamic property read through `Get-ShopdbProperty` (handles
both shapes, case-insensitive, returns $null when absent). Never dot into
parsed JSON or engine output directly.
- **Register-ScheduledTask rejects the repeating trigger** -> passing
`-RepetitionDuration [TimeSpan]::MaxValue` serializes to an out-of-range
Duration the Task Scheduler XML schema rejects -> use
`-RepetitionInterval` ALONE; it defaults to indefinite repetition
(verified Win11 / PS 5.1). See `Register-SystemTask` in
`Install-ShopdbKiosk.ps1`.
- **Engine exits 2 / "InstallerRoot not found"** -> `-InstallerRoot` and
`-LogFile` are MANDATORY engine params and InstallerRoot must EXIST ->
the runner always passes both and pre-creates the payloads dir before the
engine call. Any new caller must do the same.
- **http payload "not found: C:\...\C:\..." (path doubling)** -> the engine
resolves entry paths as `Join-Path $InstallerRoot <field>`; writing the
staged payload's ABSOLUTE path into the entry made the engine double it ->
`Resolve-ShopdbPayloads` writes the LEAF filename only, and the runner sets
`-InstallerRoot` to the payloads cache dir. Keep those two in lockstep.
- **Manifest fetch "works" but parsing fails / cache garbage** -> if the
manifest is served with a non-JSON content type, PowerShell 5.1
`Invoke-WebRequest` `.Content` comes back as a `byte[]` instead of a string
-> the server route returns `mimetype='application/json'` (see
`get_manifest`); any mock server or proxy in the chain must do the same.
The client also validates JSON before overwriting last-known-good.
- **Bootstrap download 404 (HTTP 404.3)** -> IIS refuses to serve unknown
static extensions; `.ps1`/`.psm1` have no default MIME map -> add
`<staticContent><mimeMap fileExtension=".ps1" mimeType="text/plain" />`
(and `.psm1`) in the `installers\kiosk` web.config, with `<remove>` first
if inherited.
- **IP allowlist spoofable / mysteriously not matching** -> raw
`X-Forwarded-For` is attacker-controlled (proxies append; first hop is the
caller's to write) -> `_ip_allowlisted` uses `request.remote_addr` via
`_trusted_client_ip`, which is only correct because the IIS URL-Rewrite
rule OVERWRITES X-Forwarded-For with REMOTE_ADDR and waitress trusts only
127.0.0.1 as proxy. The rule is a hard dependency: never remove it, and
verify the spoof is closed after server changes
(`curl -H "X-Forwarded-For: 10.134.48.10"` from a non-allowlisted host
must get 401).
- **Kiosk browser never appears though the dispatcher "ran fine"** -> the
enforce task runs as SYSTEM in session 0, which has no interactive
desktop; `Start-Process msedge.exe` opens INVISIBLY there -> write an
all-users Startup shortcut
(`C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup\ShopDB Kiosk.lnk`)
and let the auto-login user launch it in a visible session. This is why
the dispatcher is shortcut-based.
- **TLS/transport errors on older images ("could not create SSL/TLS secure
channel")** -> Windows PowerShell 5.1 does not reliably negotiate TLS 1.2
by default -> every network helper calls `Set-ShopdbTls`
(`[Net.ServicePointManager]::SecurityProtocol = Tls12`) first; the
bootstrap and collector force it too. Any new script that touches the API
must do the same.
- **Secrets readable by the kiosk user / visible in Task Scheduler** ->
default HKLM\SOFTWARE ACL grants BUILTIN\Users read, and task arguments
are world-readable in the task XML -> restrict the
`HKLM:\SOFTWARE\GE\ShopDB` key ACL to SYSTEM + Administrators (the
bootstrap does), and NEVER put a token in a task's `-Argument` string -
scripts read `ApiToken`/`CollectorKey` from HKLM at run time.
- **Enforcement Reports show 0/0/0 with no per-entry detail** -> the engine
historically returned nothing on the success stream, so the runner had no
counts to report -> the 2.6 engine emits the summary object as its ONLY
`Write-Output` (all logging is Write-Host), and
`ConvertTo-ShopdbSummary` tolerates non-compliant engines by zero-filling.
If reports go 0/0/0 again, the engine on that PC is pre-summary - update
it.
- **Two kiosk browsers fighting / stale kiosk launch after retarget** -> old
installs left their own Startup launchers behind, in several flavors ->
the dispatcher's sweep must match ALL of: single- AND double-dash `-kiosk`
arguments (the regex `-kiosk` matches both), shortcuts whose args carry
shopdb URLs (`tsgwp00525`, `/shopdb/`, the dead `shopfloor-dashboard`
route), the imaging installers' `GE Aerospace Dashboard*` / `GE Aerospace
Lobby*` shortcut names, and `.url` files pointing at the kiosk routes.
Extend the sweep whenever a new launcher naming appears.
---
## 9. How this was verified
Two complementary verification passes; keep BOTH for future cohorts because
they catch different bug classes:
- **Code review** finds logic bugs: the XFF spoof hole, the StrictMode
absent-key crashes, the status-derivation trap (`installed>0` is not
self-heal), the report casing mismatch.
- **VM smoke test** finds integration/OS-version bugs: the
RepetitionDuration serialization rejection, the .ps1 MIME 404, session-0
invisibility, TLS negotiation, byte[] vs string response content - none of
which a read of the code surfaces.
The VM rig:
- The win11 virt-manager VM (see `project-win11-vm` /
`reference-vm-qga-as-system` memory), driven by
`/home/camp/pxe-images/ednc-bins/qga-run.py` - qemu guest agent
`guest-exec`, which runs PowerShell AS SYSTEM. That matters: the scheduled
tasks run as SYSTEM, so testing as SYSTEM reproduced the session-0 and
profile-less behaviors an interactive test would have masked.
- A mock HTTP server on the host exposing `/api/geenforce/manifest` (served
`application/json`), `/api/geenforce/payload/<sha>`, and capturing the
`/api/geenforce/report` POST body. This exercised the full client loop -
ETag/304, cache fallback (kill the server mid-test), payload hash
verification, resolved-manifest rewrite, engine run, summary -> report
mapping - without touching prod.
- Full pass = bootstrap installer run end to end in the VM, then assert: both
tasks registered, HKLM written + ACLed, manifest cached, payloads staged by
sha, Startup shortcut written, report captured with real counts.
---
## 10. Deploy
TWO independent channels. Confusing them is the classic mistake: the client/
engine/collector/bootstrap are pxe-images SHARE artifacts, NOT deployed by the
git pipeline.
### Channel 1: backend + frontend (the git .cmd pipeline)
Prod is air-gapped from dev; code moves via a git bundle on the share
(`\\172.16.9.9\pxe-images\github\shopdb-flask-pub.bundle`) and the .cmd
scripts in `/home/camp/pxe-images/github/`, run on the work PC:
1. `pull-shopdb-bundle.cmd` - fetch the bundle into the local clone
(ff-only).
2. `update-shopdb-github.cmd` - push the clone to GitHub.
3. `update-dev-server.cmd` - robocopy the clone to `X:` (dev `/ops` tree) +
the `/ops`-base frontend dist. Validate on `/ops` FIRST.
4. `update-prod-server.cmd` - robocopy to `Y:` (`C:\inetpub\wwwroot\shopdb`)
+ the `/shopdb`-base dist (`frontend-dist-subpath-shopdb`). Both scripts
carry instance guards (web.config MOUNT_PATH check) - do not bypass them.
Then RDP: `Restart-WebAppPool shopdbflask-prod`, and if migrations/deps
changed: `flask db upgrade`, `flask plugin upgrade-all`, `flask seed
permissions`, `flask seed settings`.
(The fast path used during the pilot - robocopy just the changed plugin files
from `shopdb-migration\prod-patch-geenforce\` + pool restart, per
`deploy-server-patch.txt` - works, but the same commits must ALSO go through
the bundle pipeline or prod drifts from git.)
### Channel 2: client + engine + collector + bootstrap (the kiosk bundle)
These live at `/home/camp/pxe-images/shopdb-migration/kiosk-installer/` and
deploy by robocopy from the work PC (Z: = share, Y: = prod app dir):
```
robocopy Z:\shopdb-migration\kiosk-installer Y:\installers\kiosk /E
```
That directory (bundle contents: `Install-ShopdbKiosk.ps1`,
`Invoke-ShopdbEnforce.ps1`, `ShopdbEnforceClient.psm1`,
`lib\Install-FromManifest.ps1`, `Report-AssetToShopDB.ps1`, `web.config` with
the MIME maps) IS the distribution point - PCs download from
`{BaseUrl}/installers/kiosk/`. Reference copies of the client kit also live in
the repo at `plugins/geenforce/client/` and the engine's source of truth is
`/home/camp/pxe-images/common/lib/Install-FromManifest.ps1`; when the engine
or client changes, update the kiosk bundle copy too (nothing syncs it
automatically). PCs pick up new bytes by re-running the bootstrap one-liner.
### Server prerequisites (once per site, all three or token-less clients 401)
1. Publish the scope(s) - `seed_display_scope(publish=True)` or
`flask geenforce publish <scope>`.
2. Seed `geenforce_allowed_cidrs` = `10.134.48.0/23,10.48.249.0/26`
(Settings rail > GE-Enforce Settings, or SQL upsert into `settings`).
3. Mint tokens (Settings > API Tokens, Restrict permissions ON):
`collector.ingest` (required, the kiosk `-CollectorKey`) and
`geenforce.fetch` (fallback for non-allowlisted subnets; resource-bind it
to the scope).
4. Keep the IIS URL-Rewrite XFF-overwrite rule enabled (section 2).
---
## 11. PLAYBOOK: extending to a new pc-type
Checklist for cutting any of the remaining scopes (`gea-shopfloor-cmm`,
`-collections`, `-keyence`, `-genspect`, `-heattreat`, `-partmarker`,
`-nocollections`, `common`) over to the API.
1. **Decide the delivery model.** Does this pc-type keep SMB access? If yes,
the cheap cutover is manifest-over-API + payloads-still-smb (entries stay
`smb`, nothing to upload, the engine resolves share paths as today). Only
a genuinely share-less PC needs http/inline payload conversion. Note the
payload endpoint's 512 MB default ceiling
(`GEENFORCE_PAYLOAD_MAX_BYTES`) before promising huge installers over
HTTPS.
2. **Get the scope into shopdb.** Existing share manifest:
`flask geenforce parity` then `flask geenforce import-share --scope
<name>`. New/reworked scope: author in code following
`seed_display_scope.py` (registry heals with ValueMatches detection,
idempotent Always PS1s, data-driven tables for anything per-subtype).
3. **Convert payloads (share-less only).** Small scripts/configs ->
`store_inline_payload` / the entry payload upload endpoint. Installers ->
`flask geenforce add-payload <file>`, set
`PayloadSource=http` + `PayloadSha256` + `PayloadRef` on the entry.
Remember: `PayloadRef`'s extension decides the staged filename's
extension.
4. **Publish.** New version every publish; clients converge within one
enforce cycle. Verify with
`curl "{BaseUrl}/api/geenforce/manifest?pctype=<scope>"` from an
allowlisted host.
5. **Auth for the PCs.** Subnet already inside
`10.134.48.0/23,10.48.249.0/26` -> token-less, nothing to do. New subnet
-> add its CIDR to `geenforce_allowed_cidrs` (Settings rail validates).
Not network-trustable -> mint a `geenforce.fetch` token resource-bound to
the scope and deliver it to HKLM `ApiToken`.
6. **Bootstrap the client.** Share-attached fleet: adapt the dispatcher /
`Install-GEEnforce.ps1` path (the pilot flow in
`shopdb-migration/kiosk-api-pilot.txt`: shadow first, then flip, then
DISABLE the old share enforce task so the two do not fight). Share-less:
the `Install-ShopdbKiosk.ps1` pattern - generalize `-Scope` and skip the
display-only pieces. Decide `-IncludeCommon`: displays run without it;
a non-display share-less PC that needs the fleet-wide common entries over
HTTPS turns it on AND requires common's entries to be payload-converted
first (an SMB-payload common entry will fail on a share-less PC).
7. **Run SHADOW mode first** on one pilot PC
(`-ShadowMode -ShareManifestPath <share manifest>`): fetch + compare +
report with zero behavior change. Watch the shadow diff log lines and the
Enforcement Reports row before flipping.
8. **Re-read section 8 (gotchas).** Especially: LEAF filenames, StrictMode
property access, SYSTEM/session-0, task trigger serialization, MIME maps
if you host new downloadables.
9. **Verify on the VM** (section 9) before the pilot PC: bootstrap +
enforce cycle against a mock or the dev `/ops` instance, as SYSTEM via
qga-run.py.
10. **Pilot one PC, then the cohort.** Keep the rollback in your pocket:
disable/remove the new task, re-enable the share task, remove HKLM
`BaseUrl` - the share path is untouched by all of this.
11. **pc-type mapping.** Make sure the collector's pctype mapping
(computers plugin settings, `pctypemap`) covers the scope name so asset
reports do not warn `no ComputerType mapping`.
---
## 12. Open items / TODO
- **Name resolution for reported users.** `loggedinuser` lands as a bare
username; resolving it to a display name depends on either the
`wjf_employees` `First_Name`/`Last_Name` data or shopdb User accounts
existing for shopfloor users. Not wired up; reports show raw usernames
until it is.
- **Old imaging-installer registry cleanup.** Displays imaged before the API
cutover carry leftover state from the old imaging-time kiosk installers
(superseded shortcuts are already swept by the dispatcher; stale registry
values are not yet cleaned). A cleanup entry in the display scope is the
natural vehicle.
- **The SMB fleet is still on the share.** Only displays/kiosks are on the
API. cmm/collections/keyence/genspect/heattreat/partmarker/nocollections/
common still enforce from SFLD; section 11 is the path. Shadow mode makes
each migration observable before it changes anything.
- **3DPrintRoom kiosk target is a placeholder.** `DISPLAY_TYPE_TARGETS`
points it at `/parts-kiosk`; confirm the real route with the floor team
before publishing to production 3D-print-room displays (flagged in
`seed_display_scope.py`).
- **Fast-path prod patches vs git.** The `prod-patch-geenforce` robocopy
path can leave prod ahead of the repo; reconcile by pushing the same
changes through the bundle pipeline (section 10, channel 1).