Files
shopdb-flask/docs/COLLECTOR-INTEGRATION.md
cproudlock 4995456136 docs: take one site's name, hosts and paths off the public wiki
The publishability gate caught internal tooling names and developer paths but
nothing site-specific, so roughly sixty leaks reached the wiki: the site name in
ten documents, real fleet hostnames in the collector and GE-Enforce examples, an
internal database name through the whole import guide, imaging-share paths, and
a maintainer's username as the Deciders line of every ADR and inside a generated
curl example.

None of it is a security matter on an air-gapped fleet. It matters because these
pages are read by engineers at other plants, and a document that names one site
throughout reads as that site's notes rather than a product's documentation -
which is exactly what it then gets treated as.

Examples now use neutral hostnames, the site is "the reference site" where the
distinction carries meaning, and ADRs are decided by "ShopDB maintainers". The
gate carries all of these patterns, so the next one fails a build.

Two documents leave docs/ because they were never written for an outside reader.
PROJECT-REVIEW.md is an internal health memo pinned to a commit from July, whose
headline finding (an untracked playbook) has since been fixed - it is history,
and git holds it. PILOT-DEPLOY.md is one site's own cutover runbook, complete
with a "re-measure before publishing" placeholder; it moves next to the loader
it belongs to, in scripts/site_imports/wjf/.

ADR-015 is AMENDED rather than rewritten. Its enforcement section still said
report-only and its backlog still listed hardcodes that are now cleared, which
left the record contradicting itself. The amendment says what changed and why
the report-only period ended; the original text stays, because what the decision
looked like when it was taken is the part worth keeping.

Also corrects llms.txt's response envelope, which had errors at the top level
and pagination at meta.total. Both are nested one deeper, so anything written
against that description read undefined on every error it tried to handle.
2026-08-14 15:38:27 -04:00

711 lines
36 KiB
Markdown

# Collector integration (PC auto-update)
How the shopfloor PC fleet pushes inventory into shopdb-flask, replacing the
classic ASP `api.asp?action=updateCompleteAsset` path. The generic collector
contract is defined in ADR-006; this doc is the operational reference for wiring
a real caller (the GE-Enforce fleet agent) to it.
- Server code: `shopdb/core/api/collector.py`
- Computers schema + upsert: `plugins/computers/plugin.py`
(`get_collector_schema` / `apply_collector_payload`)
- Contract rationale: `docs/adr/ADR-006-collector-contract.md`
---
## 1. How it works now
### Auth model (header-only, env keys, fail-closed)
- The API key travels in the `X-API-Key` request header. Nothing else is
accepted. The old `?api_key=<key>` querystring fallback has been removed on
every collector endpoint (see the breaking-change section below).
- Keys come from environment variables (never the database, never a file the app
serves):
- `COLLECTOR_API_KEY_<PLUGINNAME>` - per-plugin override, uppercased plugin
name (e.g. `COLLECTOR_API_KEY_COMPUTERS`).
- `COLLECTOR_API_KEY` - shared fallback used when no per-plugin key is set.
- Resolution order per request: per-plugin key first, then the shared key
(`_plugin_api_key` in `collector.py`).
- Fail-closed: if neither variable is set server-side, the endpoint returns
HTTP 500 `Collector API key not configured` and rejects every request. An
unconfigured server never silently accepts unauthenticated data.
- A caller that sends the wrong key (or no key) gets HTTP 401 `Invalid API key`.
- In addition to the env keys, a managed API token scoped to `collector.ingest`
is accepted as a collector credential on every collector endpoint. See
"Managed collector tokens" below; env keys remain the fallback.
### Managed collector tokens (recommended)
Alongside the env keys, every collector endpoint (`/api/collector/<plugin>`,
`/pc`, `/apps`, `/heartbeat`, `/bulk`, `/status`) also accepts a **managed API
token** (PAT) scoped to the `collector.ingest` permission. The env keys stay
supported as a bootstrap/legacy fallback - nothing breaks - but a managed token
is the preferred credential because it can be minted, rotated, and revoked from
the UI (Settings > API Tokens) and its use shows up in `lastusedat` and the
audit log.
What makes a token a collector service token: it is scoped to ONLY
`collector.ingest`. That scope authorizes the collector ingest API and NOTHING
else. The existing scoped-token machinery contains it automatically - a scoped
token passes `require_permission` only for its listed permissions and is denied
on every role-gated (`require_role`) endpoint and on import mode, and
`collector.ingest` gates no normal route. So a collector token that leaks cannot
be used to read or write anything through the regular API; it can only submit
collector payloads.
Both wire transports are accepted (send whichever is convenient; GE-Enforce
sends `X-API-Key` today, so that stays ergonomic):
```
POST /api/collector/computers
X-API-Key: shopdb_pat_<40 hex>
```
or
```
POST /api/collector/computers
Authorization: Bearer shopdb_pat_<40 hex>
```
An unscoped PAT, or a PAT scoped to some other permission, is NOT a collector
token and is rejected (401) - only `collector.ingest` in the scope list counts.
A revoked or expired token is rejected (401) on both transports.
#### How to mint one (admin flow)
The simplest contained flow: an **admin** mints the token, scoped to
`collector.ingest`. Because the token is scoped, the admin-role bypass is
suspended for it, so the token is contained to the collector API even though its
owner is an admin - it cannot act with admin authority anywhere.
1. Settings > API Tokens > New Token.
2. Check **Restrict permissions**, then in the permissions grid tick only
**Submit collector payloads (fleet reporting)** (the `collector.ingest`
permission under the Collector category). Name it (e.g. `fleet-collector`),
optionally set an expiry, Create.
3. Copy the `shopdb_pat_...` secret (shown once) and deploy it to the fleet the
same way as the env key: the `collectorApiKey` field in per-site
`site-config.json` (see "Delivering the API key to clients" below). The
client sends it in `X-API-Key` exactly as it sends an env key today - no
client code change.
Service identity (documented, not built): if you prefer a non-admin owner,
create a dedicated low-privilege user (e.g. `svc-collector`) whose role holds
only `collector.ingest`, plus `apitokens.create` if that user is to mint its own
token. The scope ceiling then caps any token it mints at `collector.ingest`.
The admin-minted route above is simpler and equally contained, so it is the
recommended default.
#### Rotation
Managed tokens rotate without a fleet re-image:
1. Mint a new collector token (steps above).
2. Deploy it via `site-config.json` (`collectorApiKey`) - update the one per-site
value.
3. Confirm the new token is in use: watch its `lastusedat` climb in Settings >
API Tokens (and the old token's `lastusedat` go stale).
4. Revoke the old token once traffic has moved. Revocation is immediate.
### Generic endpoint contract: `POST /api/collector/<plugin>`
One dynamic route serves every enabled plugin that returns a collector schema.
For the PC fleet that is `POST /api/collector/computers`.
Request flow inside `generic_collect`:
1. Look up the plugin's schema. Unknown / disabled plugin -> HTTP 404
`No collector registered for plugin <plugin>`.
2. Resolve and check the API key (per-plugin then shared, as above).
3. Parse the JSON body. Missing or non-object body -> HTTP 400 `No data provided`.
4. Identity-field resolution: the schema names an `identityfield`
(`hostname` for computers). If that field is missing or blank in the payload,
HTTP 400 `<identityfield> is required`.
5. Idempotent upsert: the plugin's `apply_collector_payload` finds the existing
asset by identity and updates it, or inserts a new one. Same identity on a
later submission updates the same row - re-imaging a PC does not duplicate it,
and existing asset relationships are preserved.
6. Audit logging: every accepted submission writes an `AuditLog` row
(`action` = created/updated, entity `Collector`, `details = collector:<plugin>
action=<action>`), then commits.
Response body (HTTP 200), wrapped in the standard envelope
`{status, data, message, meta}`, with the collector result under `data`:
```json
{
"status": "success",
"data": {
"status": "ok",
"action": "created",
"assetid": 12345,
"identityvalue": "SHOPPC2335",
"warnings": ["unknown operating system: Microsoft Windows 11 Enterprise 23H2 (build 22631)"]
},
"message": "computers collector created",
"meta": { "timestamp": "...", "requestid": "..." }
}
```
`action` is `created`, `updated`, or `noop`. `warnings` is a list of soft
problems (unmapped pc-type, unknown OS, unknown app, un-stored pcsubtype) that
did NOT fail the request - the row was still written.
### Error responses
Errors use the same envelope with `status: "error"` and the detail under
`data.error`:
```json
{ "status": "error", "data": { "error": { "code": "UNAUTHORIZED", "message": "Invalid API key" } }, "meta": { } }
```
The plugin can raise `ValueError` for controlled validation failures; its message
is returned verbatim (HTTP 400). Any other exception is caught, logged
server-side, the transaction is rolled back, and the caller gets a generic
HTTP 500 `Internal error processing collector payload` with no internal detail
(by design - check the server log to see what actually failed).
### Schema discovery: `GET /api/collector/_schemas`
Returns the collector schema for every enabled plugin. This route is
JWT-protected (an interactive/admin token), NOT API-key protected - it is for
humans and tooling introspecting what payloads are accepted, not for the
headless collectors themselves.
### Legacy endpoints (computers-only, predate ADR-006)
These still exist for the older PowerShell callers and all require the same
`X-API-Key` header:
| Endpoint | Purpose |
|---|---|
| `POST /api/collector/pc` | Update one PC matched by `hostname` (lastboot, user, serial). |
| `POST /api/collector/apps` | Update installed apps for a PC (known apps only). |
| `POST /api/collector/heartbeat` | Record check-in for one or many hostnames. |
| `POST /api/collector/bulk` | Update many PCs in one call. |
| `GET /api/collector/status` | Liveness + endpoint list. |
New integrations should target `POST /api/collector/computers`, not these. Per
ADR-006 the legacy `/pc` path is deprecated and slated for removal before v1.0.
### Computers plugin field mapping
Payload naming follows the project convention (lowercase concatenated). The
identity field is `hostname`. All other fields are optional; an omitted field
leaves the existing value untouched (patch-style), so a bare report never blanks
a column.
| Payload field | Type | Server behaviour (`apply_collector_payload`) |
|---|---|---|
| `hostname` (required) | string | Identity. Matches `Computer.hostname` (case-insensitive), then falls back to `Asset.assetnumber`. New asset created if no match. |
| `machinenumber` | string | Identifies the MACHINE this PC drives, never the PC. A new PC always takes `assetnumber = hostname`, and an existing PC's `assetnumber` is left alone. The number resolves a machine asset and builds a PC -> machine `controls` relationship; an unknown number warns rather than creating a machine. The placeholder `9999` and empty string link nothing. See "Machine links" below. |
| `pctype` | string | `gea-shopfloor-*` imaging type -> `Computer.computertypeid` via the configurable `pctypemap` settings. Unmapped value -> warning, not error. |
| `pcsubtype` | string | Accepted but not stored yet -> warning. |
| `serialnumber` | string | `Asset.serialnumber`. |
| `loggedinuser` | string | `Computer.loggedinuser`. (`currentuser` is also accepted as a legacy alias.) |
| `lastboottime` | ISO-8601 datetime | `Computer.lastboottime`. Unparseable -> warning. |
| `lastcheckin` | ISO-8601 datetime | Accepted (heartbeat semantics). `lastreporteddate` is set server-side on every call regardless. |
| `ipaddress` | string | Primary `Communication` row (`isprimary=True`, type `IP`). Updated in place or created. |
| `vendorname` | string | `Computer.vendorid`. Vendor row auto-created if missing (free vocab). |
| `modelnumber` | string | `Computer.modelnumberid`, scoped to the vendor when known. Model row auto-created if missing. |
| `osname` | string | `Computer.osid`. Controlled vocab: looked up in `operatingsystems`, NOT auto-created. Unknown value -> warning (row still written, `osid` left unset). |
| `installedsoftware` | array of `{name, version}` | `ComputerInstalledApp` rows for applications shopdb already tracks. Unknown app name -> warning, skipped. |
| `defaultprinter` | string | The default printer's identifier (windows name / share / hostname / port IP). Resolved to a printer asset and linked PC -> printer as a `defaultprinter` relationship. Unresolved -> warning. |
| `printers` | array of strings | All installed network printer identifiers. Each resolves to a printer asset and is linked PC -> printer as a `connectedto` relationship (the default is skipped here since it already links as `defaultprinter`). Unresolved entries -> warning. |
Schema source of truth: `get_collector_schema` in `plugins/computers/plugin.py`.
If you change the payload, change it there and re-check this table.
### PC -> machine links, and part markers
The reported `machinenumber` says which machine the PC drives. It is not the
PC's own identifier, and writing it as one is a defect this contract used to
describe: `assets.assetnumber` is uniquely indexed and the machine already holds
that value, so the insert failed and the bay got a 500 on every report, forever.
A PC takes its hostname as its asset number and keeps it.
- Link: PC -> machine as `controls`, tagged
`assetrelationships.label = 'collector:machine'`. Only tagged rows are ever
archived by a push, so a link made by hand is never touched.
- Unknown number: warns and links nothing. A machine is never invented, because
a mistyped number would create equipment nobody can account for.
- A second PC reporting a machine another PC holds is a CLAIM, not a handover.
A PC imaged for a machine carries that number before it reaches the floor, so
the PC currently running the machine keeps the link while it is still alive -
reporting within 24 hours (`MACHINE_CLAIM_QUIET_HOURS`) and still `In Use`.
The challenger is recorded as a dormant link, which also marks the claim as
already announced so it does not alert every cycle.
- Handover completes on its own once the incumbent has been quiet past that
window, or immediately when someone moves it off `In Use` - which is the
one-step way to force a swap at the moment it happens. The old link is
archived, never deleted, so "which PC ran this machine in June" stays
answerable. The retired PC's STATUS is deliberately not changed: the collector
cannot tell shelved from broken from re-imaged.
- Alerts for handover and contested claims are gated on the
`computers_machinelink_alerts` setting, which ships OFF. A site may run
several PCs on one machine number legitimately, and there the alerts fire on
correct data. Warnings ride in the collector response either way.
The part-marker case below is one instance of a general pattern - several
devices under one parent. `docs/ASSET-COMPOSITION.md` covers when to use it,
what propagation buys you, and how to declare a new device type without a code
change.
A PC reporting `pctype = gea-shopfloor-partmarker` is handled differently,
because several markers can serve one machine number and an operation holds any
number of them. Such a PC gets its own Part Marker machine asset
(`<HOSTNAME>-PARTMARKER`), `controls` it, and the marker is filed `partof` the
operation the PC reported. Both links are tagged `collector:partmarker`. The PC
does NOT claim the operation directly: `controls` propagates through `partof`,
so control still follows from controlling the marker, without two markers
contesting a link only one can hold. Backups from a marker PC file against the
marker rather than the operation.
### PC -> printer relationship sync
When a payload carries `defaultprinter` and/or `printers`, the collector syncs
`AssetRelationship` rows so a PC page shows its printers and a printer page shows
the PCs that use it (both render in the shared Relationships card).
- Resolution: each identifier is matched, first hit wins, against the printer's
`windowsname`, `hostname`, `sharename`, its asset number/name, then any active
printer communications IP. Case-insensitive except the IP (exact). An
identifier that resolves to nothing adds a warning and is skipped; it never
fails the whole push.
- Link types: the default printer links with `defaultprinter` (directional, PC
is the source); every other reported printer links with `connectedto`
(symmetric). A printer that is both default and in `printers` links only as
the default.
- Idempotent: re-reporting the same set creates no duplicate rows (an existing
matching row is reactivated if it was archived, otherwise left as is).
- Stale-link archive: on every push, collector-created links to printers no
longer reported are set inactive. Collector-created rows are tagged in
`assetrelationships.label = 'collector:printers'`; only tagged rows are ever
archived, so links you create by hand in the UI are never touched. A payload
that omits BOTH printer keys leaves all existing printer links untouched
(report an empty `printers: []` to clear the auto links instead).
- Response: the collector response carries `printerlinkcount` and a
`printerlinks` list of `{assetid, relationshiptype}` for the links kept.
### pc-type mapping (configurable per site)
`pctype` (e.g. `gea-shopfloor-cmm`) maps to a shopdb Computer Type through
`pctypemap_<pxetype>` settings. The "Collector PC Types" settings page is
retired (ADR-012): pc-type-to-Computer-Type handling now lives in GE-Enforce
(each imaging PC type is a manifest scope with its own `computertypeid`). The
built-in defaults in `plugins/computers/pctypemap.py` are still seeded on plugin
install and the collector still reads them, so existing enrollment keeps
working. Unmapped pc-types produce a warning, not a failure.
### Classic api.asp field mapping (for porting the PowerShell reporter)
The fleet's classic-ASP reporter posts form fields to
`api.asp?action=updateCompleteAsset`. Map them to the collector JSON as follows:
| Classic `updateCompleteAsset` form field | Collector field |
|---|---|
| `hostname` | `hostname` |
| `machineNo` | `machinenumber` |
| `pcType` | `pctype` |
| `serialNumber` | `serialnumber` |
| `loggedInUser` | `loggedinuser` |
| `lastBootUpTime` / `lastBootTime` | `lastboottime` |
| `manufacturer` | `vendorname` |
| `model` | `modelnumber` |
| `osVersion` | `osname` |
| `installedApps` | `installedsoftware` |
Not carried over (no current home in the computers schema): warranty fields, DNC
config, multi-NIC detail beyond the single primary IP, VNC/WinRM flags. The
classic reporter posts a full `networkInterfaces` array; the collector accepts
only one `ipaddress`, so pick the corp/routable NIC (see the corp-range gate in
the PowerShell below).
---
## 2. Breaking change: querystring `api_key` removed
The API key must now be sent in the `X-API-Key` header. The `?api_key=<key>`
querystring form has been removed on every collector endpoint
(`/api/collector/<plugin>`, `/pc`, `/apps`, `/heartbeat`, `/bulk`, `/status`).
Querystring keys leak into web-server access logs, proxy history, and browser
history. Header-only keeps the secret out of those logs.
Before (no longer works - the key is ignored and the request is rejected 401):
```
POST /api/collector/computers?api_key=SECRET
Content-Type: application/json
{ "hostname": "SHOPPC2335", "machinenumber": "2335" }
```
After (correct):
```
POST /api/collector/computers
X-API-Key: SECRET
Content-Type: application/json
{ "hostname": "SHOPPC2335", "machinenumber": "2335" }
```
PowerShell before/after:
```powershell
# BEFORE (broken)
Invoke-RestMethod -Uri "https://$SiteHost/api/collector/computers?api_key=$key" `
-Method Post -Body $json -ContentType 'application/json'
# AFTER (correct)
Invoke-RestMethod -Uri "https://$SiteHost/api/collector/computers" `
-Method Post -Body $json -ContentType 'application/json' `
-Headers @{ 'X-API-Key' = $key }
```
Any caller still putting `api_key` in the URL must move it to the header.
---
## 3. GE-Enforce implementation guide
GE-Enforce is the SYSTEM-context fleet agent that runs every enforcement cycle on
each shopfloor PC (scheduled task, at-logon + periodic). It lives OUTSIDE this
repo. The facts below are drawn from the real scripts so the payload matches what
the PC already knows:
- `playbook/shopfloor-setup/common/GE-Enforce.ps1` - the enforcement pass. At the
end of each run it writes a status JSON to
`<share>\_outputs\logs\<hostname>\status.json` (interim transport; the fleet
dashboard reads those files). It logs via `Write-EnforceLog` to
`C:\Logs\Shopfloor\enforce-YYYYMMDD.log`. Its machine-number resolution is:
registry `HKLM:\SOFTWARE\WOW6432Node\GE Aircraft Engines\Dnc\General` then
`HKLM:\SOFTWARE\GE Aircraft Engines\Dnc\General`, property `MachineNo`,
skipping the `9999` placeholder, then `C:\Enrollment\machine-number.txt` as
fallback. hostname is `[System.Environment]::MachineName` (live NetBIOS name,
not `$env:COMPUTERNAME`, which goes stale after a post-image rename).
- `playbook/shopfloor-setup/gea-shopfloor-collections/Report-AssetToShopDB.ps1` -
the existing HTTP reporter that POSTs to the classic ASP
`api.asp?action=updateCompleteAsset`. It runs every cycle as a `Type=PS1`
manifest entry (DetectionMethod `Always`) under the SYSTEM task, logs to
`C:\Logs\Shopfloor\report-asset-YYYYMMDD.log`, always exits 0, and reads the
same machine number (reg, then `cmm\cmmid.txt`, then `machine-number.txt`) plus
BIOS serial, OS caption, make/model, logged-in user, and corp-NIC IP from WMI.
The function below is the direct-to-flask analogue of this reporter.
- `playbook/shopfloor-setup/Shopfloor/lib/Update-MachineNumber.ps1` - keeps the
reg `MachineNo` and `C:\Enrollment\machine-number.txt` in sync when a tech
reassigns a bay, so both sources agree.
### Paste-ready reporter
Drop this in as a `Type=PS1` / DetectionMethod `Always` manifest entry (mirroring
`Report-AssetToShopDB.ps1`), or dot-source `Send-ShopdbCollectorReport` from
GE-Enforce.ps1 and call it at the end of the pass. It reads the machine number
exactly the way GE-Enforce already does, builds a payload matching the computers
collector schema, and POSTs with the `X-API-Key` header over TLS 1.2. Every
field name below was checked against `get_collector_schema` in
`plugins/computers/plugin.py`.
The `X-API-Key` value can be EITHER a `COLLECTOR_API_KEY[_COMPUTERS]` env key OR
a managed token scoped to `collector.ingest` (a `shopdb_pat_...` secret; see
"Managed collector tokens"). The script is identical for both - it just carries
whatever `collectorApiKey` the site-config supplies - so switching a site from an
env key to a managed token (and rotating it) is a config change, not a script
change.
```powershell
# Send-ShopdbCollectorReport.ps1
# Reports this PC's identity to shopdb-flask via POST /api/collector/computers.
# Runs as SYSTEM from the GE-Enforce cycle. Always exits without throwing;
# failures are logged, never fatal.
function Get-ShopdbCollectorApiKey {
# NEVER hardcode the key in this script (it lives on the share). Read it
# from a per-site value the SYSTEM/machine account can already reach.
# Preferred: a field in the site-config.json GE-Enforce already parses.
# Fallback: a machine-local file staged at imaging, ACL'd to SYSTEM.
$key = ''
$siteCfg = 'C:\Enrollment\site-config.json'
if (Test-Path -LiteralPath $siteCfg) {
try {
$cfg = Get-Content -LiteralPath $siteCfg -Raw | ConvertFrom-Json
if ($cfg.collectorApiKey) { $key = "$($cfg.collectorApiKey)".Trim() }
} catch {}
}
if (-not $key) {
$keyFile = 'C:\Enrollment\collector-api-key.txt'
if (Test-Path -LiteralPath $keyFile) {
try { $key = (Get-Content -LiteralPath $keyFile -First 1 -ErrorAction Stop).Trim() } catch {}
}
}
return $key
}
function Get-ShopdbMachineNumber {
# Same resolution GE-Enforce.ps1 / Report-AssetToShopDB.ps1 use:
# eDNC registry (skip 9999 placeholder) then C:\Enrollment\machine-number.txt.
$machineNumber = ''
foreach ($rp in @(
'HKLM:\SOFTWARE\WOW6432Node\GE Aircraft Engines\Dnc\General',
'HKLM:\SOFTWARE\GE Aircraft Engines\Dnc\General'
)) {
if ($machineNumber) { break }
if (Test-Path $rp) {
try {
$v = (Get-ItemProperty -Path $rp -Name MachineNo -ErrorAction Stop).MachineNo
if ($v -and "$v".Trim() -ne '9999') { $machineNumber = "$v".Trim() }
} catch {}
}
}
if (-not $machineNumber -and (Test-Path 'C:\Enrollment\machine-number.txt')) {
try {
$v = (Get-Content 'C:\Enrollment\machine-number.txt' -First 1 -ErrorAction Stop).Trim()
if ($v -and $v -ne '9999') { $machineNumber = $v }
} catch {}
}
return $machineNumber
}
function Get-ShopdbCorpIPv4 {
# Pick the corp/AESFMA NIC IP. Same allowed-range gate as
# Report-AssetToShopDB.ps1 - update the ranges if the site re-VLANs.
$allowedRanges = @(
@{ Network = '192.0.2.0'; PrefixLen = 24 },
@{ Network = '198.51.100.0'; PrefixLen = 26 }
)
function ConvertTo-Uint32([string]$ip) {
$bytes = ([System.Net.IPAddress]::Parse($ip)).GetAddressBytes()
[Array]::Reverse($bytes)
return [BitConverter]::ToUInt32($bytes, 0)
}
try {
$ips = Get-NetIPAddress -AddressFamily IPv4 -ErrorAction Stop |
Where-Object { $_.IPAddress -notmatch '^169\.254' -and $_.IPAddress -ne '127.0.0.1' }
foreach ($ipo in $ips) {
$ipInt = ConvertTo-Uint32 $ipo.IPAddress
foreach ($r in $allowedRanges) {
$netInt = ConvertTo-Uint32 $r.Network
$mask = [uint32]([math]::Pow(2, 32) - [math]::Pow(2, 32 - $r.PrefixLen))
if (($ipInt -band $mask) -eq ($netInt -band $mask)) { return $ipo.IPAddress }
}
}
} catch {}
return ''
}
function Send-ShopdbCollectorReport {
param(
[string]$SiteHost = 'shopdb.example.net',
[string]$ApiKey = (Get-ShopdbCollectorApiKey),
[int]$TimeoutSec = 30,
[string]$LogFile = ('C:\Logs\Shopfloor\collector-{0}.log' -f (Get-Date -Format 'yyyyMMdd'))
)
$logDir = Split-Path -Parent $LogFile
if (-not (Test-Path $logDir)) {
New-Item -ItemType Directory -Path $logDir -Force -ErrorAction SilentlyContinue | Out-Null
}
function Write-CollectorLog([string]$msg) {
$ts = Get-Date -Format 'yyyy-MM-dd HH:mm:ss'
"$ts $msg" | Tee-Object -FilePath $LogFile -Append | Out-Null
}
Write-CollectorLog '=== Report to shopdb-flask collector ==='
if (-not $ApiKey) {
Write-CollectorLog 'ERROR no collector API key (site-config.json / collector-api-key.txt) - skipping.'
return
}
# PowerShell 5.1 does not always negotiate TLS 1.2 by default.
try { [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 } catch {}
# --- Gather identity (matches the computers collector schema) ---
$hostname = [System.Environment]::MachineName
if (-not $hostname) { $hostname = $env:COMPUTERNAME }
$machineNumber = Get-ShopdbMachineNumber
$ipAddress = Get-ShopdbCorpIPv4
$serialNumber = ''
try {
$serialNumber = "$((Get-CimInstance -ClassName Win32_BIOS -ErrorAction Stop).SerialNumber)".Trim()
} catch { Write-CollectorLog "WARN BIOS serial read failed: $($_.Exception.Message)" }
$vendorName = ''; $modelNumber = ''; $loggedInUser = ''
try {
$cs = Get-CimInstance -ClassName Win32_ComputerSystem -ErrorAction Stop
$vendorName = "$($cs.Manufacturer)".Trim()
$modelNumber = "$($cs.Model)".Trim()
# UserName is COMPUTERNAME\user or DOMAIN\user; keep the bare username.
if ($cs.UserName) { $loggedInUser = ($cs.UserName -split '\\')[-1].Trim() }
} catch { Write-CollectorLog "WARN ComputerSystem read failed: $($_.Exception.Message)" }
$osName = ''; $lastBootTime = ''
try {
$os = Get-CimInstance -ClassName Win32_OperatingSystem -ErrorAction Stop
$osName = "$($os.Caption)".Trim()
$dv = ''
try { $dv = (Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion' -Name DisplayVersion -ErrorAction Stop).DisplayVersion } catch {}
if ($dv) { $osName += " $dv" }
if ($os.BuildNumber) { $osName += " (build $($os.BuildNumber))" }
$osName = $osName.Trim()
# ISO-8601 so the server's datetime parse accepts it.
try { $lastBootTime = $os.LastBootUpTime.ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ') } catch {}
} catch { Write-CollectorLog "WARN OS read failed: $($_.Exception.Message)" }
$pcType = ''
if (Test-Path -LiteralPath 'C:\Enrollment\pc-type.txt') {
try { $pcType = (Get-Content -LiteralPath 'C:\Enrollment\pc-type.txt' -First 1 -ErrorAction Stop).Trim() } catch {}
}
$pcSubType = ''
if (Test-Path -LiteralPath 'C:\Enrollment\pc-subtype.txt') {
try { $pcSubType = (Get-Content -LiteralPath 'C:\Enrollment\pc-subtype.txt' -First 1 -ErrorAction Stop).Trim() } catch {}
}
# --- Installed printers (Win32_Printer). The Default flag marks the one
# default printer. We report each printer's port name (an IP or a queue
# host for network printers) and fall back to the share/printer name, which
# the collector resolves flexibly against printer windowsname/hostname/IP. ---
$defaultPrinter = ''
$printerIds = @()
try {
$printers = Get-CimInstance -ClassName Win32_Printer -ErrorAction Stop
foreach ($p in $printers) {
if ($p.Local) { continue } # skip local-only (XPS/PDF/OneNote)
# Prefer the port name (IP or queue host); fall back to ShareName,
# then the printer Name.
$identity = $p.PortName
if (-not $identity) { $identity = $p.ShareName }
if (-not $identity) { $identity = $p.Name }
if (-not $identity) { continue }
$printerIds += $identity
if ($p.Default) { $defaultPrinter = $identity }
}
$printerIds = @($printerIds | Select-Object -Unique)
} catch { Write-CollectorLog "WARN printer read failed: $($_.Exception.Message)" }
# --- Build payload. Field names MUST match get_collector_schema exactly. ---
$payload = @{ hostname = $hostname }
if ($defaultPrinter) { $payload['defaultprinter'] = $defaultPrinter }
if ($printerIds.Count) { $payload['printers'] = $printerIds }
if ($machineNumber) { $payload['machinenumber'] = $machineNumber }
if ($pcType) { $payload['pctype'] = $pcType }
if ($pcSubType) { $payload['pcsubtype'] = $pcSubType }
if ($serialNumber) { $payload['serialnumber'] = $serialNumber }
if ($loggedInUser) { $payload['loggedinuser'] = $loggedInUser }
if ($lastBootTime) { $payload['lastboottime'] = $lastBootTime }
if ($ipAddress) { $payload['ipaddress'] = $ipAddress }
if ($vendorName) { $payload['vendorname'] = $vendorName }
if ($modelNumber) { $payload['modelnumber'] = $modelNumber }
if ($osName) { $payload['osname'] = $osName }
$payload['lastcheckin'] = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')
# installedsoftware is optional; add an array of @{ name=..; version=.. }
# here if the manifest introspection is wired to real Application names.
$uri = "https://$SiteHost/api/collector/computers"
$json = $payload | ConvertTo-Json -Depth 5
$headers = @{ 'X-API-Key' = $ApiKey }
Write-CollectorLog ("POST {0} host={1} machineNo={2} pcType={3} serial={4} ip={5}" -f `
$uri, $hostname, $machineNumber, $pcType, $serialNumber, $ipAddress)
try {
$resp = Invoke-RestMethod -Uri $uri -Method Post -Body $json `
-ContentType 'application/json' -Headers $headers `
-TimeoutSec $TimeoutSec -ErrorAction Stop
$d = $resp.data
Write-CollectorLog ("OK action={0} assetid={1} identity={2} warnings={3}" -f `
$d.action, $d.assetid, $d.identityvalue, ((@($d.warnings)) -join '; '))
} catch {
$code = $null
try { $code = [int]$_.Exception.Response.StatusCode.value__ } catch {}
Write-CollectorLog "ERROR POST failed (http=$code): $($_.Exception.Message)"
}
}
# Entry point when run as a standalone manifest PS1 entry:
Send-ShopdbCollectorReport
```
### Where the call slots into the GE-Enforce run
Put the POST at the END of the enforcement pass, after the enforcement scopes are
processed and the PC identity is settled. Two placements, pick one:
- Preferred - a sibling manifest entry, mirroring `Report-AssetToShopDB.ps1`:
add `Send-ShopdbCollectorReport.ps1` as a `Type=PS1`, DetectionMethod `Always`
manifest entry in `common\manifest.json`. It runs every cycle under the SYSTEM
task, keeps GE-Enforce.ps1 untouched, and gets its own log file. This matches
the established pattern the classic reporter already uses.
- Alternative - inline: dot-source the function and call
`Send-ShopdbCollectorReport` inside GE-Enforce.ps1's status-write-back `try`
block (right after `status.json` is written, near the end of the pass). Use
this only if you want the HTTP push to share GE-Enforce's own log and lifecycle.
Either way, run it ALONGSIDE the existing `status.json` share write (and
alongside the classic `Report-AssetToShopDB.ps1`) during rollout - do not remove
the share write until the HTTP path is proven.
### Delivering the API key to clients
Do NOT hardcode the key in the script - the script itself lives on the SFLD
share, so a literal key there is effectively published to everyone with share
read. Two viable options:
1. Per-site value the machine account already reads. GE-Enforce already parses
`C:\Enrollment\site-config.json` (its local copy of the per-site config on the
share). Add one field, `collectorApiKey`, populated per site. The script reads
it with the machine/SYSTEM identity it already runs as. Rotation = update the
one config value at the site, no re-image, no script redeploy.
2. Baked at imaging into a machine-local file (`C:\Enrollment\collector-api-key.txt`)
or an HKLM value, ACL'd to SYSTEM/Administrators, written by the enrollment
pipeline. Tighter blast radius (the key never sits on the share at all), but
rotation requires touching every PC or re-imaging.
Recommendation: use option 1 (`collectorApiKey` in the per-site config), matching
how GE-Enforce already sources its share paths and mirroring how SFLD credentials
are provisioned per site. It keeps the secret out of source control and off the
script, sits at the same trust boundary as everything else the SYSTEM agent
already reads, and supports rotation without a fleet touch. `Get-ShopdbCollectorApiKey`
above already prefers this and falls back to the imaging-baked file, so a site
can start on option 1 and tighten to option 2 later with no code change. The
server side pairs this with `COLLECTOR_API_KEY_COMPUTERS` (per-plugin) so the PC
fleet's key is scoped to the computers collector only.
### Staged rollout
1. Keep everything as-is: `status.json` share write + classic
`Report-AssetToShopDB.ps1` continue running. Provision `collectorApiKey` per
site and set `COLLECTOR_API_KEY_COMPUTERS` on the shopdb-flask server.
2. Add `Send-ShopdbCollectorReport` as a new manifest entry on a handful of pilot
bays (one per pc-type: collections, cmm, keyence, waxtrace, ...). It logs to
`C:\Logs\Shopfloor\collector-YYYYMMDD.log` and cannot break enforcement (it
never throws, exits cleanly).
3. Verify on the server: pilot PCs appear/update in shopdb-flask, `action` is
`created`/`updated`, warnings are understood (map any unmapped `pctype`, add
any missing OS strings to the `operatingsystems` vocab). Cross-check the
collector log's `OK` lines against the audit log.
4. Roll the manifest entry out fleet-wide (it deploys from `common\`, so every
pc-type gets it). Continue running both the share write and the HTTP POST.
5. Once shopdb-flask is the system of record and stable across a full imaging
cycle, retire the classic `Report-AssetToShopDB.ps1` and, if desired, the
`status.json` share write.
---
## 4. Troubleshooting
| Symptom | HTTP | Cause / fix |
|---|---|---|
| `Invalid API key` | 401 | The `X-API-Key` header is missing or wrong. Confirm the client key matches `COLLECTOR_API_KEY_COMPUTERS` (or the shared `COLLECTOR_API_KEY`) on the server. Check the key is being read (`Get-ShopdbCollectorApiKey` returned non-empty). Remember the querystring form no longer works. |
| `Collector API key not configured` | 500 | Fail-closed: neither `COLLECTOR_API_KEY_COMPUTERS` nor `COLLECTOR_API_KEY` is set in the server environment. Set one and restart the app. This is a SERVER config gap, not a client problem. |
| `No collector registered for plugin computers` | 404 | The computers plugin is disabled or not loaded on that instance. Enable it. |
| `hostname is required` / `No data provided` | 400 | Empty body, non-JSON body, or missing identity field. Check `-ContentType 'application/json'` and that `hostname` is set. |
| `Internal error processing collector payload` | 500 | Generic by design - the server does not leak the cause to the caller. The real error (DB, unexpected exception) is in the server log (`Collector upsert failed for computers`). Check there. |
| `warnings` present but `action` is created/updated | 200 | Soft issues only; the row WAS written. Common: unmapped `pctype` (fix the pctypemap setting), unknown `osname` (add it to the `operatingsystems` vocab), unknown app name, `pcsubtype` not stored. No action needed unless the warning matters to you. |
Client-side log for the fleet reporter: `C:\Logs\Shopfloor\collector-YYYYMMDD.log`.
Server-side: the Flask app log (the collector logs upsert failures and per-plugin
schema failures there).