shopdb-flask/plugins/endpoints/AGENT-CONTRACT.md

# Endpoint agent contract (C# build reference)

Companion to DESIGN-DECISIONS-agent.md. This is the concrete wire contract the
future C# agent targets, derived from `EndpointsPlugin.get_collector_schema()`
and the generic collector dispatcher in `shopdb/core/api/collector.py`. When the
schema changes, update this file in the same change.

## 1. Transport

- Method/URL: `POST {baseurl}/api/collector/endpoints`
- Auth header: `X-API-Key: <collector key>` (Bearer token also accepted). The
  key is a UI-managed, endpoints-only service token minted from Settings ->
  Endpoint agent key (section 7). A bootstrap `.env` collector key still works
  until the first UI key is generated, then it is switched off (section 7). No
  querystring fallback.
- Body: `application/json`, a single flat object (NOT an array). One POST per
  report per endpoint.
- Identity field: `hostname` (required, case-insensitive).

## 2. Request payload

Only `hostname` is required. Every other field is optional; the server upserts
only the keys that are present, so a lightweight report can omit inventory
arrays. `adopt` is first-contact only.

```jsonc
{
  "hostname": "SHOP-PC-014",     // required, resolved to computers.hostname
  "adopt": true,                 // first run ONLY; delegates PC creation to computers plugin
  "loggedinuser": "j.smith",
  "serialnumber": "5CD1234ABC",  // adoption subset (forwarded to computers)
  "pctype": "CNC Controller",    // adoption subset
  "machinenumber": "12345",      // adoption subset; "" or "9999" == none
  "agentversion": "1.0.0",
  "trayversion": "1.0.0",            // file version of the interactive tray exe, omitted if no tray
  "lastboottime": "2025-01-14T06:12:00Z",   // ISO-8601 UTC
  "uptimeseconds": 86400,
  "idleseconds": 120,
  "cpupercent": 12.5,
  "memorypercent": 47.0,
  "services": [ /* opaque array, stored as JSON */ ],
  "ports": [ /* opaque array */ ],
  "interfaces": [
    {
      "alias": "Intel(R) Ethernet Connection",  // adapter description
      "ipv4": "10.20.30.40",                     // "" if none
      "mac": "AA:BB:CC:DD:EE:FF",
      "gateway": "10.20.30.1",                   // "" for the static machine-IPS NIC
      "subnet": "255.255.255.0",
      "dhcp": false
    }
  ],
  "scheduledtasks": [ /* opaque array */ ],
  "disks": [
	{
	  "drive": "C:",
	  "label": "OS",
	  "filesystem": "NTFS",
	  "sizebytes": 511000000000,
	  "freebytes": 128000000000,
	  "usedbytes": 383000000000,
	  "freepercent": 25.0            // precomputed; a threshold is a direct compare
	}
  ],
  "bios": { /* opaque object: vendor, version, release date */ },
  "memory": { /* opaque object: totalbytes, freebytes */ },
  "processor": { /* opaque object: name, cores, logicalprocessors, maxclockmhz */ },
  "osinfo": { /* opaque object: build, version, install date */ },
  "domain": "shopfloor.example.com",  // domain or workgroup membership
  "pendingreboot": false,             // omit when unknown; absent != false
  "rebootreasons": [],                // [] = checked, nothing pending; see notes
  "bootinfo": {
    "hiberbootenabled": false,        // fast startup CONFIGURED
    "lastboottype": "Cold"            // what the last boot DID: Cold|Fast startup|Resume|Unknown
  },
  "defender": {                       // persisted, deliberately not surfaced
    "serviceenabled": true,
    "realtimeenabled": true,
    "signatureagedays": 0,            // real 0 = updated today; null = did not report
    "antispywareagedays": 0,
    "quickscanagedays": 0
  },
  "diskhealth": [                     // per PHYSICAL device; see notes
    {
      "deviceid": "0",
      "model": "Samsung SSD 990 PRO 1TB",
      "mediatype": "SSD",
      "bustype": "NVMe",
      "healthstatus": "Healthy",
      "predictfailure": false,        // precomputed; null = drive did not answer
      "source": "physicaldisk"        // or "diskdrive"
    }
  ],
  "usercontext": {                    // omitted entirely when nobody is signed in
	"netuse": [
	  {
		"localname": "Z:",
		"remotename": "\\\\fileserver\\programs",
		"status": "OK",
		"username": "SHOPFLOOR\\operator",
		"persistent": true
	  }
	],
	"userlisteners": [
	  { "localaddress": "0.0.0.0", "localport": 7878, "processid": 4512, "processname": "DncMain" }
	],
	"idleseconds": 45,
	"username": "operator",
	"collectedat": "2025-01-14T06:12:00Z"
  },
  "events": [
	{
	  "eventid": 6008,                        // required per item; non-catalog IDs are dropped, not errored
	  "eventrecordid": 184412,                // Windows record id; dedupe key, see notes
	  "eventchannel": "System",
	  "eventsource": "EventLog",
	  "eventtime": "2025-01-14T06:11:58Z",
	  "eventmessage": "The previous system shutdown was unexpected.",
	  "fields": { "any": "extra" }            // opaque object, stored as JSON
	}
  ]
}
```

Notes:
- `adopt=true` is sent ONLY by the installer's first run. Steady-state reports
  omit it (or send `false`). After adoption, an unknown hostname is a hard
  validation error again (DESIGN-DECISIONS-agent.md 1a).
- The adoption subset (`serialnumber`, `pctype`, `machinenumber`, plus
  `loggedinuser`) is forwarded to the `computers` collector so the newly created
  PC is not a bare shell. Send them on first contact.
- `services`/`ports`/`interfaces`/`scheduledtasks` are stored verbatim as JSON;
  the server does not validate their inner shape yet. Keep them stable so the UI
  can render them later. `interfaces` reports EVERY hardware NIC (not just the
  primary) so the gatewayless static NIC that links a PC to its machine's IPS is
  preserved (DESIGN-DECISIONS-agent.md 9.3); each entry is
  `{alias, ipv4, mac, gateway, subnet, dhcp}`.
- `machinenumber` is discovered on-box by the agent from the eDNC registry
  (`HKLM\SOFTWARE\[WOW6432Node\]GE Aircraft Engines\DNC\General:MachineNo`) or
  the UDC `udc_settings_*.json` (`GeneralSettings.MachineNumber`), in that order
  (DESIGN-DECISIONS-agent.md 9.4/9.5). `""` or `"9999"` means none.
- `eventrecordid` is the Windows event record id: monotonic and unique within a
  channel on a machine. The agent uses it as a per-channel harvest watermark and
  the server enforces a unique index on
  `(assetid, eventchannel, eventrecordid)`, so one real event is stored exactly
  once. It is nullable: an older agent omits it, and MySQL permits repeated
  NULLs in a unique index, so those reports keep the previous append behavior
  rather than failing ingest. Send it whenever the source record exposes one.
- `disks`/`bios`/`memory`/`processor`/`osinfo`/`usercontext` are stored verbatim
  as JSON snapshots on the endpoint state, newest-wins. `domain` and
  `pendingreboot` are scalar columns because the fleet view filters and alerts
  on them.
- OMITTING a key is not the same as sending an empty value. Every one of these
  fields is applied only when the key is PRESENT, so a report that omits it
  preserves the last-known value. Send `null`/`[]` only when you mean "this is
  genuinely empty now"; omit when you simply did not collect it this cycle.
- `pendingreboot` follows that rule strictly: omit it when the check failed. A
  literal `false` asserts the endpoint is confirmed NOT holding a reboot.
- `rebootreasons` is NOT redundant with `pendingreboot`. That key is a single
  bool OR-ed from three registry signals and cannot say which fired, and the
  difference is operational: `pendingfilerename` means files are staged for
  replacement on next boot, so the box is mid-servicing, while a lone
  `componentbasedservicing` or `windowsupdate` key can sit set for weeks on a
  machine nobody has rebooted. Same boolean, very different urgency. An EMPTY
  array means checked and nothing pending; OMIT the key when the check failed.
- `bootinfo` carries two distinct facts that must not be collapsed:
  `hiberbootenabled` is whether fast startup is CONFIGURED (registry, always
  readable) and `lastboottype` is what the last boot ACTUALLY did (Kernel-Boot
  event 27). The registry cannot tell you what the last boot did, and the event
  may have aged out of the log, so neither substitutes for the other. This
  exists because uptime LIES under fast startup: a box reports days of uptime
  after what the operator experienced as a shutdown.
- `defender` is stored but deliberately NOT surfaced. Defender produces known
  false positives in this environment, so wiring it to a card now would train
  operators to ignore that card. Signature/scan ages are in DAYS; a real `0`
  means updated today, so send `null` when Defender did not report rather than
  defaulting to zero.
- `interfaces` entries gained `linkspeedbps` and `fullduplex`, paired on by MAC
  from `MSFT_NetAdapter`. Both are `null` when link state was unreadable or the
  adapter is disconnected - an unknown duplex must not read as half-duplex.
- `diskhealth` is per PHYSICAL device and is NOT the same list as `disks`, which
  is per volume. One physical drive can back several volumes or none the
  operator sees, so the two do not share a key and are not merged.
- `diskhealth` carries BOTH agent sources and the server does not reconcile
  them. `source: "physicaldisk"` is MSFT_PhysicalDisk, the only source that
  reports NVMe at all - the classic ATA SMART class
  (`MSStorageDriver_FailurePredictStatus`) returns zero instances on an NVMe
  machine, so an agent collecting only that would report an empty health list on
  modern hardware and look like "no problems". `source: "diskdrive"` is
  Win32_DiskDrive.Status, covering ATA/SATA and hosts where the Storage
  namespace is unavailable. When both answer for one drive and disagree, both
  rows are shown; that disagreement is the signal.
- `predictfailure` is precomputed by the agent so an alert is a boolean test
  rather than a string match against a Windows enum the server would have to
  keep in sync. `null` means the drive did not answer: unknown, NOT healthy.
  `healthstatus` keeps the human-readable source value alongside it.
- `usercontext` is gathered by a probe the TRAY runs inside the signed-in
  operator's session, because mapped drives, user-owned listeners and idle time
  do not exist in the agent's own LocalSystem session 0. It is omitted when no
  one is signed in, which is a normal unattended state, not a fault. The
  security-context split that makes this work is normative; see section 5.
- `ports` carries LISTENING sockets only. The full connection table is available
  on demand via the `network_connections` command instead, because established
  connections churn constantly and would change every report for no diagnostic
  gain.

## 3. Response

Success (HTTP 200), ADR-006 envelope:

```jsonc
{
  "status": "ok",
  "action": "created",           // "created" first time, "updated" thereafter
  "assetid": 4821,
  "identityvalue": "SHOP-PC-014",
  "warnings": [
	"adopted new computer for hostname: 'SHOP-PC-014'"   // present only on adoption
  ]
}
```

Validation failure (HTTP 400) - e.g. unknown hostname with `adopt` absent:

```jsonc
{ "status": "error", "error": { "code": "VALIDATION_ERROR",
  "message": "could not resolve hostname to a computer: 'SHOP-PC-014'" } }
```

Agent handling: treat non-200 as retryable except a `VALIDATION_ERROR` that
names an unresolved hostname on a NON-first-contact report - that means the PC
was removed server-side; surface it, do not spin.

## 4. C# DTOs

Match the JSON keys EXACTLY (lowercase, no underscores) to satisfy the naming
rule; use attributes so C# properties can stay PascalCase without a camelCase
serializer changing the wire keys.

```csharp
using System.Text.Json.Serialization;

public sealed class EndpointReport
{
	[JsonPropertyName("hostname")]      public string Hostname { get; set; } = "";
	[JsonPropertyName("adopt")]         public bool? Adopt { get; set; }
	[JsonPropertyName("loggedinuser")]  public string? LoggedInUser { get; set; }
	[JsonPropertyName("serialnumber")]  public string? SerialNumber { get; set; }
	[JsonPropertyName("pctype")]        public string? PcType { get; set; }
	[JsonPropertyName("machinenumber")] public string? MachineNumber { get; set; }
	[JsonPropertyName("agentversion")]  public string? AgentVersion { get; set; }
	[JsonPropertyName("trayversion")]   public string? TrayVersion { get; set; }
	[JsonPropertyName("lastboottime")]  public DateTimeOffset? LastBootTime { get; set; }
	[JsonPropertyName("uptimeseconds")] public long? UptimeSeconds { get; set; }
	[JsonPropertyName("idleseconds")]   public long? IdleSeconds { get; set; }
	[JsonPropertyName("cpupercent")]    public double? CpuPercent { get; set; }
	[JsonPropertyName("memorypercent")] public double? MemoryPercent { get; set; }
	[JsonPropertyName("services")]      public object[]? Services { get; set; }
	[JsonPropertyName("ports")]         public object[]? Ports { get; set; }
	[JsonPropertyName("interfaces")]    public object[]? Interfaces { get; set; }
	[JsonPropertyName("scheduledtasks")]public object[]? ScheduledTasks { get; set; }
	[JsonPropertyName("disks")]         public object[]? Disks { get; set; }
	[JsonPropertyName("bios")]          public object? Bios { get; set; }
	[JsonPropertyName("memory")]        public object? Memory { get; set; }
	[JsonPropertyName("processor")]     public object? Processor { get; set; }
	[JsonPropertyName("osinfo")]        public object? OsInfo { get; set; }
	[JsonPropertyName("domain")]        public string? Domain { get; set; }
	[JsonPropertyName("pendingreboot")] public bool? PendingReboot { get; set; }
	[JsonPropertyName("usercontext")]   public object? UserContext { get; set; }
	[JsonPropertyName("applications")]  public object[]? Applications { get; set; }
	[JsonPropertyName("events")]        public EndpointEventRecord[]? Events { get; set; }
}

public sealed class EndpointEventRecord
{
	[JsonPropertyName("eventid")]       public int EventId { get; set; }
	[JsonPropertyName("eventrecordid")]  public long? EventRecordId { get; set; }
	[JsonPropertyName("eventchannel")]  public string? EventChannel { get; set; }
	[JsonPropertyName("eventsource")]  public string? EventSource { get; set; }
	[JsonPropertyName("eventtime")]    public DateTimeOffset? EventTime { get; set; }
	[JsonPropertyName("eventmessage")] public string? EventMessage { get; set; }
	[JsonPropertyName("fields")]       public Dictionary<string, object>? Fields { get; set; }
}

public sealed class CollectorResponse
{
	[JsonPropertyName("status")]        public string Status { get; set; } = "";
	[JsonPropertyName("action")]        public string? Action { get; set; }
	[JsonPropertyName("assetid")]       public int? AssetId { get; set; }
	[JsonPropertyName("identityvalue")] public string? IdentityValue { get; set; }
	[JsonPropertyName("warnings")]      public string[]? Warnings { get; set; }
}
```

Serializer settings: `DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull`
so omitted fields are not sent (server upserts only present keys). Send UTC ISO-8601.

## 5. Security contexts, spool handoff and log destinations (NORMATIVE)

Three processes run as three different principals. Confusing them is what makes
`usercontext` silently empty, so the boundaries are fixed by contract.

| Process | Runs as | May write to |
|---|---|---|
| Service (`shopdbagent.exe`) | LocalSystem, session 0 | `%ProgramData%\ShopDB\EndpointAgent\logs\agent-<date>.log` |
| Tray + probe (`shopdb-endpoint-tray.exe`, `shopdbagent.exe --collect-usercontext`) | `ShopFloor` operator, interactive session | `C:\Users\ShopFloor\ShopDB\` and `C:\Users\ShopFloor\ShopDB\Logs\<datetime>-trayagent.log` |
| Installer (`install-shopdbagent.ps1`) | Elevated admin, one-shot | `%ProgramData%\ShopDB\Logs\<datetime>-shopdbinstalls.log` |

Rules:

1. **The operator cannot write to `C:\Program Files` or to the SYSTEM-owned
   `%ProgramData%\ShopDB\EndpointAgent\logs`.** Anything running in the user's
   session MUST log under `C:\Users\ShopFloor\ShopDB\Logs`. The file log sink
   degrades to a silent no-op when it cannot create its directory, so violating
   this does not raise an error - it discards the log line and the failure
   becomes invisible.
2. **Only the tray collects user context**, because mapped drives, user-owned
   listeners and idle time are per-session and do not exist in session 0. The
   service MUST NOT try to descend into the session to collect them.
3. **The tray WRITES the spool; the service READS it and then REMOVES it.**
   The service consumes the file after a successful parse, so a tray that has
   stopped running cannot keep re-reporting stale mapped drives as current. A
   file that fails to parse (a read that raced the write) is left in place.
4. **The spool location is fixed at `C:\Users\ShopFloor\ShopDB\usercontext.json`.**
   Every shop-floor endpoint uses the same local `ShopFloor` account. Resolving
   it through `WTSQueryUserToken` is a fallback only: that call is privileged,
   endpoint protection blocks it, and when it fails the writer and reader
   disagree about the location with no error on either side.
5. **Losing user context is never fatal.** If the `ShopFloor` profile is absent
   or unwritable, `usercontext` is omitted and telemetry, commands, inventory
   and the command channel continue unaffected.
6. **The installer provisions** `C:\Users\ShopFloor\ShopDB\` and its `Logs`
   subdirectory with `ShopFloor: Modify` before the tray first runs.

The three log destinations are deliberately separate: an install is one-shot and
elevated, the service log is a rolling runtime log, and the tray log is the only
one written from the operator's session. Merging them makes an install
indistinguishable from runtime behaviour when reading a failure back.

## 6. "Build site agent" settings flow (build cadence)

Goal: an operator in ShopDB Settings produces an instance-locked `.exe` that
knows its server URL and its collector key, needing no manual config on the
client.

1. Settings -> "Build site agent": operator picks/creates a collector-scoped
   managed token (the same token accepted by `X-API-Key` on
   `/api/collector/endpoints`). This binds the build to one ShopDB instance.
2. Server stamps a small config (base URL + token + reporting cadence) into the
   agent build artifact. Instance-locked = the token only works against that
   server, and the server can revoke it to kill a leaked build.
3. First run on a client:
   - Collect the ShopDB PowerShell asset-collector data set (minus the CSV
	 emit): hostname, logged-in user, serial number, pc type, machine number,
	 boot time, plus the curated event tail.
   - POST once with `adopt=true`. The endpoints plugin delegates PC creation to
	 the `computers` collector, then writes the first `EndpointState` in the
	 same commit. The endpoint appears in the Fleet UI immediately - no second
	 call, no separate enrollment endpoint.
4. Subsequent runs POST on the configured cadence with `adopt` omitted; strict
   hostname resolution applies from then on.

Open item (from DESIGN-DECISIONS-agent.md 1a): whether `adopt` stays a
per-report flag or is promoted to a site setting (strict vs self-adopting
fleet). Default proposed: agent sends `adopt=true` on first run only.

## 7. UI-managed agent key (kill switch)

The endpoint agent authenticates with a single UI-managed key scoped to ONLY
`collector.ingest.endpoints`, so rotating or revoking it stops all
endpoint-agent traffic without affecting the computers or geenforce collectors.

- Settings -> Endpoint agent key (admin only) generates the key. The full secret
  is shown ONCE at generation and stored only as a hash afterwards; configure it
  as the agent `X-API-Key`.
- Generating a key is a hard cutover: any previous endpoint key is revoked in the
  same request, and the bootstrap `.env` fallback for endpoints is switched off
  (`plugin.endpoints.envkeyfallback` -> false), so the managed key becomes the
  sole credential.
- Regenerate to rotate (old key dies immediately). Revoke to stop all
  endpoint-agent reporting until a new key is generated - the kill switch for a
  leaked agent.
- Backend: `GET/POST /api/endpoints/agentkey`, `POST
  /api/endpoints/agentkey/regenerate`, `DELETE /api/endpoints/agentkey`. These
  use the `mint_service_token`/`list_service_tokens`/`revoke_service_tokens`
  contract helpers (shopdb.api, contract 0.22.0).

The bootstrap `.env` key (`COLLECTOR_API_KEY[_ENDPOINTS]`) remains only as a
first-run path before a UI key exists; it cannot be the runtime kill switch
because container env is injected at boot and cannot be rotated live.

## 8. Command channel (WebUI -> agent)

The WebUI can send an endpoint an ALLOWLISTED command (never an arbitrary
command string). Delivery is agent-initiated long-poll, so shop-floor PCs open
no inbound port: the agent makes an outbound request the server holds open until
a command is queued, runs it locally, and posts the result back. All three
routes reuse the same endpoints collector key (section 7) via `X-API-Key`, so
the command channel shares the kill switch and adds no second secret.

Allowlisted actions (server source of truth: `plugins/endpoints/commands.py`):

- `mtconnect_start` - run the MTConnect UDC agent start script
  (`C:\ProgramData\UDC\MTConnect_UDC\Agent\run.bat`).
- `mtconnect_stop` - stop the running MTConnect UDC agent process.
- `mtconnect_status` - report whether the MTConnect UDC agent is running.
- `network_connections` - return the full TCP connection table with owning
  process ids and names. Read-only diagnostic. The result is capped at 500
  connections with a `truncated` flag in `detail`, because an oversized payload
  rejected by the server returns nothing at all and a partial answer is more
  useful than a failed one. Standard reports carry listening ports only.

### 7.1 Claim the next command (long-poll)

`GET {baseurl}/api/endpoints/commands/next?hostname=<hostname>`

- Auth: `X-API-Key` (endpoints collector key).
- The server holds the request open up to ~25s. If a command is queued it is
  atomically claimed (marked running) and returned; only one poll ever claims a
  given command.
- Command queued (HTTP 200):

```jsonc
{ "status": "ok", "data": { "endpointcommandid": 91, "action": "mtconnect_start" } }
```

- Nothing queued within the hold window: HTTP 204 (no body). Re-poll.

### 7.2 Report a command result

`POST {baseurl}/api/endpoints/commands/<endpointcommandid>/result`

- Auth: `X-API-Key` (endpoints collector key).
- Body:

```jsonc
{
  "succeeded": true,               // required; false marks the command failed
  "message": "MTConnect started",  // short summary, optional (<= 2000 chars)
  "mtconnectrunning": true,        // optional; observed run state after the action
  "detail": { "any": "extra" }     // optional structured detail, stored as JSON
}
```

- When `mtconnectrunning` is present the server caches it on the endpoint's
  `EndpointState` (`mtconnectrunning` + `mtconnectcheckedat`) so the WebUI toggle
  shows a real state between polls. Every `mtconnect_*` action SHOULD probe and
  report the resulting run state so the toggle never lies.

### 7.3 Enqueue (WebUI side, for reference)

`POST /api/endpoints/<assetid>/commands` with `{ "action": "mtconnect_start" }`,
gated by user JWT + `endpoints.command`. Only one command may be in flight per
endpoint (a second enqueue returns HTTP 409 until the first finalizes).