shopdb-flask/docs/COLLECTOR-PAYLOADS.md

Collector payload contract

What a fleet client may send to POST /api/collector/<plugin> and what the server does with each field. Authentication is a separate subject and lives in COLLECTOR-AUTH; the rationale for the contract shape is ADR-006.

Source of truth is get_collector_schema in each plugin (plugins/computers/plugin.py, plugins/printers/plugin.py). If you change a payload, change it there and re-check the table here.

---

1. The generic endpoint

One dynamic route serves every enabled plugin that returns a collector schema. For the PC fleet that is POST /api/collector/computers.

Request flow inside generic_collect:

1. Look up the plugin's schema. Unknown or disabled plugin -> 404 No collector registered for plugin <plugin>.
2. Authenticate (COLLECTOR-AUTH).
3. Parse the JSON body. Missing or non-object -> 400 No data provided.
4. Resolve the identity field the schema names (hostname for both collectors below). Missing or blank -> 400 <identityfield> is required.
5. Idempotent upsert: the plugin's apply_collector_payload finds the existing asset by identity and updates it, or inserts a new one. The same identity on a later submission updates the same row - re-imaging a PC does not duplicate it, and existing asset relationships are preserved.
6. Audit: every accepted submission writes an AuditLog row (action created/updated, entity Collector, details = collector:<plugin> action=<action>), then commits.

Response

HTTP 200, in the standard envelope, with the collector result under data:

{
  "status": "success",
  "data": {
    "status": "ok",
    "action": "created",
    "assetid": 12345,
    "identityvalue": "SHOPPC2335",
    "warnings": ["unknown operating system: Windows 11 Enterprise 23H2 (build 22631)"]
  },
  "message": "computers collector created",
  "meta": { "timestamp": "...", "requestid": "..." }
}

action is created, updated or noop. warnings is a list of soft problems (unmapped pc-type, unknown OS, unknown app, un-stored pcsubtype) that did NOT fail the request - the row was still written.

Errors use the same envelope with status: "error" and the detail under data.error:

{ "status": "error", "data": { "error": { "code": "UNAUTHORIZED", "message": "Invalid API key" } }, "meta": { } }

A plugin raises ValueError for controlled validation failures and its message is returned verbatim (400). Any other exception is caught, logged server-side, the transaction rolled back, and the caller gets a generic 500 Internal error processing collector payload with no internal detail - by design. Check the server log to see what actually failed.

Schema discovery

GET /api/collector/_schemas returns the schema for every enabled plugin. It is JWT-protected, not key-protected: it is for humans and tooling introspecting what payloads are accepted, not for the headless collectors themselves.

---

2. Computers payload

Identity field hostname. Every other field is optional and patch-style: an omitted field leaves the existing value untouched, so a bare report never blanks a column.

Payload field	Type	Server behaviour (apply_collector_payload)
hostname (required)	string	Identity. Matches Computer.hostname (case-insensitive), then falls back to Asset.assetnumber. New asset created if no match.
machinenumber	string	Identifies the MACHINE this PC drives, never the PC. A new PC always takes assetnumber = hostname, and an existing PC's assetnumber is left alone. The number resolves a machine asset and builds a PC -> machine controls relationship; an unknown number warns rather than creating a machine. The placeholder 9999 and empty string link nothing. See section 3.
deviceid	string	Identifies the DEVICE hanging off this PC, read from C:\Enrollment\asset-id.txt. It names the asset and nothing else - it does NOT say what kind of device it is, because the pc-type already does - so the same file serves a part marker, a Keyence, a Genspect or anything declared later. SEPARATE from machinenumber on purpose: "which bay is this" and "which device is this" are different facts, and machinenumber is what a site's own bay-gated configuration keys off, so naming a device there would silently stop every bay-gated rule from matching. An unknown value warns rather than minting; a value resolving to the wrong KIND of asset is refused with the asset named. See section 4.
measuringtoolid	string	The 0.12.0 name for the same thing, read from C:\Enrollment\measuringtool-id.txt. Still accepted as an alias so a bay already staged with that file keeps reporting; deviceid wins when both arrive. Prefer asset-id.txt for anything new.
pctype	string	gea-shopfloor-* imaging type -> Computer.computertypeid via the configurable pctypemap settings. Unmapped value -> warning, not error. See section 5.
pcsubtype	string	Accepted but not stored yet -> warning.
serialnumber	string	Asset.serialnumber.
loggedinuser	string	Computer.loggedinuser. (currentuser is also accepted as a legacy alias.)
lastboottime	ISO-8601 datetime	Computer.lastboottime. Unparseable -> warning.
lastcheckin	ISO-8601 datetime	Accepted (heartbeat semantics). lastreporteddate is set server-side on every call regardless.
interfaces	array of {ipaddress, macaddress, adaptername, isdefaultroute}	EVERY connected hardware NIC with a routable IPv4, in no meaningful order. One Communication row per NIC. See "Which NIC is the PC's address" below.
ipaddress	string	The one-address form, kept for bays on an older collector. A payload carrying both wins on interfaces.
vendorname	string	Computer.vendorid. Vendor row auto-created if missing (free vocab).
modelnumber	string	Computer.modelnumberid, scoped to the vendor when known. Model row auto-created if missing.
osname	string	Computer.osid. Controlled vocab: looked up in operatingsystems, NOT auto-created. Unknown value -> warning (row still written, osid left unset).
installedsoftware	array of {name, version}	ComputerInstalledApp rows for applications shopdb already tracks. Unknown app name -> warning, skipped.
accessprotocols	array of strings	Remote-access protocols this PC actually exposes, by catalog name (VNC, WinRM, RDP). PRESENCE of the key drives the sync: reported protocols are activated, catalogued ones not reported are deactivated. Omit the key entirely to leave existing rows alone - most came from the legacy isvnc/iswinrm migration.

defaultprinter and printers are retired. See section 6.

Which NIC is the PC's address

The bay reports every NIC and takes no view. The server picks, in this order:

1. The first reported address inside a CIDR in the collector_corporate_ranges setting.
2. Else the NIC the bay flagged isdefaultroute.
3. Else the first reported.

Step 3 is a guess, and configuring the ranges is what makes it stop being one. Choosing on the bay was the wrong place for it: a bay cannot know a site's addressing, so a CMM with a fast link to its controller reported the machine-side address as the PC's and never sent the corporate one at all.

Rows are keyed on the MAC where the bay sent one, because an address moves between adapters and a MAC does not, and fall back to the address for a bay that reports none. Exactly one row is flagged primary, and only among the rows this report touched. A NIC the bay did NOT report is left alone: a NIC that is down at check-in is not evidence the record is wrong, and hand-entered detail lives on those same rows.

3. PC -> machine links, and part markers

The reported machinenumber says which machine the PC drives. It is not the PC's own identifier, and writing it as one is a defect this contract used to describe: assets.assetnumber is uniquely indexed and the machine already holds that value, so the insert failed and the bay got a 500 on every report, forever. A PC takes its hostname as its asset number and keeps it.

• Link: PC -> machine as controls, tagged assetrelationships.label = 'collector:machine'. Only tagged rows are ever archived by a push, so a link made by hand is never touched.
• Unknown number: warns and links nothing. A machine is never invented, because a mistyped number would create equipment nobody can account for.
• A second PC reporting a machine another PC holds is a CLAIM, not a handover. A PC imaged for a machine carries that number before it reaches the floor, so the PC currently running the machine keeps the link while it is still alive - reporting within 24 hours (MACHINE_CLAIM_QUIET_HOURS) and still In Use. The challenger is recorded as a dormant link, which also marks the claim as already announced so it does not alert every cycle.
• Handover completes on its own once the incumbent has been quiet past that window, or immediately when someone moves it off In Use - which is the one-step way to force a swap at the moment it happens. The old link is archived, never deleted, so "which PC ran this machine in June" stays answerable. The retired PC's STATUS is deliberately not changed: the collector cannot tell shelved from broken from re-imaged.
• Alerts for handover and contested claims are gated on the computers_machinelink_alerts setting, which ships OFF. A site may run several PCs on one machine number legitimately, and there the alerts fire on correct data. Warnings ride in the collector response either way.

A PC reporting pctype = gea-shopfloor-partmarker is handled differently, because several markers can serve one machine number and an operation holds any number of them. Such a PC gets its own Part Marker machine asset (<HOSTNAME>-PARTMARKER), controls it, and the marker is filed partof the operation the PC reported. Both links are tagged collector:partmarker. The PC does NOT claim the operation directly: controls propagates through partof, so control still follows from controlling the marker, without two markers contesting a link only one can hold. Backups from a marker PC file against the marker rather than the operation.

That is one instance of a general pattern - several devices under one parent. ASSET-COMPOSITION covers when to use it, what propagation buys you, and how to declare a new device type without a code change.

4. PC -> device links (asset-id.txt)

Some pc-types mean the PC drives an attached DEVICE: a part marker, or a metrology instrument such as a CMM, Keyence, Genspect or wax-trace unit. SUBORDINATE_DEVICE_MAP in plugins/computers/pctypemap.py declares which, and a site can add its own through a subordinatedevice_<pctype> setting without a code change.

On this line only the part-marker case does anything. The four metrology pc-types declare assettype: measuring_tool, and the plugin that owned that asset class is not bundled here (ADR-018: metrology instruments are machines). The collector detects the absence, skips the link, and says so in the response warnings - it does not fail the report, and it does not mint anything. File a CMM as a machine and link it by hand.

C:\Enrollment\asset-id.txt names that device: one line, the device's assetnumber. It does NOT record what kind of device it is - the pc-type already does - so a bay declared later needs no new file and no client change.

Resolution runs most-stable-identity first, and ADOPTS before it mints:

1. deviceid - the asset named in asset-id.txt.
2. A prior collector link from this PC, reactivated.
3. An existing device this PC already controls that the collector did NOT create - a legacy MT-#### row, or one somebody made by hand. Adopting stamps the label so it is ours from then on.
4. Mint, only when none of the above matched.

Why the file exists, and why it is step 1. Every other identity here is derived from the PC. Step 2 looks for a link from THIS PC asset; step 4 mints <PC number>-<SUFFIX>, and a PC's asset number is its hostname. All of them survive a re-image and NONE of them survives a PC SWAP: a new hostname is a new PC asset with no prior link and a predicted number that has never existed, so the same physical device gets a second record while the first keeps its config and backup history under a dead PC's name. That is how 43 legacy MT-#### tools ended up shadowed by minted twins, three records deep in places. Treat anything minted as a placeholder to be reconciled.

Two guards on the file's contents:

• An unresolvable value warns and links nothing, rather than inventing a phantom device nobody can account for.
• A value resolving to the wrong kind of asset is refused, naming the asset it hit. A machine number pasted into asset-id.txt would otherwise file that MACHINE under a device label the collector also owns - a link that reads as an instrument, or as a marker, everywhere downstream.

A part marker additionally stays partof the operation from machinenumber, because several markers serve one operation. Naming the marker in asset-id.txt changes which marker asset the PC controls; it does not change which operation that marker files under.

5. pc-type mapping (configurable per site)

pctype (e.g. gea-shopfloor-cmm) maps to a shopdb Computer Type through pctypemap_<pxetype> settings. The built-in defaults in plugins/computers/pctypemap.py are seeded on plugin install and the collector reads them, so existing enrollment keeps working; add a setting of that name to map an imaging type of your own. Unmapped pc-types produce a warning, not a failure, because a bay must not stop reporting its inventory over a naming question.

6. PC -> printer relationships are NOT set here (retired)

A collector report has never been allowed to set a printer assignment, and as of 2026-08-24 it no longer can. defaultprinter and printers used to be accepted on the computers payload and written as defaultprinter/connectedto relationship rows. That contradicted "observed is not assigned" below, and this contract carried both claims at once while the code followed the wrong one.

If you are still sending either key: your push still succeeds - they are ignored, and the response carries a warning naming them. To report what a PC actually has, use POST /api/collector/printers; to turn that into an assignment, a person calls POST /api/printers/assignments/seed-from-observed/<assetid> after reviewing the comparison.

7. Printers payload (observed print queues)

POST /api/collector/printers. ShopDB has always known what a bay SHOULD have - GET /api/printers/for-host/<hostname>, applied by Set-ShopdbPrinters.ps1 - and has never known what it actually has. This payload is that missing half. Same dispatcher, same auth, same audit row; only the payload and the plugin differ.

• Schema + apply: get_collector_schema / apply_collector_payload in plugins/printers/plugin.py.
• Client: plugins/printers/client/Report-PrintersToShopDB.ps1 (SYSTEM context, Type=PS1 / DetectionMethod Always manifest entry, logs to C:\Logs\Shopfloor\report-printers-YYYYMMDD.log, always exits 0).
• Storage: printerobservedqueues, one row per observed queue, owned by the printers plugin. A separate table from the assignment on purpose.

The identity field is hostname. Unlike the computers payload this one is NOT patch-style: queues is required, and the reported set replaces everything previously recorded for that host.

Payload field	Type	Server behaviour
hostname (required)	string	Identity of the report. COMPUTERNAME or its FQDN; matched case-insensitively. Rows are keyed by the NAME, so a bay that reports before its PC record exists still records everything - the report resolves to an asset the moment that record appears. An unknown hostname is a warning, never an error.
queues (required)	array of objects	Every real print queue on the host. This REPLACES the host's previous set. An empty array is a valid report meaning "this bay has no queues" and clears the host's rows. A missing key is rejected (400), because one client bug that dropped the field would otherwise erase the fleet's observed state host by host.
queues[].queuename (required)	string	Windows printer name. A queue with no name is skipped with a warning; a repeated name within one report is dropped with a warning (Windows cannot hold two queues of one name).
queues[].drivername	string	Driver name verbatim, as the INF spells it. Compared against the assignment's driver to detect drift.
queues[].portname	string	Windows port name. Also tried as a match key, since a port created outside the client is usually named after the host address.
queues[].portaddress	string	PrinterHostAddress of a TCP/IP port - an IP or FQDN. The PRIMARY key for matching a queue to a printer asset. Omit it for a non-TCP port (USB, WSD, redirected); such a queue is still worth reporting and matches on name alone.
queues[].isdefault	boolean	True on the one queue that is the user's default. If several arrive true, the first is kept and the rest are cleared with a warning - a bay has exactly one default, and keeping both would leave a seed picking one at random.
queues[].isshared	boolean	True when the queue is shared off this PC. Recorded, not acted on.
observedat	ISO-8601 datetime	Accepted and ignored, as is any other client timestamp. The server stamps observedat at ingest, one value for the whole report, so a bay with a wrong clock cannot report itself fresh or stale.

Response data is the standard collector shape: action is always updated (this endpoint replaces rows and creates no asset - calling an identical re-report noop would hide that the bay is still checking in), assetid is the resolved PC or null, extra.queuecount is how many rows were stored, and warnings carries the soft problems above.

POST /api/collector/printers
X-Endpoint-Credential: sdbe_...
Content-Type: application/json

{
  "hostname": "WORKSTATION01",
  "queues": [
    {
      "queuename": "Bay Label Printer",
      "drivername": "Generic / Text Only",
      "portname": "IP_192.0.2.40",
      "portaddress": "192.0.2.40",
      "isdefault": true
    },
    {
      "queuename": "Office Laser",
      "drivername": "HP Universal Printing PCL 6",
      "portname": "IP_192.0.2.41",
      "portaddress": "192.0.2.41",
      "isdefault": false
    }
  ]
}

The latest report replaces the previous one

This is CURRENT STATE, not history. Each report deletes every row previously recorded for that hostname and writes the reported set in its place, inside the dispatcher's transaction. Nothing accumulates, "what does this bay have" is never a question about time, and a queue removed from a bay disappears from ShopDB on the next cycle without anyone tidying up.

That has one sharp edge, and it belongs to the client: an empty queues array is a legitimate report that wipes the host's rows. A client whose enumeration FAILED must therefore send nothing at all rather than an empty list. Reporting nothing loses one cycle; reporting [] after a WMI hiccup deletes real state and reads as a bay that lost its printers. Report-PrintersToShopDB.ps1 tracks this with an $enumerated flag and exits without posting when both the cmdlet and the WMI fallback failed.

Observed is not assigned

Stated plainly, because it is the whole point of keeping two tables:

A collector report NEVER becomes an assignment. Nothing on this path writes a usesprinter or defaultprinter row. The moment a drifted bay's observed state is treated as correct, enforcement stops meaning anything - a bay that installed the wrong printer would make itself right simply by reporting it.

Observed state becomes assigned state only when a person asks for it, through POST /api/printers/assignments/seed-from-observed/<assetid> (requires printers.edit), after reviewing the comparison. That route refuses rather than guesses: it will not seed from an ambiguous host, will not seed queues that match no printer unless told to, and will not write an empty set.

Matching, and what "unknown" means

Nothing is matched at ingest - the raw observed strings are stored as reported, and resolution to a printer asset happens at READ time, so a printer added to ShopDB tomorrow matches yesterday's report without the bay reporting again.

At read time each queue is matched by PORT ADDRESS first (an IP or FQDN names one device unambiguously), then by queue name. There is no fuzzier fallback: a queue that matches nothing is reported as unknown rather than guessed, because a wrong match seeds a wrong assignment, which is worse than no assignment.

Reading it back

Endpoint	Purpose
GET /api/printers/observed/<hostname>	What the host last reported, every queue classified against the assignment it resolves to (matching, drifted, extra, missing, unknown), with observedat, a summary, and a seedcandidate preview. Read-only. Needs printers.view.
POST /api/printers/assignments/seed-from-observed/<assetid>	The one path from observed to assigned, human-triggered. Needs printers.edit. Normally posted against the MACHINE, so the assignment survives a reimage.

Server-side the printers key resolves per-plugin first: set COLLECTOR_API_KEY_PRINTERS if you scope env keys per collector, or rely on the shared COLLECTOR_API_KEY. A site that has scoped the computers key per-plugin and set no printers key gets 401 here until one of the two is in place. A per-PC credential and a collector.ingest managed token both work here exactly as they do for computers.

8. Porting from classic api.asp

The fleet's classic-ASP reporter posts form fields to api.asp?action=updateCompleteAsset. Map them to the collector JSON as follows:

Classic updateCompleteAsset form field	Collector field
hostname	hostname
machineNo	machinenumber
deviceId	deviceid
pcType	pctype
serialNumber	serialnumber
loggedInUser	loggedinuser
lastBootUpTime / lastBootTime	lastboottime
manufacturer	vendorname
model	modelnumber
osVersion	osname
installedApps	installedsoftware

Not carried over (no current home in the computers schema): warranty fields, DNC config, VNC/WinRM flags as booleans (report them through accessprotocols instead). The classic reporter's full networkInterfaces array maps onto interfaces.