ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix user-context collection, split agent log destinations, widen upload cap

Browse Source 
Mapped drives stopped reporting because the user-context path failed silently
in three separate places, and none of them left a trace to read back.

- Probe: launch via -EncodedCommand instead of staging a temp .ps1 and running
  powershell.exe -File. A missing or quarantined script file fails BEFORE any
  statement runs, so the script's own try/catch never sees it and the only
  symptom is exit -196608.
- Spool: pin to the ShopFloor profile. Resolving it through WTSQueryUserToken
  is privileged, endpoint protection blocks it, and on failure the tray
  (writer) and service (reader) silently disagree about the path.
- Spool: consume after a successful parse, so a stopped tray cannot keep
  re-reporting stale mapped drives as current. Unparseable files are left in
  place for the next cycle.
- Logging: user-session components cannot write to Program Files or to the
  SYSTEM-owned ProgramData log directory, and the file sink degrades to a
  silent no-op when it cannot create its directory. Tray and probe now log to
  C:\Users\ShopFloor\ShopDB\Logs, installs transcript to ProgramData, and the
  sink records a SinkProblem instead of swallowing UnauthorizedAccessException.
- Installer: provision the operator spool/log directories with ShopFloor:Modify
  before the tray first runs, and stop the transcript via trap so a throw
  cannot orphan exactly the failed install worth reading.
- Upload cap 64 -> 128 MB: the self-contained tray publishes at ~69 MB and was
  rejected on SIZE, but the size check returns before the PE is parsed, so the
  operator only ever saw a version-detection failure.

AGENT-CONTRACT.md gains a normative section defining the three security
contexts, their log destinations, the fixed spool path and consume-on-read.

Agent 1.8.0 -> 1.8.1, tray 1.1.0 -> 1.1.1 so the deployed build is
identifiable from a log line.
This commit is contained in:
benjamin.baker
2026-09-05 14:20:07 -04:00
parent 7f84e8ca1e
commit 86985efcf2
23 changed files with 1261 additions and 192 deletions
Download Patch File Download Diff File Expand all files Collapse all files

63
plugins/endpoints/AGENT-CONTRACT.md
View File

@@ -10,8 +10,8 @@ schema changes, update this file in the same change.
- Method/URL: `POST {baseurl}/api/collector/endpoints`
- Auth header: `X-API-Key: <collector key>` (Bearer token also accepted). The
key is a UI-managed, endpoints-only service token minted from Settings ->
Endpoint agent key (section 6). A bootstrap `.env` collector key still works
until the first UI key is generated, then it is switched off (section 6). No
Endpoint agent key (section 7). A bootstrap `.env` collector key still works
until the first UI key is generated, then it is switched off (section 7). No
querystring fallback.
- Body: `application/json`, a single flat object (NOT an array). One POST per
report per endpoint.
@@ -194,10 +194,11 @@ Notes:
rather than a string match against a Windows enum the server would have to
keep in sync. `null` means the drive did not answer: unknown, NOT healthy.
`healthstatus` keeps the human-readable source value alongside it.
- `usercontext` is gathered by a probe the agent runs inside the signed-in
user's session, because mapped drives, user-owned listeners and idle time do
not exist in the agent's own LocalSystem session 0. It is omitted when no one
is signed in, which is a normal unattended state, not a fault.
- `usercontext` is gathered by a probe the TRAY runs inside the signed-in
operator's session, because mapped drives, user-owned listeners and idle time
do not exist in the agent's own LocalSystem session 0. It is omitted when no
one is signed in, which is a normal unattended state, not a fault. The
security-context split that makes this work is normative; see section 5.
- `ports` carries LISTENING sockets only. The full connection table is available
on demand via the `network_connections` command instead, because established
connections churn constantly and would change every report for no diagnostic
@@ -294,7 +295,49 @@ public sealed class CollectorResponse
Serializer settings: `DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull`
so omitted fields are not sent (server upserts only present keys). Send UTC ISO-8601.
## 5. "Build site agent" settings flow (build cadence)
## 5. Security contexts, spool handoff and log destinations (NORMATIVE)
Three processes run as three different principals. Confusing them is what makes
`usercontext` silently empty, so the boundaries are fixed by contract.
| Process | Runs as | May write to |
|---|---|---|
| Service (`shopdbagent.exe`) | LocalSystem, session 0 | `%ProgramData%\ShopDB\EndpointAgent\logs\agent-<date>.log` |
| Tray + probe (`shopdb-endpoint-tray.exe`, `shopdbagent.exe --collect-usercontext`) | `ShopFloor` operator, interactive session | `C:\Users\ShopFloor\ShopDB\` and `C:\Users\ShopFloor\ShopDB\Logs\<datetime>-trayagent.log` |
| Installer (`install-shopdbagent.ps1`) | Elevated admin, one-shot | `%ProgramData%\ShopDB\Logs\<datetime>-shopdbinstalls.log` |
Rules:
1. **The operator cannot write to `C:\Program Files` or to the SYSTEM-owned
`%ProgramData%\ShopDB\EndpointAgent\logs`.** Anything running in the user's
session MUST log under `C:\Users\ShopFloor\ShopDB\Logs`. The file log sink
degrades to a silent no-op when it cannot create its directory, so violating
this does not raise an error - it discards the log line and the failure
becomes invisible.
2. **Only the tray collects user context**, because mapped drives, user-owned
listeners and idle time are per-session and do not exist in session 0. The
service MUST NOT try to descend into the session to collect them.
3. **The tray WRITES the spool; the service READS it and then REMOVES it.**
The service consumes the file after a successful parse, so a tray that has
stopped running cannot keep re-reporting stale mapped drives as current. A
file that fails to parse (a read that raced the write) is left in place.
4. **The spool location is fixed at `C:\Users\ShopFloor\ShopDB\usercontext.json`.**
Every shop-floor endpoint uses the same local `ShopFloor` account. Resolving
it through `WTSQueryUserToken` is a fallback only: that call is privileged,
endpoint protection blocks it, and when it fails the writer and reader
disagree about the location with no error on either side.
5. **Losing user context is never fatal.** If the `ShopFloor` profile is absent
or unwritable, `usercontext` is omitted and telemetry, commands, inventory
and the command channel continue unaffected.
6. **The installer provisions** `C:\Users\ShopFloor\ShopDB\` and its `Logs`
subdirectory with `ShopFloor: Modify` before the tray first runs.
The three log destinations are deliberately separate: an install is one-shot and
elevated, the service log is a rolling runtime log, and the tray log is the only
one written from the operator's session. Merging them makes an install
indistinguishable from runtime behaviour when reading a failure back.
## 6. "Build site agent" settings flow (build cadence)
Goal: an operator in ShopDB Settings produces an instance-locked `.exe` that
knows its server URL and its collector key, needing no manual config on the
@@ -321,7 +364,7 @@ Open item (from DESIGN-DECISIONS-agent.md 1a): whether `adopt` stays a
per-report flag or is promoted to a site setting (strict vs self-adopting
fleet). Default proposed: agent sends `adopt=true` on first run only.
## 6. UI-managed agent key (kill switch)
## 7. UI-managed agent key (kill switch)
The endpoint agent authenticates with a single UI-managed key scoped to ONLY
`collector.ingest.endpoints`, so rotating or revoking it stops all
@@ -346,13 +389,13 @@ The bootstrap `.env` key (`COLLECTOR_API_KEY[_ENDPOINTS]`) remains only as a
first-run path before a UI key exists; it cannot be the runtime kill switch
because container env is injected at boot and cannot be rotated live.
## 7. Command channel (WebUI -> agent)
## 8. Command channel (WebUI -> agent)
The WebUI can send an endpoint an ALLOWLISTED command (never an arbitrary
command string). Delivery is agent-initiated long-poll, so shop-floor PCs open
no inbound port: the agent makes an outbound request the server holds open until
a command is queued, runs it locally, and posts the result back. All three
routes reuse the same endpoints collector key (section 6) via `X-API-Key`, so
routes reuse the same endpoints collector key (section 7) via `X-API-Key`, so
the command channel shares the kill switch and adds no second secret.
Allowlisted actions (server source of truth: `plugins/endpoints/commands.py`):

16
plugins/endpoints/api/agentupload.py
View File

@@ -26,10 +26,18 @@ from shopdb.api import (success_response, error_response, ErrorCodes, AuditLog,
from .routes import endpoints_bp
from . import agentstore
# Hard cap on an uploaded agent binary. The real agent is a few MB; this is a
# generous ceiling that still refuses an operator who points at a giant wrong
# file before it is buffered fully into memory-staged storage.
_MAX_UPLOAD_BYTES = 64 * 1024 * 1024
# Hard cap on an uploaded agent binary. This is a generous ceiling that still
# refuses an operator who points at a giant wrong file before it is buffered
# fully into memory-staged storage.
#
# 128 MB, not 64: the self-contained single-file tray publishes at ~69 MB
# (it carries the whole .NET runtime plus WinForms), which the old 64 MB cap
# rejected outright. The upload failed on SIZE, but the operator saw only a
# version-detection failure, because the size check returns before the PE is
# ever parsed. Config.MAX_CONTENT_LENGTH must stay above this - it bounds the
# whole multipart envelope, so a cap equal to it would reject a legal file on
# the boundary bytes alone.
_MAX_UPLOAD_BYTES = 128 * 1024 * 1024
@endpoints_bp.route('/siteagent/binaries/<binkey>', methods=['GET'])

131
plugins/endpoints/api/routes.py
View File

@@ -1413,6 +1413,137 @@ def panel_interfaces(asset_id: int):
return success_response(rows)
# Columns for the two cards carved out of the old hand-written Network card on
# the Details tab. That card listed current addresses with a Primary badge and
# nothing else, which undersold what the rows actually hold.
_IPHISTORY_COLUMNS = [
{'key': 'ipaddress', 'label': 'IP address'},
{'key': 'macaddress', 'label': 'MAC'},
{'key': 'status', 'label': 'Status'},
{'key': 'firstseen', 'label': 'First seen'},
{'key': 'lastchanged', 'label': 'Last changed'},
]
_IPCONFIG_COLUMNS = [
{'key': 'ipaddress', 'label': 'IP address'},
{'key': 'subnetmask', 'label': 'Subnet mask'},
{'key': 'gateway', 'label': 'Gateway'},
{'key': 'dns1', 'label': 'DNS 1'},
{'key': 'dns2', 'label': 'DNS 2'},
{'key': 'assignment', 'label': 'Assignment'},
]
def _isostamp(value):
"""ISO-8601 with a Z suffix, matching every other timestamp this file emits.
Formatting is left to the client so it can render in the viewer's locale
and timezone; returning a preformatted string here would pin every reader
to the server's.
"""
return value.isoformat() + 'Z' if value else None
def _communication_rows(asset_id: int):
"""Every Communication row for an asset, newest change first.
These rows are an ACCIDENTAL history rather than a designed one.
_record_interfaces in the computers plugin matches an incoming NIC on MAC
then on IP and inserts when neither matches, and it never deletes - so an
address the machine has not held for a year is still sitting here next to
the current one. That is genuinely useful (it answers "what was this box on
before?"), so it is surfaced as history rather than cleaned away.
"""
from shopdb.api import Communication
# MySQL has no NULLS LAST, so sort on an explicit "is null" key first.
# Rows that never got a modifieddate sink to the bottom on every dialect.
return (Communication.query
.filter_by(assetid=asset_id)
.order_by(Communication.modifieddate.is_(None),
Communication.modifieddate.desc(),
Communication.communicationid.desc())
.all())
def _iphistory_status(comm):
"""What this row IS, in the vocabulary somebody troubleshooting would use.
Primary is the address the collector last picked as this box's main one.
APIPA is called out by name because a 169.254.x.x address is not an address
the machine was given - it is what Windows assigns when DHCP failed, so its
presence in the list is a record of a past network fault, not a past
subnet.
"""
address = (comm.ipaddress or '').strip()
if address.startswith('169.254.'):
return 'APIPA (no DHCP)'
if comm.isprimary:
return 'Primary'
if comm.ismachinenetwork:
return 'Machine network'
return 'Previous'
@endpoints_bp.route('/asset/<int:asset_id>/iphistory', methods=['GET'])
@require_permission('endpoints.view')
def panel_iphistory(asset_id: int):
"""Table panel: every address this asset has held, newest change first.
Replaces the hand-written Network card that used to sit on the Details tab
and show only the current addresses. Dates come from the row's own
lifecycle columns, so they are labelled for what they are: 'Last changed'
is when a field on the row last moved, NOT when the address was last seen.
An address the machine still holds and reports unchanged does not update
modifieddate, so calling that column 'Last seen' would be a lie the data
cannot support.
"""
rows = []
for comm in _communication_rows(asset_id):
address = (comm.ipaddress or '').strip()
if not address:
continue
rows.append({
'ipaddress': address,
'macaddress': (comm.macaddress or '').strip() or '-',
'status': _iphistory_status(comm),
'firstseen': _isostamp(comm.createddate),
'lastchanged': _isostamp(comm.modifieddate),
})
return success_response(rows)
@endpoints_bp.route('/asset/<int:asset_id>/ipconfig', methods=['GET'])
@require_permission('endpoints.view')
def panel_ipconfig(asset_id: int):
"""Table panel: subnet, gateway and DNS per recorded address.
Split from IP history rather than folded into it: history answers "what has
this box been", configuration answers "how is this address set up", and one
table trying to do both would be eleven columns wide. Rows with no
configuration recorded are dropped - an address with no subnet, gateway or
DNS contributes nothing here and already appears in the history card.
"""
rows = []
for comm in _communication_rows(asset_id):
address = (comm.ipaddress or '').strip()
subnet = (comm.subnetmask or '').strip()
gateway = (comm.gateway or '').strip()
dns1 = (comm.dns1 or '').strip()
dns2 = (comm.dns2 or '').strip()
if not address or not (subnet or gateway or dns1 or dns2):
continue
rows.append({
'ipaddress': address,
'subnetmask': subnet or '-',
'gateway': gateway or '-',
'dns1': dns1 or '-',
'dns2': dns2 or '-',
'assignment': 'DHCP' if comm.isdhcp else 'Static',
})
return success_response(rows)
@endpoints_bp.route('/asset/<int:asset_id>/applications', methods=['GET'])
@require_permission('endpoints.view')
def panel_applications(asset_id: int):

18
plugins/endpoints/frontend/views/AgentKeySettings.vue
View File

@@ -319,9 +319,9 @@
<div
v-for="binary in binaries"
:key="binary.key"
class="binary-block"
class="card binary-block"
>
<h4>{{ binary.label }}</h4>
<h4 class="binary-title">{{ binary.label }}</h4>
<p class="field-hint">
Upload {{ binary.name }} updated EXE. Uploading here will update the
binaries available for your "Build Site Agent" install package.
@@ -910,10 +910,18 @@ function editConfiguration() {
.settings-section > h3 { margin-bottom: 0.25rem; }
.info-list { margin: 0.25rem 0 0; padding-left: 1.25rem; }
.info-list li { margin-bottom: 0.5rem; }
/* Each agent is its own nested card inside the collapsible, so the two
binaries read as distinct units instead of one continuous wall of
controls separated by a hairline rule. */
.binary-block {
margin-top: 1.25rem;
padding-top: 1rem;
border-top: 1px solid var(--color-border, #333);
margin-top: 1rem;
padding: 1rem;
}
.binary-block:first-of-type { margin-top: 0.75rem; }
.binary-title {
margin: 0 0 0.5rem;
padding-bottom: 0.5rem;
border-bottom: 1px solid var(--color-border, #333);
}
.binary-upload {
display: flex;

61
plugins/endpoints/plugin.py
View File

@@ -507,7 +507,66 @@ class EndpointsPlugin(BasePlugin):
],
'empty': 'No network interfaces reported.',
'position': 16,
# adapter state is watched, not something you act on
# adapter state is watched, not something you act on.
# Declared in monitors but RENDERED at the top of the Details
# tab (PCDetail.vue lifts it by id and excludes it here): which
# adapter is up and what address it holds is asked at the same
# moment as "what is this box".
'group': 'monitors',
},
{
# What this box has been on the network, carved out of the
# hand-written Network card that used to sit on Details and
# show only current addresses with a Primary badge.
#
# The history is ACCIDENTAL: _record_interfaces matches on MAC
# then IP and never deletes, so old addresses accumulate. That
# turned out to be worth reading rather than worth cleaning
# away, so it is presented as what it already is.
'id': 'endpoints-iphistory',
'title': 'IP History',
'assettypes': ['computer'],
'endpoint': '/api/endpoints/asset/{assetid}/iphistory',
'render': 'table',
'collapsible': True,
'columns': [
{'key': 'ipaddress', 'label': 'IP address', 'mono': True},
{'key': 'macaddress', 'label': 'MAC', 'mono': True},
{'key': 'status', 'label': 'Status'},
{'key': 'firstseen', 'label': 'First seen',
'format': 'date'},
# NOT "last seen": modifieddate only moves when a field on
# the row changes, so an address still held and reported
# unchanged keeps an old stamp here.
{'key': 'lastchanged', 'label': 'Last changed',
'format': 'date'},
],
'empty': 'No addresses on record for this endpoint.',
'position': 16.5,
'group': 'monitors',
},
{
# The other half of the old Network card: how each address is
# configured. Split from history because one table carrying
# both would be eleven columns wide, and the two answer
# different questions - what has this box been, versus how is
# this address set up.
'id': 'endpoints-ipconfig',
'title': 'IP Configuration',
'assettypes': ['computer'],
'endpoint': '/api/endpoints/asset/{assetid}/ipconfig',
'render': 'table',
'collapsible': True,
'columns': [
{'key': 'ipaddress', 'label': 'IP address', 'mono': True},
{'key': 'subnetmask', 'label': 'Subnet mask', 'mono': True},
{'key': 'gateway', 'label': 'Gateway', 'mono': True},
{'key': 'dns1', 'label': 'DNS 1', 'mono': True},
{'key': 'dns2', 'label': 'DNS 2', 'mono': True},
{'key': 'assignment', 'label': 'Assignment'},
],
'empty': 'No addressing configuration on record.',
'position': 16.6,
'group': 'monitors',
},
{