784964c907
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.
What arrives:
endpoints plugin - a ShopDB agent reporting endpoint state, telemetry,
services and interfaces, with a command channel the agent
long-polls. Ingest stays on the generic collector route
(ADR-006); the agent key is a UI-managed single-scope
token whose revocation is an instant kill switch.
computers policies - Configuration and Business Policies (RMM-style), scope
trees validated at write time, actions checked against
the endpoints command allowlist. Feature-gated: every
route 404s while the setting is off, so enabling it can
never half-expose the surface.
asset CSV import - templates, a collector script carrying the site path, and
a validate-then-commit import. A plugin can own a virtual
column plus a post-write row hook, which is how a PC's
machine number becomes a controls link without a
hand-built relationships sheet.
agent/ - the Windows agent and its installer.
Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.
MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).
Review fixes carried on top:
- six policy GETs were jwt_required(optional=True), which authenticates a
caller without checking anything; they require computers.view now, and six
redundant jwt_required decorators are gone
- the agent installer took a default BaseUrl and CollectorKey, so a mistyped
run silently pointed an endpoint at the wrong instance; both are mandatory
and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
- a committing CSV import wrote no audit row
Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.
Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
167 lines
7.8 KiB
Markdown
167 lines
7.8 KiB
Markdown
# ShopDB Endpoint Agent
|
|
|
|
A Windows telemetry agent that reports endpoint state to a ShopDB server's
|
|
`endpoints` plugin. It runs as a Windows service in production and as a console
|
|
app for development and testing.
|
|
|
|
## Projects
|
|
|
|
- `ShopDbEndpointAgent.Core` - contracts, config, logging, collection, and the
|
|
resilient collector HTTP client. No host concerns; unit-testable.
|
|
- `ShopDbEndpointAgent` - the service host (generic host + Windows service
|
|
lifetime). Composes Core through DI. This is the telemetry runtime.
|
|
- `ShopDbEndpointAgent.Tray` - interactive user-session tray app: a system-tray
|
|
icon with "Send Status now" (on-demand report) and the 5-click issue reporter
|
|
(ServiceNow flow is still a scaffold). Separate process from the service,
|
|
because a SYSTEM service cannot draw a tray icon in a user session.
|
|
|
|
## Configuration
|
|
|
|
Runtime config is layered. Non-secret defaults live in `appsettings.json`
|
|
(shipped with the build). The collector key and base URL are provisioned out of
|
|
band in the registry and win over `appsettings.json`, so a distributed build
|
|
never carries the secret and the site can rotate the key without a rebuild.
|
|
|
|
### Registry (production)
|
|
|
|
Create these values under `HKLM\SOFTWARE\GE\ShopDB` (64-bit view):
|
|
|
|
| Value | Type | Meaning |
|
|
|----------------|--------|--------------------------------------------------|
|
|
| `BaseUrl` | String | `https://shopdb.example.net` (server root) |
|
|
| `CollectorKey` | String | The endpoint agent key from Settings -> Endpoints -> Key management |
|
|
|
|
Example (elevated PowerShell):
|
|
|
|
```
|
|
New-Item -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Force | Out-Null
|
|
Set-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Name 'BaseUrl' -Value 'https://shopdb.example.net'
|
|
Set-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Name 'CollectorKey' -Value '<paste the generated key>'
|
|
```
|
|
|
|
### appsettings.json (all environments)
|
|
|
|
| Key | Default | Meaning |
|
|
|-------------------|-------------|------------------------------------------------|
|
|
| `baseurl` | "" | Dev fallback for the server root (registry wins). |
|
|
| `apikey` | placeholder | Dev fallback for the collector key (registry wins). |
|
|
| `cadenceseconds` | 900 | Report interval (15 min). |
|
|
| `jitterseconds` | 60 | Random +/- spread on the interval (anti-herd). |
|
|
| `adoptonfirstrun` | true | Send `adopt=true` on first contact only. |
|
|
| `utilitiespath` | "" | Directory of site PowerShell "agent utilities". |
|
|
| `logpath` | "" | Override the rolling log directory. |
|
|
|
|
For a dev box, set `baseurl` and `apikey` here and skip the registry.
|
|
|
|
## Running
|
|
|
|
Console (dev), one report then exit:
|
|
|
|
```
|
|
dotnet run --project ShopDbEndpointAgent -- --once
|
|
```
|
|
|
|
Console (dev), continuous cadence loop:
|
|
|
|
```
|
|
dotnet run --project ShopDbEndpointAgent -- --console
|
|
```
|
|
|
|
## Install as a Windows service
|
|
|
|
Publish self-contained, then register with `sc.exe` (elevated):
|
|
|
|
```
|
|
dotnet publish ShopDbEndpointAgent -c Release -r win-x64 --self-contained -o C:\ProgramData\ShopDB\EndpointAgent\bin
|
|
sc.exe create "ShopDBEndpointAgent" binPath= "C:\ProgramData\ShopDB\EndpointAgent\bin\shopdb-endpoint-agent.exe" start= auto DisplayName= "ShopDB Endpoint Agent"
|
|
sc.exe description "ShopDBEndpointAgent" "Reports endpoint telemetry to ShopDB."
|
|
sc.exe start "ShopDBEndpointAgent"
|
|
```
|
|
|
|
The service runs as LocalSystem by default, which it needs to read the Security
|
|
event channel and system WMI.
|
|
|
|
## Tray app (system-tray icon)
|
|
|
|
The tray is a per-user process, not a service, so it starts at sign-in rather
|
|
than under `sc.exe`. `install-shopdbagent.ps1` handles it: it copies
|
|
`shopdb-endpoint-tray.exe` next to the agent, registers an HKCU logon autostart
|
|
(`Software\Microsoft\Windows\CurrentVersion\Run\ShopDbEndpointTray`), and
|
|
launches it for the current user. Run the installer from the operator's own
|
|
session so the autostart lands in the right user hive. Pass `-NoTray` for a
|
|
headless, service-only install.
|
|
|
|
Right-click the tray icon for:
|
|
|
|
- `Send Status now` - reports this PC to ShopDB immediately so the monitor page
|
|
refreshes without waiting for the next cadence tick. It shells out to
|
|
`shopdbagent.exe --once`, reusing the exact production collection path; the
|
|
tray holds no collector key. A balloon tip reflects success or failure (on
|
|
failure, see the rolling agent log).
|
|
- `Report an issue...` - the ServiceNow ticket flow (scaffold; submission path
|
|
is an open decision).
|
|
|
|
A desktop shortcut to `shopdb-endpoint-tray.exe` can be added later for an
|
|
explicitly launchable ServiceNow surface.
|
|
|
|
## WebUI command execution model (security context)
|
|
|
|
The WebUI can send an endpoint an allowlisted command (server source of truth:
|
|
`plugins/endpoints/commands.py`; agent handler: `CommandExecutor`). The agent
|
|
service runs as LocalSystem, but not every command runs in that context. The
|
|
launch context is chosen per action so processes surface to the operator while
|
|
privileged operations keep the token they need:
|
|
|
|
- Generic `process_start` (for example DncMain, UDC) launches into the ACTIVE
|
|
CONSOLE USER's session at medium integrity, so the app is visible on the
|
|
operator's desktop. CyberArk/EPM brokers any elevation afterward.
|
|
- Generic `process_stop` / `process_status` run as SYSTEM (kill and probe).
|
|
- `service_start` / `service_stop` / `service_status` run as SYSTEM, because
|
|
service control requires the admin/SYSTEM token.
|
|
- `mtconnect_start` / `mtconnect_stop` / `mtconnect_status` all run as SYSTEM,
|
|
because MTConnect is the "UDC MTConnect Agent" Windows service and service
|
|
control requires the admin/SYSTEM token. `mtconnect_start` runs `install.bat`
|
|
(`Agent.exe install`) to install or repair the service, then ensures it is
|
|
running; `mtconnect_stop` stops the service; `mtconnect_status` queries it.
|
|
`run.bat` (`agent.exe run agent.cfg`) is a foreground TESTING launcher and is
|
|
deliberately NOT used by the agent.
|
|
- `system_restart` runs as SYSTEM and reboots the machine immediately via
|
|
`shutdown.exe /r /t 0` (no grace timer). It is a fixed action with no target,
|
|
gated server-side by the dedicated `endpoints.restart` permission on top of
|
|
`endpoints.command`, and fronted by a WebUI confirmation gate.
|
|
|
|
Do not route MTConnect through the generic `process_start` user-session path or
|
|
service control breaks. MTConnect is intentionally absent from the
|
|
`process_start` launch map for the same reason; the fixed `mtconnect_*` handler
|
|
owns it.
|
|
|
|
Behavior note for redeploys: the MTConnect service-control model is a runtime
|
|
change, so the agent must be rebuilt and redeployed for it to take effect.
|
|
Server-side command/allowlist/RBAC changes are picked up by the container
|
|
rebuild; rebuild and redeploy the agent whenever the runtime launch/control
|
|
logic itself changes.
|
|
|
|
## Diagnostics
|
|
|
|
- Rolling file log: `%ProgramData%\ShopDB\EndpointAgent\logs\agent-YYYYMMDD.log`
|
|
(14-day retention).
|
|
- Windows Event Log: agent runtime failures and unhandled exceptions are written
|
|
to the `Application` log under source `ShopDB Endpoint Agent`. The source is
|
|
created on first elevated run (or by the installer). Filter Event Viewer by
|
|
that source to see agent/system interop failures.
|
|
|
|
## Agent utilities (PowerShell seam)
|
|
|
|
If `utilitiespath` points at a directory, the runtime runs each `*.ps1` there
|
|
per report, parses its JSON stdout, and merges the result into the report. This
|
|
is the seam the future server-side "Agent configuration" settings page uses to
|
|
push site-authored collectors down. Scripts must be code-signed and
|
|
allow-listed before this is enabled in the field (rollout concern).
|
|
|
|
## Logging extensibility
|
|
|
|
All runtime code logs through `Microsoft.Extensions.Logging` (`ILogger`). Adding
|
|
a sink later (a ShopDB HTTP sink, Seq, syslog) is a host-builder change in
|
|
`Program.cs::ConfigureLogging` with no changes to the worker, collector, or
|
|
client.
|