ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import

Browse Source 
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.

What arrives:

  endpoints plugin  - a ShopDB agent reporting endpoint state, telemetry,
                      services and interfaces, with a command channel the agent
                      long-polls. Ingest stays on the generic collector route
                      (ADR-006); the agent key is a UI-managed single-scope
                      token whose revocation is an instant kill switch.
  computers policies - Configuration and Business Policies (RMM-style), scope
                      trees validated at write time, actions checked against
                      the endpoints command allowlist. Feature-gated: every
                      route 404s while the setting is off, so enabling it can
                      never half-expose the surface.
  asset CSV import  - templates, a collector script carrying the site path, and
                      a validate-then-commit import. A plugin can own a virtual
                      column plus a post-write row hook, which is how a PC's
                      machine number becomes a controls link without a
                      hand-built relationships sheet.
  agent/            - the Windows agent and its installer.

Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.

MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).

Review fixes carried on top:

  - six policy GETs were jwt_required(optional=True), which authenticates a
    caller without checking anything; they require computers.view now, and six
    redundant jwt_required decorators are gone
  - the agent installer took a default BaseUrl and CollectorKey, so a mistyped
    run silently pointed an endpoint at the wrong instance; both are mandatory
    and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
  - a committing CSV import wrote no audit row

Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.

Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
This commit is contained in:
cproudlock
2026-08-28 16:17:17 -04:00
parent 93ca7587ce
commit 784964c907
149 changed files with 17545 additions and 627 deletions
Download Patch File Download Diff File Expand all files Collapse all files

274
plugins/endpoints/AGENT-CONTRACT.md Normal file
View File

@@ -0,0 +1,274 @@
# Endpoint agent contract (C# build reference)
Companion to DESIGN-DECISIONS-agent.md. This is the concrete wire contract the
future C# agent targets, derived from `EndpointsPlugin.get_collector_schema()`
and the generic collector dispatcher in `shopdb/core/api/collector.py`. When the
schema changes, update this file in the same change.
## 1. Transport
- Method/URL: `POST {baseurl}/api/collector/endpoints`
- Auth header: `X-API-Key: <collector key>` (Bearer token also accepted). The
key is a UI-managed, endpoints-only service token minted from Settings ->
Endpoint agent key (section 6). A bootstrap `.env` collector key still works
until the first UI key is generated, then it is switched off (section 6). No
querystring fallback.
- Body: `application/json`, a single flat object (NOT an array). One POST per
report per endpoint.
- Identity field: `hostname` (required, case-insensitive).
## 2. Request payload
Only `hostname` is required. Every other field is optional; the server upserts
only the keys that are present, so a lightweight report can omit inventory
arrays. `adopt` is first-contact only.
```jsonc
{
"hostname": "SHOP-PC-014", // required, resolved to computers.hostname
"adopt": true, // first run ONLY; delegates PC creation to computers plugin
"loggedinuser": "j.smith",
"serialnumber": "5CD1234ABC", // adoption subset (forwarded to computers)
"pctype": "CNC Controller", // adoption subset
"machinenumber": "12345", // adoption subset; "" or "9999" == none
"agentversion": "1.0.0",
"lastboottime": "2025-01-14T06:12:00Z", // ISO-8601 UTC
"uptimeseconds": 86400,
"idleseconds": 120,
"cpupercent": 12.5,
"memorypercent": 47.0,
"services": [ /* opaque array, stored as JSON */ ],
"ports": [ /* opaque array */ ],
"interfaces": [
{
"alias": "Intel(R) Ethernet Connection", // adapter description
"ipv4": "10.20.30.40", // "" if none
"mac": "AA:BB:CC:DD:EE:FF",
"gateway": "10.20.30.1", // "" for the static machine-IPS NIC
"subnet": "255.255.255.0",
"dhcp": false
}
],
"scheduledtasks": [ /* opaque array */ ],
"events": [
{
"eventid": 6008, // required per item; non-catalog IDs are dropped, not errored
"eventchannel": "System",
"eventsource": "EventLog",
"eventtime": "2025-01-14T06:11:58Z",
"eventmessage": "The previous system shutdown was unexpected.",
"fields": { "any": "extra" } // opaque object, stored as JSON
}
]
}
```
Notes:
- `adopt=true` is sent ONLY by the installer's first run. Steady-state reports
omit it (or send `false`). After adoption, an unknown hostname is a hard
validation error again (DESIGN-DECISIONS-agent.md 1a).
- The adoption subset (`serialnumber`, `pctype`, `machinenumber`, plus
`loggedinuser`) is forwarded to the `computers` collector so the newly created
PC is not a bare shell. Send them on first contact.
- `services`/`ports`/`interfaces`/`scheduledtasks` are stored verbatim as JSON;
the server does not validate their inner shape yet. Keep them stable so the UI
can render them later. `interfaces` reports EVERY hardware NIC (not just the
primary) so the gatewayless static NIC that links a PC to its machine's IPS is
preserved (DESIGN-DECISIONS-agent.md 9.3); each entry is
`{alias, ipv4, mac, gateway, subnet, dhcp}`.
- `machinenumber` is discovered on-box by the agent from the eDNC registry
(`HKLM\SOFTWARE\[WOW6432Node\]GE Aircraft Engines\DNC\General:MachineNo`) or
the UDC `udc_settings_*.json` (`GeneralSettings.MachineNumber`), in that order
(DESIGN-DECISIONS-agent.md 9.4/9.5). `""` or `"9999"` means none.
## 3. Response
Success (HTTP 200), ADR-006 envelope:
```jsonc
{
"status": "ok",
"action": "created", // "created" first time, "updated" thereafter
"assetid": 4821,
"identityvalue": "SHOP-PC-014",
"warnings": [
"adopted new computer for hostname: 'SHOP-PC-014'" // present only on adoption
]
}
```
Validation failure (HTTP 400) - e.g. unknown hostname with `adopt` absent:
```jsonc
{ "status": "error", "error": { "code": "VALIDATION_ERROR",
"message": "could not resolve hostname to a computer: 'SHOP-PC-014'" } }
```
Agent handling: treat non-200 as retryable except a `VALIDATION_ERROR` that
names an unresolved hostname on a NON-first-contact report - that means the PC
was removed server-side; surface it, do not spin.
## 4. C# DTOs
Match the JSON keys EXACTLY (lowercase, no underscores) to satisfy the naming
rule; use attributes so C# properties can stay PascalCase without a camelCase
serializer changing the wire keys.
```csharp
using System.Text.Json.Serialization;
public sealed class EndpointReport
{
[JsonPropertyName("hostname")] public string Hostname { get; set; } = "";
[JsonPropertyName("adopt")] public bool? Adopt { get; set; }
[JsonPropertyName("loggedinuser")] public string? LoggedInUser { get; set; }
[JsonPropertyName("serialnumber")] public string? SerialNumber { get; set; }
[JsonPropertyName("pctype")] public string? PcType { get; set; }
[JsonPropertyName("machinenumber")] public string? MachineNumber { get; set; }
[JsonPropertyName("agentversion")] public string? AgentVersion { get; set; }
[JsonPropertyName("lastboottime")] public DateTimeOffset? LastBootTime { get; set; }
[JsonPropertyName("uptimeseconds")] public long? UptimeSeconds { get; set; }
[JsonPropertyName("idleseconds")] public long? IdleSeconds { get; set; }
[JsonPropertyName("cpupercent")] public double? CpuPercent { get; set; }
[JsonPropertyName("memorypercent")] public double? MemoryPercent { get; set; }
[JsonPropertyName("services")] public object[]? Services { get; set; }
[JsonPropertyName("ports")] public object[]? Ports { get; set; }
[JsonPropertyName("interfaces")] public object[]? Interfaces { get; set; }
[JsonPropertyName("scheduledtasks")]public object[]? ScheduledTasks { get; set; }
[JsonPropertyName("events")] public EndpointEventRecord[]? Events { get; set; }
}
public sealed class EndpointEventRecord
{
[JsonPropertyName("eventid")] public int EventId { get; set; }
[JsonPropertyName("eventchannel")] public string? EventChannel { get; set; }
[JsonPropertyName("eventsource")] public string? EventSource { get; set; }
[JsonPropertyName("eventtime")] public DateTimeOffset? EventTime { get; set; }
[JsonPropertyName("eventmessage")] public string? EventMessage { get; set; }
[JsonPropertyName("fields")] public Dictionary<string, object>? Fields { get; set; }
}
public sealed class CollectorResponse
{
[JsonPropertyName("status")] public string Status { get; set; } = "";
[JsonPropertyName("action")] public string? Action { get; set; }
[JsonPropertyName("assetid")] public int? AssetId { get; set; }
[JsonPropertyName("identityvalue")] public string? IdentityValue { get; set; }
[JsonPropertyName("warnings")] public string[]? Warnings { get; set; }
}
```
Serializer settings: `DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull`
so omitted fields are not sent (server upserts only present keys). Send UTC ISO-8601.
## 5. "Build site agent" settings flow (build cadence)
Goal: an operator in ShopDB Settings produces an instance-locked `.exe` that
knows its server URL and its collector key, needing no manual config on the
client.
1. Settings -> "Build site agent": operator picks/creates a collector-scoped
managed token (the same token accepted by `X-API-Key` on
`/api/collector/endpoints`). This binds the build to one ShopDB instance.
2. Server stamps a small config (base URL + token + reporting cadence) into the
agent build artifact. Instance-locked = the token only works against that
server, and the server can revoke it to kill a leaked build.
3. First run on a client:
- Collect the ShopDB PowerShell asset-collector data set (minus the CSV
emit): hostname, logged-in user, serial number, pc type, machine number,
boot time, plus the curated event tail.
- POST once with `adopt=true`. The endpoints plugin delegates PC creation to
the `computers` collector, then writes the first `EndpointState` in the
same commit. The endpoint appears in the Fleet UI immediately - no second
call, no separate enrollment endpoint.
4. Subsequent runs POST on the configured cadence with `adopt` omitted; strict
hostname resolution applies from then on.
Open item (from DESIGN-DECISIONS-agent.md 1a): whether `adopt` stays a
per-report flag or is promoted to a site setting (strict vs self-adopting
fleet). Default proposed: agent sends `adopt=true` on first run only.
## 6. UI-managed agent key (kill switch)
The endpoint agent authenticates with a single UI-managed key scoped to ONLY
`collector.ingest.endpoints`, so rotating or revoking it stops all
endpoint-agent traffic without affecting the computers or geenforce collectors.
- Settings -> Endpoint agent key (admin only) generates the key. The full secret
is shown ONCE at generation and stored only as a hash afterwards; configure it
as the agent `X-API-Key`.
- Generating a key is a hard cutover: any previous endpoint key is revoked in the
same request, and the bootstrap `.env` fallback for endpoints is switched off
(`plugin.endpoints.envkeyfallback` -> false), so the managed key becomes the
sole credential.
- Regenerate to rotate (old key dies immediately). Revoke to stop all
endpoint-agent reporting until a new key is generated - the kill switch for a
leaked agent.
- Backend: `GET/POST /api/endpoints/agentkey`, `POST
/api/endpoints/agentkey/regenerate`, `DELETE /api/endpoints/agentkey`. These
use the `mint_service_token`/`list_service_tokens`/`revoke_service_tokens`
contract helpers (shopdb.api, contract 0.22.0).
The bootstrap `.env` key (`COLLECTOR_API_KEY[_ENDPOINTS]`) remains only as a
first-run path before a UI key exists; it cannot be the runtime kill switch
because container env is injected at boot and cannot be rotated live.
## 7. Command channel (WebUI -> agent)
The WebUI can send an endpoint an ALLOWLISTED command (never an arbitrary
command string). Delivery is agent-initiated long-poll, so shop-floor PCs open
no inbound port: the agent makes an outbound request the server holds open until
a command is queued, runs it locally, and posts the result back. All three
routes reuse the same endpoints collector key (section 6) via `X-API-Key`, so
the command channel shares the kill switch and adds no second secret.
Allowlisted actions (server source of truth: `plugins/endpoints/commands.py`):
- `mtconnect_start` - run the MTConnect UDC agent start script
(`C:\ProgramData\UDC\MTConnect_UDC\Agent\run.bat`).
- `mtconnect_stop` - stop the running MTConnect UDC agent process.
- `mtconnect_status` - report whether the MTConnect UDC agent is running.
### 7.1 Claim the next command (long-poll)
`GET {baseurl}/api/endpoints/commands/next?hostname=<hostname>`
- Auth: `X-API-Key` (endpoints collector key).
- The server holds the request open up to ~25s. If a command is queued it is
atomically claimed (marked running) and returned; only one poll ever claims a
given command.
- Command queued (HTTP 200):
```jsonc
{ "status": "ok", "data": { "endpointcommandid": 91, "action": "mtconnect_start" } }
```
- Nothing queued within the hold window: HTTP 204 (no body). Re-poll.
### 7.2 Report a command result
`POST {baseurl}/api/endpoints/commands/<endpointcommandid>/result`
- Auth: `X-API-Key` (endpoints collector key).
- Body:
```jsonc
{
"succeeded": true, // required; false marks the command failed
"message": "MTConnect started", // short summary, optional (<= 2000 chars)
"mtconnectrunning": true, // optional; observed run state after the action
"detail": { "any": "extra" } // optional structured detail, stored as JSON
}
```
- When `mtconnectrunning` is present the server caches it on the endpoint's
`EndpointState` (`mtconnectrunning` + `mtconnectcheckedat`) so the WebUI toggle
shows a real state between polls. Every `mtconnect_*` action SHOULD probe and
report the resulting run state so the toggle never lies.
### 7.3 Enqueue (WebUI side, for reference)
`POST /api/endpoints/<assetid>/commands` with `{ "action": "mtconnect_start" }`,
gated by user JWT + `endpoints.command`. Only one command may be in flight per
endpoint (a second enqueue returns HTTP 409 until the first finalizes).

443
plugins/endpoints/DESIGN-DECISIONS-agent.md Normal file
View File

@@ -0,0 +1,443 @@
# Agent Design Decisions (running preference list)
Server-side decisions made while building the `endpoints` plugin that the
future C# agent (and the ServiceNow ticketing surface) must honor. This is a
living document: add to it as the agent build raises questions. ASCII only, no
em-dashes, no emojis, per repo style.
## Scope of this document
The `endpoints` plugin (server) is the authoritative contract. The C# agent is
a separate deliverable in its own solution/repo. Anything the agent sends must
match what this plugin ingests; when the two disagree, this plugin wins and the
agent changes.
## 1. Transport and identity
- Agent posts one JSON object per host to `POST /api/collector/endpoints`
(the generic ADR-006 collector route, auto-registered by the core loader).
- Auth is a collector-scoped credential in the request HEADER, not a user JWT
and not a querystring. Server reads `COLLECTOR_API_KEY_ENDPOINTS` first, then
falls back to `COLLECTOR_API_KEY`. Provision the key the same way the
computers reporter does: out of band, e.g. `HKLM:\SOFTWARE\GE\ShopDB
CollectorKey`. The key is never baked into a distributed script/binary.
- Identity field is `hostname` (case-insensitive). The server resolves it to an
existing computer asset via the `computers` plugin. In STEADY STATE an unknown
hostname is REJECTED with a validation error, never auto-created by this
plugin. Decision: the fleet is provisioned in ShopDB first (imports/Azure
device sync) OR adopted at first contact via the delegation shim in section
1a; the `endpoints` plugin itself does not get to invent assets.
## 1a. First-contact adoption (delegation shim, not a design reversal)
The agent installer's first run reaches a PC that may not exist in ShopDB yet.
Rather than letting `endpoints` create assets (which would break ADR-001 and the
section 1 rule that this plugin never invents assets), first contact DELEGATES
adoption to the `computers` collector, which is already the sole authoritative
creator of computer Assets and PC rows.
Decision:
- The `endpoints` plugin NEVER constructs an `Asset` or a `Computer` row itself.
Section 1 stands. This is a clarification, not a reversal.
- When `apply_collector_payload` cannot resolve the hostname AND the payload
opts in to adoption (see the `adopt` flag below), the plugin calls the
`computers` plugin's `apply_collector_payload` with the PC-identifying subset
of the payload, flushes, then re-resolves the hostname and proceeds with
telemetry. All of this runs inside the ONE transaction the generic collector
dispatcher owns (`shopdb/core/api/collector.py`): a `ValueError` still rolls
the whole thing back, so a half-adopted PC can never be committed.
- Steady-state reports send `adopt=false` (or omit it once a site chooses strict
mode); only the installer's first run sends `adopt=true`. After adoption the
strict "must already resolve" invariant holds for every subsequent report.
First-contact payload (the PC-identifying subset the shim forwards to the
`computers` collector - every field except `hostname` is optional but recommended
so the adopted PC is not a bare shell):
- `hostname` (required), `loggedinuser`, `serialnumber`, `pctype`,
`machinenumber` (only when the PC is bound to a machine; `''`/`'9999'` are
treated as none by the computers collector).
Net effect for the agent:
- ONE POST to `/api/collector/endpoints` with `adopt=true` on first run creates
the Asset + PC (via the shim) and writes the first `EndpointState` in the same
commit, so the endpoint appears in the UI immediately (no second call, no
separate enrollment endpoint).
- OPEN: whether to keep `adopt` per-report or promote it to a site setting
(strict vs. self-adopting fleet). Default proposed: adoption allowed, agent
sends `adopt=true` only on first run.
## 2. Report shape (current server contract)
Top-level payload fields the server currently reads (see
`plugins/endpoints/plugin.py` `get_collector_schema`):
- `hostname` (required), `loggedinuser`, `agentversion`
- Metrics: `lastboottime` (ISO-8601), `uptimeseconds`, `idleseconds`,
`cpupercent`, `memorypercent`
- Inventory arrays (stored as latest-snapshot JSON, not history):
`services`, `ports`, `interfaces`, `scheduledtasks`
- `events`: array of curated Windows Event Log records (see section 4)
Decision: inventory and metrics are a SNAPSHOT overwrite (one `EndpointState`
row per asset), not time-series. The agent sends the full current list each
cycle; the server does not diff or version them. If per-sample history is later
required, it goes in a new append-only table and the contract stays additive.
## 3. Reporting rhythm
- Default report interval: 15 minutes (`report_interval_minutes` in the plugin
manifest settings; a site setting can override). Agent design should make the
interval configurable and jittered to avoid a fleet-wide thundering herd.
- There is NO separate heartbeat. `lastreportedat` is stamped on every report;
"quiet" endpoints are those past the quiet-report window (default 24h, site
setting `endpoints_quietreporthours`). Agent does not need a distinct ping.
## 4. Curated Windows Event catalog (LOCKED to the server list)
The server stores ONLY these IDs; anything else in `events` is ignored (not an
error). Source of truth: `CURATED_EVENT_IDS` in `plugins/endpoints/plugin.py`.
| Event ID | Channel | Meaning |
|----------|--------------|--------------------------------------------|
| 4624 | Security | Successful logon |
| 4625 | Security | Failed logon |
| 4634 | Security | Logoff |
| 4672 | Security | Special privileges assigned to new logon |
| 4697 | Security | Service installed |
| 4698 | Security | Scheduled task created |
| 41 | System | Kernel-Power, unexpected shutdown |
| 4618 | Security | Monitored security event pattern |
| 55 | System | NTFS file system corruption |
| 7011 | System | Service Control Manager timeout |
| 10016 | System | DCOM permission error |
| 1001 | Application | Application error / bugcheck report |
- Per event, the agent sends: `eventid` (required), `eventchannel`,
`eventsource`, `eventtime` (ISO-8601, event time not report time),
`eventmessage`, and `fields` (an object of extracted structured values, e.g.
target account on a 4625). `fields` is stored as `payloadjson`.
- OPEN QUESTION for the agent build: which IDs need structured `fields` parsing
vs. raw message pass-through. Start with pass-through message; add structured
parsing per ID as alerting rules demand it. The teammate flagged event parsing
as the hard part; keep the catalog fixed and small until baselined.
- Scheduled-task history being disabled on some hosts means 4698 and task
"last run" may be sparse; the agent should read current task state directly,
not rely on event history alone.
## 5. Monitored services/processes
Known targets to watch (non-exhaustive, confirm during agent build): UDC, eDnc,
Intune-related services. Server stores whatever list the agent sends in
`servicesjson` without a fixed schema yet. Decision deferred: once the watched
set is final, consider promoting it to a server-known enum for alerting.
## 6. ServiceNow ticketing surface (future, creds confirmed available)
Not built server-side yet. Recorded here so the agent and a future server
endpoint agree:
- Interactive C# piece (system-tray/desktop launcher) owns the 5-click-max flow:
network precheck -> issue dropdown -> optional screenshot -> submit.
- Fixed dropdown reasons (from the source conversation): Can't logon to CSF;
Data collections down; Can't download program; Machine communication failing;
Network drives down; Other (free text).
- Screenshot saved to local `Pictures\Screenshots` before submission.
- OPEN: whether ticket submission goes direct to the ServiceNow API from the
agent, or is proxied through a ShopDB server endpoint (preferred: proxy, so
ServiceNow creds live server-side and the agent carries only its collector
key). Decide before wiring the integration.
## 7. Runtime split (from the planning conversation)
- Interactive bits (tray launcher, ticketing, screenshot): C# / .NET, likely a
Windows service + tray app.
- Collections MAY be locally hosted PowerShell scripts the service invokes,
where that is simpler than reimplementing WMI/CIM queries in C#.
- Code signing and Intune deployment are agent-side concerns, out of scope for
the server plugin but must be planned before fleet rollout.
### 7a. Tray deployment and on-demand "Send Status" (built)
The tray icon is a PER-USER process, not a service. A Windows service runs in
session 0 and cannot draw a tray icon in the operator's session, so the tray
ships as a second executable (`shopdb-endpoint-tray.exe`) that the installer
copies next to `shopdbagent.exe`, registers for logon autostart via the HKCU
Run key (`ShopDbEndpointTray`), and launches for the current user. Because the
Run key is per-user, the installer must be run from the operator's own session;
`-NoTray` yields a headless, service-only install.
DECISION - "Send Status now" reuses `--once`, it does not reimplement collection.
The tray's Send Status action shells out to `shopdbagent.exe --once`, the same
production collect-and-report path the service uses on its cadence. Consequences:
the tray holds NO collector key (the agent exe reads it from registry/appsettings
itself), there is one collection code path to maintain, and an operator can force
the ShopDB monitor page to refresh immediately instead of waiting for the next
cadence tick. The tray surfaces the agent's exit code as a balloon tip (0 =
sent, non-zero = check the rolling agent log). Note: `--once` honors
`adoptonfirstrun`, so an on-demand send can carry `adopt=true`; the server dedups
on hostname, so this is safe.
The ServiceNow ticket action stays a scaffold (section 6). A future desktop
shortcut to the same tray exe is the intended explicitly-launchable entry point
for that workflow; nothing about the Send Status wiring blocks it.
## 8. Web UI display (server-side, built)
The collected data surfaces through ShopDB's existing declaration-driven UI
contract (ADR-010). The plugin declares WHAT to show; generic core components
render it. No plugin-specific Vue components were written. Audience is DT
support / IT, NOT the shop-floor operator (the operator's surface is the
agent-side ticket tool).
Surfaces implemented:
- Per-PC asset panels (computer assets only), on the PC detail page:
- Endpoint status (keyvalue): logged-in user, uptime, idle, last boot,
CPU/memory %, agent version, last reported. Feeds
`GET /api/endpoints/asset/{assetid}/status`.
- Services and processes (list + badge): monitored services with a green/red
run-state badge. Feeds `.../services`.
- Recent events (list + severity badge): last 10 curated events. Feeds
`.../recentevents`.
- Dashboard exception cards (hide when empty):
- Endpoints gone quiet (warning): `.../dashboard/quiet`.
- Endpoints with crashes or shutdowns (critical): `.../dashboard/stability`,
bounded to a 48h window and the critical event catalog.
DECISION - freshness and cost. There is NO live streaming and NO polling. The
freshness ceiling is the agent report cadence (~15 min), so a websocket/poll
buys nothing but load. Data is fetched on page/dashboard load only, matching the
existing computers-quiet card. Asset panels are single-asset indexed reads
(negligible). Dashboard cards run once per dashboard load, cap at 50 rows, and
the stability card is time-bounded so it does not grow with total history. If
fleet-wide aggregation ever gets heavy, memoize dashboard aggregates with the
core `cache` helper for ~60s - documented lever, not built yet.
Agent implication: the agent does not drive any UI refresh. Its only job is to
report on its rhythm; the UI reflects whatever the last report wrote. The
`services` payload should send a per-service run STATE (e.g. running/stopped) so
the badge can color it; a missing/unknown state renders as a gray Unknown badge.
## 9. Shop-floor collection knowledge base (reference, NOT a schema change)
This section is a body of reference, not a change request. Nothing here alters
the shape of ShopDB as it stands. It records WHAT a shop-floor endpoint really
is, WHICH signals identify it, and WHY each signal matters to the enterprise, so
that whoever grows the collector understands the stakes behind each field before
touching the wire contract. The source material is the site's own extant
tooling: `scripts/site_imports/assetcollector/Collect-ShopDB.ps1` (the mature
CSV collector), the migration/assessment flow under `scripts/.temp`
(`Assess-Orchestrator.ps1`, `Setup_ShopPC.ps1`, `Setup_InspectionPC.ps1`,
`eDncRegToUDC.ps1`, `machinetypeselector.bat`), and the eDNC/UDC/MTConnect config
formats those scripts read. Treat it as the "why" that the terse contract in
sections 1 through 8 cannot carry.
### 9.0 Why this matters beyond one plugin
This collector is a wedge into an enterprise-wide body of knowledge. A shop-floor
PC is never just a PC: it is the human-facing half of a machine tool, the thing
that feeds NC programs to a controller and streams data back off it. Getting its
identity, hardware, network, and application picture right lets ShopDB answer
questions that today require a person to walk the floor: which controller does
this box talk to, is its data path healthy, is the hardware under warranty, what
is it actually running. Every signal below earns its place by answering one of
those questions.
### 9.1 The endpoint taxonomy (how the floor classifies its own PCs)
`machinetypeselector.bat` plus the two `Setup_*` scripts show the site already
thinks in three roles. The collector should be able to DERIVE the role from
on-box evidence rather than ask an operator:
- Shop PC: runs UDC (Universal Data Collector) and/or eDNC, wired to one CNC
machine (FANUC / Siemens / Okuma controller). This is the ~2/3 majority.
- Inspection PC: metrology stack. Marked by `C:\GE Aerospace\Genspect\genspect.true`
(a flag file `Setup_InspectionPC.ps1` writes) and/or `C:\GE Aerospace\Gagecal`
(GageCal 6.4).
- Non-shop-floor PC: office box, no machine link, minimal shop apps.
Role is derivable from application/marker presence (section 9.4), which is a more
honest signal than the raw WMI `Model` string the current `pctype` probe sends.
### 9.2 Hardware facts, and WHY
`Collect-ShopDB.ps1 :: Get-PcFacts` reads `Win32_ComputerSystem`,
`Win32_OperatingSystem`, and `Win32_BIOS` for: vendor/manufacturer, model, OS
caption, serial (BIOS serial first, ComputerSystem serial as fallback), last
boot, logged-in user.
Why hardware is not cosmetic:
- Dell Warranty API linkage. Vendor + model family + service tag (BIOS serial)
is exactly the tuple the Dell Warranty API keys on. Capturing it lets ShopDB
resolve warranty status and hardware line remotely, so a support tech can
diagnose a hardware-level fault (failing disk, EOL platform, out-of-warranty
chassis) without physically finding the machine. The serial is the join key;
the model family is the diagnostic context.
- Remote hardware diagnosis generally. Knowing the exact hardware line turns
"some PC is flaky" into "this is a known-bad model/BIOS combination," which is
an enterprise-scale pattern once the fleet reports uniformly.
Collection note: prefer BIOS `SerialNumber` (the Dell service tag) and capture
manufacturer + model as distinct fields, since the warranty join needs the raw
service tag, not a normalized `pctype`.
### 9.3 Network interfaces, and WHY (the second-NIC reality)
`Get-PcFacts` and `Assess-Orchestrator.ps1 :: Get-EthernetNetworkSnapshot`
enumerate hardware adapters only and capture alias, IPv4, MAC, gateway, and
subnet (the orchestrator even reconstructs the mask from prefix length).
Why the interface list is first-class, not trivia:
- The second interface IS the machine link. Upwards of two-thirds of shop-floor
PCs have a SECOND network interface with a STATIC, manually mapped IP address
whose sole purpose is to talk to the related machine's onboard IPS
(industrial PC / controller network). That private NIC is physical evidence of
the PC-to-machine relationship and often the only place the machine-side
address is recorded. Losing it loses the wiring diagram of the floor.
- MAC as stable identity. MAC is the most stable hardware identifier across
re-image and rename, which is why every extant script grabs it. It is the
durable anchor when hostname/asset number churn during migration.
Collection note: send ALL hardware interfaces (alias, IPv4, MAC, and where
available gateway/subnet), not just the primary. The primary/gateway NIC is the
corporate LAN; the gatewayless static NIC is very likely the machine IPS link and
is the interesting one.
### 9.4 Installed applications, and WHY (the extrapolation engine)
`Get-PcFacts` reads the registry Uninstall keys (name, version, publisher,
skipping `SystemComponent`). On these boxes the installed-app set is small and
highly diagnostic: they primarily run a handful of known apps.
Why the app inventory is the richest signal we have:
- Apps imply role (section 9.1): UDC/eDNC => Shop PC; Genspect/GageCal =>
Inspection PC; neither => office.
- Apps imply the MACHINE binding. This is the crux. If UDC or eDNC (or MTConnect)
is installed, its config file carries the four-digit number ShopDB calls the
Machine Number, and from it we can extrapolate the ShopDB `machines` entry the
PC should be related to. The same value appears under different names across
the app generations:
- eDNC `.reg` / live registry: `MachineNo` (under
`HKLM\SOFTWARE\[WOW6432Node\]GE Aircraft Engines\DNC\General`).
- UDC `udc_settings_*.json`: `GeneralSettings.MachineNumber`.
- MTConnect: workstation / workstation ID / MachineNo, carried in the
adapter/agent config under the fixed directory tree documented in section
9.9.
Treat "MachineNo", "workstation", "workstation ID", and "MachineNumber" as the
same concept: the ShopDB Machine Number. Version and publisher are worth
keeping too (they date the app generation, which tells you which config format
to parse).
Collection note: the app inventory is not just an asset list; it is the pointer
that tells the collector WHICH config file to open next to find the machine
binding.
### 9.5 The machine binding (DNC/UDC/MTConnect config), and WHY
`Collect-ShopDB.ps1 :: Resolve-DncSource` resolves the machine config with a
SOURCE-PRIORITY chain, then three parsers normalize the formats into one shape:
- Priority: explicit JSON path -> explicit `.reg` path -> live registry
(`HKLM\...\GE Aircraft Engines\DNC`) -> local `.reg` export -> local
`udc_settings_*.json`. This ordering makes one probe work across both the
eDNC-era and UDC-era boxes.
- Parsers: `ConvertFrom-RegExport` (parses exported `.reg`, including
`dword:` decoding), `ConvertFrom-LiveRegistry`, `ConvertFrom-UdcJson`.
- Extracted: machine number, site, control type, DNC type, and connection detail
(TCP address/port for ethernet; COM port/baud for serial).
- Controller vendor is DERIVED, not stored raw: `Resolve-ControllerVendor` maps
`NcIF = EFOCAS -> FANUC`, `Cnc = SIEMENS -> Siemens`, `Cnc = OKUMA -> Okuma`.
`eDncRegToUDC.ps1` confirms the same mapping and the eFocas/PPDCS key layout.
Why it matters: this binding is what upgrades a bare endpoint into "the PC that
controls machine 1234 over eFOCAS at 10.x.x.x." That relationship is the point of
the whole exercise; the hardware and app facts are supporting evidence for it.
### 9.6 Printers, and WHY
`Assess-Orchestrator.ps1 :: Get-PrinterSnapshot` / `Get-PrinterPortSnapshot`
capture name, driver, port, host address, shared/published, and the default
flag, filtering out virtual outputs (Print to PDF, XPS, Fax, OneNote). Shop and
inspection PCs are tied to specific physical/label printers, so the printer set
is part of a workstation's operational identity and a common migration break
point (a replaced PC that lost its label printer is a floor stoppage).
### 9.7 Patterns from the extant tooling worth carrying forward
Two engineering habits in these scripts are worth honoring in the collector,
because they encode hard-won migration lessons:
- Field-level idempotent merge (`Merge-CsvStore`): an empty incoming cell NEVER
overwrites a previously captured value; later partial runs fill gaps without
wiping earlier data. The server's endpoints ingest already mirrors this instinct
(`if 'field' in payload`), and it should stay that way: a partial report means
"leave the rest alone," not "clear it."
- Normalized intermediate shape + sidecars: heterogeneous sources (registry,
`.reg`, JSON, WMI) all converge on one normalized record before import, with
phase-2 data (interfaces, installed apps, PC-to-machine link) written as JSON
sidecars. The same decoupling lets a future non-agent source (Intune/Azure
`Get-MgDevice`, a MES/CNC database) feed the same model without a new pipeline.
### 9.9 MTConnect footprint, and WHY (three parallel stacks per box)
Source: `scripts/.temp/Install_MTConnect_ExisDir.ps1`,
`Install_MTConnect_ExisDir_BatConvert.ps1`, its usage guide
`Guide-Install_MTConnect_ExisDir_BatConvert.md`, and
`Firewall_Rules_MTConnect_All.bat`. MTConnect is the standardized data-off-the-
machine layer, and the install scripts reveal its exact on-disk shape, which is
what a collector needs to detect it reliably.
The critical structural fact: an endpoint can host up to THREE parallel MTConnect
stacks, one per data path, each an Adapter (talks to the machine) plus an Agent
(serves the MTConnect HTTP feed):
- Machine-direct: `C:\MTConnect\Adapter` + `C:\MTConnect\Agent` (Agent exe
`C:\MTConnect\Agent\Agent.exe`).
- eDNC-fed: `C:\MTConnect_eDNC\Adapter` + `C:\MTConnect_eDNC\Agent`
(`...\Agent\agent.exe`). The eDNC adapter is `DNCAdapter.exe` driven by
`DNCAdapter.ini`, tying this path back to the eDNC config world in section 9.5.
- UDC-fed: `C:\ProgramData\udc\mtconnect_udc\agent` (`...\agent\agent.exe`,
launched via `run.bat`). This binds MTConnect to the UDC install and therefore
to the same `udc_settings_*.json` that carries `GeneralSettings.MachineNumber`.
Why the collector should care about these paths:
- Presence is role/path evidence. Which of the three trees exist tells you HOW the
machine's data is being lifted (direct vs eDNC vs UDC), which is a real
operational distinction, not a cosmetic one.
- The MTConnect Agent is a network service. The firewall bat and the
`Ensure-FirewallProgramRule` calls open inbound rules named
`Allow_MTConnect_Machine`, `Allow_MTConnect_UDC`, `Allow_MTConnect_eDNC` for
each `Agent.exe`. That means each agent listens on an HTTP port, so its live
MTConnect feed (which publishes a device UUID / station identity) is itself a
future data source the collector could probe, not just a process to count.
- Machine Number lives here too. Because the eDNC path is `DNCAdapter.ini` and the
UDC path is the UDC settings JSON, the MTConnect footprint is a second route to
the same four-digit Machine Number described in 9.4/9.5, cross-checkable against
the eDNC/UDC values.
Collection note: detect MTConnect by the three directory trees, capture which
Agent exes exist and whether their service/process is running (section 5 already
covers MTConnect as a monitored service), and record the data path
(direct/eDNC/UDC).
(`Install_MTConnect_ExisDir_BatConvert.ps1`) is a deployment concern, not a
collection concern, but it explains why these `install.bat`/`start.bat` files
were rewritten to self-elevate without a `pause`: so they can run unattended
under DSC on locked-down machines. Do not treat a self-elevating batch as
tampering.
### 9.8 Boundary (explicit)
None of section 9 is a ShopDB schema proposal. Where a signal above has no home
in the current model, that is fine: it stays reference until a separate, deliberate
decision (its own ADR / contract bump) decides to store it. The value here is
comprehension: anyone extending the collector should know that hardware feeds
warranty, the second NIC is the machine link, and the installed apps are the
thread that leads to the Machine Number. Build the agent with that understanding;
do not let this document, by itself, move the schema.

5
plugins/endpoints/__init__.py Normal file
View File

@@ -0,0 +1,5 @@
"""Endpoints plugin package."""
from .plugin import EndpointsPlugin
__all__ = ['EndpointsPlugin']

5
plugins/endpoints/api/__init__.py Normal file
View File

@@ -0,0 +1,5 @@
"""Endpoints plugin API."""
from .routes import endpoints_bp
__all__ = ['endpoints_bp']

724
plugins/endpoints/api/routes.py Normal file
View File

@@ -0,0 +1,724 @@
"""Endpoints plugin API endpoints (read + dashboard).
Ingest is NOT here: agent reports arrive on the generic collector route
POST /api/collector/endpoints (ADR-006), authorized by a collector-scoped
service token, and are handled by EndpointsPlugin.apply_collector_payload.
These routes are the human-facing read surface, gated by user JWT + RBAC.
"""
import json
import time
from datetime import datetime, timedelta, timezone
from flask import Blueprint, request, current_app
from flask_jwt_extended import current_user
from shopdb.api import (
db, Asset, Setting, success_response, error_response, paginated_response,
ErrorCodes, get_pagination_params, paginate_query, require_permission,
require_role, mint_service_token, list_service_tokens, revoke_service_tokens,
AuditLog, collector_authorized,
)
from ..models import EndpointState, EndpointEvent, EndpointCommand
from ..commands import COMMAND_ALLOWLIST, is_allowed_action, ALLOWED_ACTIONS_FOR, row_actions_for, requires_target, action_targetkind, elevated_permission_for
from ..eventcatalog import CRITICAL_EVENT_IDS, event_severity, event_title
endpoints_bp = Blueprint('endpoints', __name__)
def _naive_utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
def _format_uptime(seconds):
"""Whole-seconds uptime -> 'Nd Nh Nm', or None."""
if seconds is None:
return None
try:
seconds = int(seconds)
except (TypeError, ValueError):
return None
days, rem = divmod(seconds, 86400)
hours, rem = divmod(rem, 3600)
minutes = rem // 60
parts = []
if days:
parts.append('{}d'.format(days))
if hours:
parts.append('{}h'.format(hours))
parts.append('{}m'.format(minutes))
return ' '.join(parts)
# =============================================================================
# Endpoint state
# =============================================================================
@endpoints_bp.route('/', methods=['GET'])
@require_permission('endpoints.view')
def list_endpoints():
"""List reported endpoint states.
Joins through Computer so each row carries computerid and can deep-link
into the existing PC detail page (/pcs/{computerid}). Endpoint state only
exists for hostnames resolved to a Computer, so the join is not lossy.
"""
from plugins.computers.models import Computer
page, per_page = get_pagination_params(request)
query = (db.session.query(EndpointState, Computer.computerid)
.join(Asset, Asset.assetid == EndpointState.assetid)
.join(Computer, Computer.assetid == EndpointState.assetid)
.filter(Asset.isactive.is_(True)))
if search := request.args.get('search'):
query = query.filter(EndpointState.hostname.ilike(f'%{search}%'))
query = query.order_by(EndpointState.lastreportedat.desc())
items, total = paginate_query(query, page, per_page)
data = []
for state, computerid in items:
row = state.to_dict()
row['computerid'] = computerid
row['uptimedisplay'] = _format_uptime(state.uptimeseconds)
data.append(row)
return paginated_response(data, page, per_page, total)
@endpoints_bp.route('/<int:asset_id>', methods=['GET'])
@require_permission('endpoints.view')
def get_endpoint(asset_id: int):
"""Get the current state for one endpoint by asset id."""
state = EndpointState.query.filter_by(assetid=asset_id).first()
if not state:
return error_response(
ErrorCodes.NOT_FOUND,
f'No endpoint state for asset {asset_id}',
http_code=404,
)
return success_response(state.to_dict())
@endpoints_bp.route('/<int:asset_id>/events', methods=['GET'])
@require_permission('endpoints.view')
def list_endpoint_events(asset_id: int):
"""List curated Windows events for one endpoint, newest first."""
page, per_page = get_pagination_params(request)
query = EndpointEvent.query.filter_by(assetid=asset_id)
if eventid := request.args.get('eventid'):
try:
query = query.filter(EndpointEvent.eventid == int(eventid))
except (TypeError, ValueError):
return error_response(ErrorCodes.VALIDATION_ERROR,
'eventid must be an integer')
query = query.order_by(EndpointEvent.eventtime.desc())
items, total = paginate_query(query, page, per_page)
data = [e.to_dict() for e in items]
return paginated_response(data, page, per_page, total)
# =============================================================================
# Asset panels (ADR-010) - shaped for the generic PluginAssetPanels renderer
# =============================================================================
@endpoints_bp.route('/asset/<int:asset_id>/status', methods=['GET'])
@require_permission('endpoints.view')
def panel_status(asset_id: int):
"""Keyvalue panel: the endpoint's current status snapshot.
Single indexed lookup on endpointstates.assetid. Returns the {fields:[...]}
shape the keyvalue renderer expects; an absent state yields no fields so the
panel hides itself.
"""
state = EndpointState.query.filter_by(assetid=asset_id).first()
if not state:
return success_response({'fields': []})
def field(label, value, mono=False):
return {'label': label, 'value': value, 'mono': mono}
fields = [
field('Logged in user', state.loggedinuser),
field('Uptime', _format_uptime(state.uptimeseconds)),
field('Idle', _format_uptime(state.idleseconds)),
field('Last boot',
state.lastboottime.isoformat() + 'Z' if state.lastboottime else None),
field('CPU %', state.cpupercent),
field('Memory %', state.memorypercent),
field('Agent', state.agentversion, mono=True),
field('Last reported',
state.lastreportedat.isoformat() + 'Z' if state.lastreportedat else None),
]
fields = [f for f in fields if f['value'] not in (None, '')]
return success_response({'fields': fields})
@endpoints_bp.route('/asset/<int:asset_id>/telemetrymeta', methods=['GET'])
@require_permission('endpoints.view')
def panel_telemetry_meta(asset_id: int):
"""Lightweight freshness probe for the panels.
Returns just the endpoint's lastreportedat so the WebUI can tell whether a
genuinely new telemetry snapshot has landed after it forces a collect. The
UI captures this before enqueuing collect_now (or a row Start/Stop), then
polls until the stamp advances - a command reaching 'succeeded' only means
the agent ran it, not that it has yet posted the fresh services snapshot, so
gating the refresh on this timestamp is what avoids re-rendering stale rows.
"""
state = EndpointState.query.filter_by(assetid=asset_id).first()
stamp = (state.lastreportedat.isoformat() + 'Z'
if state and state.lastreportedat else None)
return success_response({'lastreportedat': stamp})
@endpoints_bp.route('/asset/<int:asset_id>/services', methods=['GET'])
@require_permission('endpoints.view')
def panel_services(asset_id: int):
"""List panel: monitored services/processes with a run-state badge per row.
Reshapes the latest servicesjson snapshot into rows the list renderer maps
to name + status badge. Tolerant of a missing or unexpected snapshot shape:
an unknown state renders as an Unknown badge rather than failing.
"""
state = EndpointState.query.filter_by(assetid=asset_id).first()
if not state or not state.servicesjson:
return success_response([])
try:
services = json.loads(state.servicesjson)
except (TypeError, ValueError):
return success_response([])
rows = []
for entry in (services or []):
if isinstance(entry, dict):
name = entry.get('name') or entry.get('service') or 'Unknown'
raw = (entry.get('state') or entry.get('status') or '').strip().lower()
app = entry.get('app')
kind = entry.get('kind')
else:
name, raw, app, kind = str(entry), '', None, None
running = raw in ('running', 'started', 'up', 'ok', 'active')
row = {
'name': name,
'kind': (kind or '').lower() or None,
'statelabel': 'Running' if running else (raw.title() or 'Unknown'),
'statecolor': 'green' if running else ('red' if raw else 'gray'),
}
targets = row_actions_for(app, kind, name)
if targets:
row['action'] = {
'running': running,
'start': targets['start'],
'stop': targets['stop'],
'status': targets['status'],
'target': targets['target'],
'targetkind': targets['targetkind'],
}
rows.append(row)
return success_response(rows)
@endpoints_bp.route('/asset/<int:asset_id>/recentevents', methods=['GET'])
@require_permission('endpoints.view')
def panel_recent_events(asset_id: int):
"""List panel: the endpoint's most recent curated events, newest first.
Indexed range scan on (assetid, eventtime), capped small - this is the
at-a-glance history card, not the full report (that is the events route
with pagination).
"""
events = (EndpointEvent.query
.filter_by(assetid=asset_id)
.order_by(EndpointEvent.eventtime.desc())
.limit(10)
.all())
rows = [{
'title': event_title(e.eventid),
'severity': event_severity(e.eventid),
'eventtime': e.eventtime.isoformat() + 'Z' if e.eventtime else None,
} for e in events]
return success_response(rows)
@endpoints_bp.route('/asset/<int:asset_id>/interfaces', methods=['GET'])
@require_permission('endpoints.view')
def panel_interfaces(asset_id: int):
"""Table panel: every hardware NIC the agent last reported.
Reshapes the latest interfacesjson snapshot into one row per adapter with
the {alias, ipv4, mac, gateway, subnet, dhcp} shape the collector emits. A
gatewayless adapter is flagged as the likely machine-IPS link (see
DESIGN-DECISIONS-agent.md 9.3): those NICs speak to the machine, not the
corporate LAN. Tolerant of a missing or unexpected snapshot: an unreadable
payload yields no rows so the panel hides itself.
"""
state = EndpointState.query.filter_by(assetid=asset_id).first()
if not state or not state.interfacesjson:
return success_response([])
try:
interfaces = json.loads(state.interfacesjson)
except (TypeError, ValueError):
return success_response([])
rows = []
for entry in (interfaces or []):
if not isinstance(entry, dict):
continue
gateway = (entry.get('gateway') or '').strip()
ipv4 = (entry.get('ipv4') or '').strip()
rows.append({
'alias': entry.get('alias') or 'Unknown',
'ipv4': ipv4 or '-',
'mac': (entry.get('mac') or '').strip() or '-',
'gateway': gateway or '-',
'subnet': (entry.get('subnet') or '').strip() or '-',
'dhcp': 'DHCP' if entry.get('dhcp') else 'Static',
# A gatewayless NIC with an address is the machine link; the gateway
# NIC is the corporate LAN.
'role': 'Machine link' if (ipv4 and not gateway) else 'LAN',
})
return success_response(rows)
# =============================================================================
# Dashboard
# =============================================================================
@endpoints_bp.route('/dashboard/summary', methods=['GET'])
@require_permission('endpoints.view')
def dashboard_summary():
"""High-level fleet counts for the endpoints dashboard."""
total = db.session.query(EndpointState).join(
Asset, Asset.assetid == EndpointState.assetid
).filter(Asset.isactive.is_(True)).count()
hours = _quiet_hours()
cutoff = _naive_utcnow() - timedelta(hours=hours)
reporting = db.session.query(EndpointState).join(
Asset, Asset.assetid == EndpointState.assetid
).filter(
Asset.isactive.is_(True),
EndpointState.lastreportedat.isnot(None),
EndpointState.lastreportedat >= cutoff,
).count()
return success_response({
'total': total,
'reporting': reporting,
'quiet': total - reporting,
'quiethours': hours,
})
@endpoints_bp.route('/dashboard/quiet', methods=['GET'])
@require_permission('endpoints.view')
def dashboard_quiet():
"""Endpoints that reported before, then went quiet past the window.
Never-reported endpoints are excluded: those are records without an agent,
not agents that stopped. Silence is the only signal - there is no heartbeat
separate from the report.
"""
hours = _quiet_hours()
cutoff = _naive_utcnow() - timedelta(hours=hours)
now = _naive_utcnow()
from plugins.computers.models import Computer
rows = (db.session.query(EndpointState, Computer.computerid)
.join(Asset, Asset.assetid == EndpointState.assetid)
.join(Computer, Computer.assetid == EndpointState.assetid)
.filter(Asset.isactive.is_(True),
EndpointState.lastreportedat.isnot(None),
EndpointState.lastreportedat < cutoff)
.all())
out = [{
'computerid': computerid,
'assetid': s.assetid,
'hostname': s.hostname,
'lastreported': s.lastreportedat.isoformat() + 'Z',
'quietdays': (now - s.lastreportedat).days,
} for s, computerid in rows]
out.sort(key=lambda r: -r['quietdays'])
return success_response(out[:50])
@endpoints_bp.route('/dashboard/stability', methods=['GET'])
@require_permission('endpoints.view')
def dashboard_stability():
"""Endpoints with recent crash/shutdown/corruption events.
Bounded to a fixed 48h window and the critical event catalog, capped at 50
rows: cost is a range scan on eventtime filtered to a short id list, not a
scan of all history. Joins through Computer so each row links to the PC
detail page.
"""
from plugins.computers.models import Computer
if not CRITICAL_EVENT_IDS:
return success_response([])
cutoff = _naive_utcnow() - timedelta(hours=48)
rows = (db.session.query(
Computer.computerid,
EndpointState.hostname,
db.func.count(EndpointEvent.endpointeventid),
db.func.max(EndpointEvent.eventtime),
)
.select_from(EndpointEvent)
.join(Asset, Asset.assetid == EndpointEvent.assetid)
.join(Computer, Computer.assetid == EndpointEvent.assetid)
.outerjoin(EndpointState,
EndpointState.assetid == EndpointEvent.assetid)
.filter(Asset.isactive.is_(True),
EndpointEvent.eventid.in_(CRITICAL_EVENT_IDS),
EndpointEvent.eventtime.isnot(None),
EndpointEvent.eventtime >= cutoff)
.group_by(Computer.computerid, EndpointState.hostname)
.all())
out = [{
'computerid': computerid,
'hostname': hostname,
'eventcount': count,
'lastevent': lasttime.isoformat() + 'Z' if lasttime else None,
} for computerid, hostname, count, lasttime in rows]
out.sort(key=lambda r: -r['eventcount'])
return success_response(out[:50])
def _quiet_hours() -> int:
"""Quiet-report window in hours, from a site setting, default 24."""
hours = 24
setting = Setting.query.filter_by(key='endpoints_quietreporthours').first()
if setting and (setting.value or '').strip():
try:
hours = int(setting.value)
except (TypeError, ValueError):
pass
return hours
# =============================================================================
# Endpoint agent key (UI-managed collector.ingest.endpoints kill switch)
# =============================================================================
# The single permission the endpoint agent key authorizes. A managed token
# scoped to ONLY this authorizes the endpoints collector and nothing else, so
# rotating/revoking it stops endpoint-agent traffic without touching the
# computers/geenforce collectors.
AGENT_KEY_SCOPE = 'collector.ingest.endpoints'
AGENT_KEY_NAME = 'Endpoint agent key'
def _agentkey_status_payload():
"""Current state of the endpoint agent key for the settings page.
Reports whether an active managed key exists (and its non-secret metadata)
plus whether the bootstrap .env fallback is still accepted. Never exposes a
secret; the full secret is only ever returned once, at regenerate time.
"""
tokens = list_service_tokens(AGENT_KEY_SCOPE)
active = tokens[0] if tokens else None
envfallback = Setting.get('plugin.endpoints.envkeyfallback', default=True)
provisioned = Setting.get('plugin.endpoints.agentkeyprovisioned',
default=False)
return {
'provisioned': bool(provisioned) or active is not None,
'envkeyfallback': bool(envfallback),
'key': active.to_dict() if active is not None else None,
}
@endpoints_bp.route('/agentkey', methods=['GET'])
@require_role('admin')
def get_agentkey():
"""Report the endpoint agent key state (no secret)."""
return success_response(_agentkey_status_payload())
@endpoints_bp.route('/agentkey/regenerate', methods=['POST'])
@require_role('admin')
def regenerate_agentkey():
"""Mint a fresh endpoint agent key, revoking any previous one.
Hard cutover: the old key is deactivated in the same call, and the .env
fallback is switched off, so the new managed token becomes the sole
credential and revoking it is a true kill switch. The full secret is
returned ONCE here and never again.
"""
try:
token, secret = mint_service_token(
AGENT_KEY_SCOPE, AGENT_KEY_NAME, current_user.userid)
except PermissionError as error:
return error_response(ErrorCodes.FORBIDDEN, str(error), http_code=403)
Setting.set('plugin.endpoints.agentkeyprovisioned', True,
valuetype='boolean', category='plugin.endpoints')
Setting.set('plugin.endpoints.envkeyfallback', False,
valuetype='boolean', category='plugin.endpoints')
result = _agentkey_status_payload()
# The secret appears here and NOWHERE else, ever. Not stored, not logged.
result['secret'] = secret
result['warning'] = ('Save this key now. It will not be shown again. '
'Configure it as the agent X-API-Key.')
return success_response(result, message='Endpoint agent key generated',
http_code=201)
@endpoints_bp.route('/agentkey', methods=['DELETE'])
@require_role('admin')
def revoke_agentkey():
"""Revoke the endpoint agent key: stop all managed-token endpoint traffic.
Leaves envkeyfallback off (the kill switch stays armed). To restore
reporting an operator regenerates a new key.
"""
revoked = revoke_service_tokens(AGENT_KEY_SCOPE)
Setting.set('plugin.endpoints.agentkeyprovisioned', False,
valuetype='boolean', category='plugin.endpoints')
result = _agentkey_status_payload()
result['revoked'] = revoked
return success_response(result, message='Endpoint agent key revoked')
# =============================================================================
# Endpoint commands (WebUI -> agent, allowlisted actions only)
# =============================================================================
#
# Operators enqueue a named, server-allowlisted action for one endpoint (JWT +
# endpoints.command). The agent long-polls /commands/next with its collector
# token, claims the single next pending command, runs it locally, and reports
# the outcome to /commands/<id>/result. The wire never carries a command string
# - only an allowlisted action name - so a compromised WebUI cannot ask an
# endpoint to run something arbitrary.
# How long the agent claim poll is held open server-side before returning empty.
# Kept below the agent HttpClient timeout and typical proxy idle timeouts so a
# quiet fleet re-polls cleanly instead of tripping a read timeout.
_CLAIM_HOLD_SECONDS = 25
_CLAIM_POLL_INTERVAL = 1.0
def _collector_auth_or_error():
"""Authorize an agent-facing command route on the endpoints collector token.
Reuses the collector auth path so the command channel shares the endpoint
agent key (kill switch) and introduces no second secret.
"""
return collector_authorized('endpoints')
def _resolve_asset_for_hostname(hostname):
"""Resolve a reported hostname to its asset id via the computers plugin."""
from plugins.computers.models import Computer
computer = (Computer.query
.filter(db.func.lower(Computer.hostname) == hostname.lower())
.first())
return computer.assetid if computer is not None else None
@endpoints_bp.route('/<int:asset_id>/commands', methods=['GET'])
@require_permission('endpoints.view')
def list_endpoint_commands(asset_id: int):
"""Recent commands for one endpoint, newest first (operator view)."""
rows = (EndpointCommand.query
.filter_by(assetid=asset_id)
.order_by(EndpointCommand.createdat.desc())
.limit(20)
.all())
return success_response([row.to_dict() for row in rows])
@endpoints_bp.route('/<int:asset_id>/commands', methods=['POST'])
@require_permission('endpoints.command')
def enqueue_endpoint_command(asset_id: int):
"""Queue an allowlisted command for one endpoint.
Rejects unknown actions and refuses to stack a second pending/running
command on the same endpoint, so the queue stays at-most-one-in-flight and
the WebUI toggle cannot race itself.
"""
payload = request.get_json(silent=True) or {}
action = (payload.get('action') or '').strip()
if not is_allowed_action(action):
return error_response(ErrorCodes.VALIDATION_ERROR,
f'Unknown or disallowed action: {action!r}')
# Destructive actions (for example system_restart) demand a stricter grant
# ON TOP OF endpoints.command, so a remote reboot cannot ride the routine
# toggle permission.
elevated = elevated_permission_for(action)
if elevated and not current_user.haspermission(elevated):
return error_response(
ErrorCodes.FORBIDDEN,
f'Action {action!r} requires the {elevated} permission',
http_code=403)
if action not in ALLOWED_ACTIONS_FOR(assetid=asset_id):
return error_response(
ErrorCodes.VALIDATION_ERROR,
f'Action not permitted by policy for this endpoint: {action!r}')
# Generic actions (service_* / process_*) must carry the subject to act on;
# fixed-target actions (mtconnect_*) must not. Validate both directions so a
# caller cannot send a targetless generic action (nothing to act on) or
# smuggle a target onto a pinned action.
target = (payload.get('target') or '').strip() or None
targetkind = action_targetkind(action)
if requires_target(action):
if not target:
return error_response(
ErrorCodes.VALIDATION_ERROR,
f'Action {action!r} requires a target')
elif target is not None:
return error_response(
ErrorCodes.VALIDATION_ERROR,
f'Action {action!r} does not accept a target')
state = EndpointState.query.filter_by(assetid=asset_id).first()
if state is None:
return error_response(ErrorCodes.NOT_FOUND,
f'No endpoint state for asset {asset_id}',
http_code=404)
pending = (EndpointCommand.query
.filter(EndpointCommand.assetid == asset_id,
EndpointCommand.status.in_((
EndpointCommand.STATUS_PENDING,
EndpointCommand.STATUS_RUNNING)))
.first())
if pending is not None:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'A command is already in flight for this endpoint',
http_code=409)
command = EndpointCommand(
assetid=asset_id,
action=action,
target=target,
targetkind=targetkind if target else None,
status=EndpointCommand.STATUS_PENDING,
requestedby=getattr(current_user, 'userid', None),
)
db.session.add(command)
AuditLog.log('created', 'EndpointCommand',
entityname=action,
details=f'action={action} target={target} asset={asset_id}')
db.session.commit()
return success_response(command.to_dict(),
message='Command queued', http_code=201)
@endpoints_bp.route('/commands/next', methods=['GET'])
def claim_next_command():
"""Agent long-poll: claim the next pending command for a hostname.
Collector-token auth (not JWT). Holds the request open briefly and returns
the single oldest pending command for the caller's endpoint, atomically
marking it running so no other poll claims it. Returns 204 when nothing is
queued within the hold window - the agent simply re-polls.
"""
error = _collector_auth_or_error()
if error is not None:
return error
hostname = (request.args.get('hostname') or '').strip()
if not hostname:
return error_response(ErrorCodes.VALIDATION_ERROR,
'hostname is required')
asset_id = _resolve_asset_for_hostname(hostname)
if asset_id is None:
return error_response(ErrorCodes.NOT_FOUND,
f'Unknown hostname: {hostname!r}', http_code=404)
deadline = time.monotonic() + _CLAIM_HOLD_SECONDS
while True:
command = (EndpointCommand.query
.filter_by(assetid=asset_id,
status=EndpointCommand.STATUS_PENDING)
.order_by(EndpointCommand.createdat.asc())
.first())
if command is not None:
command.status = EndpointCommand.STATUS_RUNNING
command.claimedat = _naive_utcnow()
db.session.commit()
return success_response({
'endpointcommandid': command.endpointcommandid,
'action': command.action,
'target': command.target,
'targetkind': command.targetkind,
})
# Nothing queued: release the session so a concurrent enqueue is visible
# on the next poll, wait a beat, then re-check until the hold expires.
db.session.rollback()
if time.monotonic() >= deadline:
return ('', 204)
time.sleep(_CLAIM_POLL_INTERVAL)
@endpoints_bp.route('/commands/<int:command_id>/result', methods=['POST'])
def report_command_result(command_id: int):
"""Agent reports a claimed command's outcome (collector-token auth).
Finalizes the command row and, for MTConnect actions that observed the run
state, caches it on the EndpointState so the WebUI toggle reflects reality
between polls. Both writes land in one commit.
"""
error = _collector_auth_or_error()
if error is not None:
return error
command = db.session.get(EndpointCommand, command_id)
if command is None:
return error_response(ErrorCodes.NOT_FOUND,
f'No command {command_id}', http_code=404)
payload = request.get_json(silent=True) or {}
succeeded = bool(payload.get('succeeded'))
command.status = (EndpointCommand.STATUS_SUCCEEDED if succeeded
else EndpointCommand.STATUS_FAILED)
command.resultmessage = (payload.get('message') or '')[:2000] or None
detail = payload.get('detail')
command.resultjson = json.dumps(detail, separators=(',', ':')) \
if detail is not None else None
command.completedat = _naive_utcnow()
# Cache the observed MTConnect state so the toggle shows a real value. The
# agent sends mtconnectrunning on any mtconnect_* result that probed it.
if 'mtconnectrunning' in payload:
state = EndpointState.query.filter_by(assetid=command.assetid).first()
if state is not None:
running = payload.get('mtconnectrunning')
state.mtconnectrunning = bool(running) if running is not None else None
state.mtconnectcheckedat = _naive_utcnow()
db.session.commit()
return success_response(command.to_dict(), message='Result recorded')

198
plugins/endpoints/commands.py Normal file
View File

@@ -0,0 +1,198 @@
"""Server-side allowlist of endpoint agent commands.
The WebUI never sends an arbitrary command string. It sends an allowlisted
ACTION NAME; the concrete on-endpoint behavior (which script to run, how to stop
a process) lives in the agent. This module is the single source of truth for
which action names are legal and what they mean, so both the enqueue route and
the settings/UI surface validate against the same set.
The structure anticipates per-PC-type allowlists: today every computer may run
every action in COMMAND_ALLOWLIST, but ALLOWED_ACTIONS_FOR is the seam a later
change swaps for a type-scoped lookup (inspection PCs vs shopfloor PCs).
"""
# action name -> operator-facing metadata. label/description feed the UI; the
# agent maps the action name to its local handler (see CommandExecutor).
#
# Two families of action:
# * Named MTConnect actions - fixed target, no parameters. The original,
# bay-specific controls kept for the inline services-panel toggle.
# * Generic RMM actions (service_* / process_*) - the target is supplied by
# the policy action's params ('target' = a Windows service name or a process
# image like 'DncMain.exe'). This is what populates the Configuration Policy
# builder with reusable controls that any scope (PC type or application) can
# attach. The 'targetkind' hint tells the builder which target field to show.
# NOTE: agent-side execution of the generic actions is not wired yet; the
# server allowlist and policy plumbing land first (the whole feature is
# flag-gated), so a generic action saved today is authored and gated but is
# a no-op on the endpoint until the CommandExecutor handlers ship.
COMMAND_ALLOWLIST = {
'service_start': {
'label': 'Start service',
'description': 'Start a Windows service by name on the endpoint.',
'targetkind': 'service',
},
'service_stop': {
'label': 'Stop service',
'description': 'Stop a running Windows service by name on the endpoint.',
'targetkind': 'service',
},
'service_status': {
'label': 'Check service',
'description': 'Report whether a named Windows service is running.',
'targetkind': 'service',
},
'process_start': {
'label': 'Start process',
'description': 'Launch a process by executable path on the endpoint.',
'targetkind': 'process',
},
'process_stop': {
'label': 'Stop process',
'description': 'Stop a running process by image name on the endpoint.',
'targetkind': 'process',
},
'process_status': {
'label': 'Check process',
'description': 'Report whether a named process is running.',
'targetkind': 'process',
},
'mtconnect_start': {
'label': 'Start MTConnect',
'description': 'Run the MTConnect UDC agent start script on the endpoint.',
},
'mtconnect_stop': {
'label': 'Stop MTConnect',
'description': 'Stop the running MTConnect UDC agent on the endpoint.',
},
'mtconnect_status': {
'label': 'Check MTConnect',
'description': 'Report whether the MTConnect UDC agent is running.',
},
'system_restart': {
'label': 'Restart computer',
'description': 'Restart the endpoint computer immediately.',
},
'collect_now': {
'label': 'Collect now',
'description': 'Force the endpoint agent to report fresh telemetry off-cadence.',
},
}
# Actions that demand a permission stricter than the base endpoints.command.
# The enqueue route requires the caller to ALSO hold the mapped permission, so a
# destructive action (a remote reboot) is a heavier grant than a routine toggle.
ELEVATED_ACTION_PERMISSIONS = {
'system_restart': 'endpoints.restart',
}
def elevated_permission_for(action):
"""The extra permission an action requires, or None for routine actions."""
return ELEVATED_ACTION_PERMISSIONS.get(action)
# Maps a monitored services/processes row to the start/stop/status actions that
# control it, so the services panel can render an inline toggle on exactly the
# rows that have a registered command action. The key is the agent-reported
# 'app' label; 'kind' pins the toggle to the authoritative row (for MTConnect
# the agent.exe PROCESS is the source of truth - run.bat launches it and the
# Windows service state is not reliable), so the service row stays badge-only.
ROW_COMMAND_TARGETS = {
('MTConnect', 'process'): {
'start': 'mtconnect_start',
'stop': 'mtconnect_stop',
'status': 'mtconnect_status',
},
}
def row_actions_for(app, kind, name=None):
"""Return the {start, stop, status, target, targetkind} map for a row.
Two paths:
* A row registered in ROW_COMMAND_TARGETS (MTConnect) keeps its fixed,
named actions with no wire target - agent.exe is the source of truth and
the behavior is pinned in the agent.
* Any other process/service row maps to the generic primitives
(process_* / service_*) with the row's OWN reported name as the wire
target. This is what lets a new app light up start/stop/status without
an agent rebuild: the target rides the wire.
Rows with no usable kind return None and render as a plain status badge.
"""
if not app and not name:
return None
fixed = ROW_COMMAND_TARGETS.get((app, (kind or '').lower()))
if fixed:
return {**fixed, 'target': None, 'targetkind': None}
normalized = (kind or '').lower()
if normalized == 'process' and name:
return {
'start': 'process_start',
'stop': 'process_stop',
'status': 'process_status',
'target': name,
'targetkind': 'process',
}
if normalized == 'service' and name:
return {
'start': 'service_start',
'stop': 'service_stop',
'status': 'service_status',
'target': name,
'targetkind': 'service',
}
return None
def is_allowed_action(action):
"""True when action is a known, allowlisted command name."""
return action in COMMAND_ALLOWLIST
def action_targetkind(action):
"""The targetkind an action requires ('service'/'process'), or None.
Generic actions declare a targetkind in COMMAND_ALLOWLIST; fixed-target
actions (mtconnect_*) declare none and must not carry a wire target.
"""
meta = COMMAND_ALLOWLIST.get(action) or {}
return meta.get('targetkind')
def requires_target(action):
"""True when the action is generic and needs a wire target."""
return action_targetkind(action) is not None
def ALLOWED_ACTIONS_FOR(pctype=None, assetid=None):
"""The action names an endpoint may run.
Base allowlist is flat (every action for every computer). When the computers
Configuration/Business Policy feature is enabled AND an assetid is given, the
result is narrowed to the actions the endpoint's effective policy permits -
this is the allowlist-gating half of the policy design. With the feature off
(or no assetid), it returns the full base allowlist, so nothing changes until
a site opts in. Policy resolution is imported lazily to keep this module free
of a hard dependency on the computers plugin.
"""
base = set(COMMAND_ALLOWLIST.keys())
if assetid is None:
return base
try:
from plugins.computers.services.policyresolver import effective_policy_for
except Exception:
return base
effective = effective_policy_for(assetid)
allowed = effective.allowed_actions()
if not allowed:
# No policy spoke for this endpoint. With the feature off the resolver
# returns empty and we fall back to the base allowlist (unchanged
# behavior). This is intentionally permissive; deny-by-default would be
# a separate, explicit setting.
return base
# Never widen past the code-level allowlist: policy can only gate within it.
return base & allowed

46
plugins/endpoints/eventcatalog.py Normal file
View File

@@ -0,0 +1,46 @@
"""Curated Windows event catalog: id -> human title and severity.
Single server-side source of truth shared by the recent-events asset panel and
the stability dashboard card, so a 4625 reads as "Failed logon (warning)" in
both places without either restating the mapping. The set of IDs here MUST stay
in step with CURATED_EVENT_IDS in plugin.py; a test asserts they match.
Severity vocabulary matches the dashboard card renderer: 'critical', 'warning',
'info'. The IDs marked critical are the ones an operator/tech should act on
without being asked - unexpected shutdown, crash, filesystem corruption, SCM
timeout. Logon activity is informational history unless it is a failure.
"""
# id -> (title, severity, channel)
EVENT_CATALOG = {
4624: ('Successful logon', 'info', 'Security'),
4625: ('Failed logon', 'warning', 'Security'),
4634: ('Logoff', 'info', 'Security'),
4672: ('Special privileges assigned', 'info', 'Security'),
4697: ('Service installed', 'warning', 'Security'),
4698: ('Scheduled task created', 'warning', 'Security'),
41: ('Unexpected shutdown', 'critical', 'System'),
4618: ('Monitored security event', 'warning', 'Security'),
55: ('File system corruption', 'critical', 'System'),
7011: ('Service timeout', 'critical', 'System'),
10016: ('DCOM permission error', 'warning', 'System'),
1001: ('Application crash', 'critical', 'Application'),
}
# The IDs a tech should see surfaced fleet-wide on the dashboard stability card.
CRITICAL_EVENT_IDS = tuple(
eventid for eventid, (_t, severity, _c) in EVENT_CATALOG.items()
if severity == 'critical'
)
def event_title(eventid):
"""Human title for a curated event id, or a generic fallback."""
entry = EVENT_CATALOG.get(eventid)
return entry[0] if entry else 'Event {}'.format(eventid)
def event_severity(eventid):
"""Severity ('critical'|'warning'|'info') for a curated event id."""
entry = EVENT_CATALOG.get(eventid)
return entry[1] if entry else 'info'

26
plugins/endpoints/frontend/routes.js Normal file
View File

@@ -0,0 +1,26 @@
/**
* Endpoints plugin routes
*
* Groundwork for the fleet view. There is no endpoint-specific detail route
* yet: rows link into the existing PC detail page (/pcs/:id) so telemetry
* stays interoperable with the computers plugin instead of duplicating it.
*/
export default [
{
path: 'endpoints',
name: 'endpoints',
component: () => import('./views/EndpointsList.vue'),
meta: { plugin: 'endpoints' }
},
{
path: 'settings/endpoints/agentkey',
name: 'endpoints-agentkey-settings',
component: () => import('./views/AgentKeySettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true, plugin: 'endpoints' }
}
]
// No full-screen routes: every view in this plugin renders inside
// AppLayout. Declared so routes.gen.js can read `.toplevel` off the module
// without Rollup warning about a missing export.
export const toplevel = []

293
plugins/endpoints/frontend/views/AgentKeySettings.vue Normal file
View File

@@ -0,0 +1,293 @@
<template>
<div>
<div class="page-header">
<h2>Endpoints</h2>
</div>
<p class="setting-description">
Manage how shop-floor endpoint agents authenticate to this server, build
the agent for this site, and set what it reports and how often.
</p>
<!-- ====================================================================
Key management
================================================================= -->
<section class="settings-section">
<h3>Key management</h3>
<p class="field-hint">
The endpoint agent reports with a single managed key scoped to endpoint
telemetry only. It is separate from every other collector, so rotating
or revoking it stops all endpoint-agent traffic immediately without
touching the PC or GE-Enforce collectors.
</p>
<div class="card form-card">
<div v-if="message" class="settings-success">{{ message }}</div>
<div v-if="error" class="error-message">{{ error }}</div>
<div v-if="loading" class="field-hint">Loading...</div>
<template v-else>
<div class="form-group">
<label>Status</label>
<p v-if="key" class="field-hint">
Active key <code class="mono">{{ key.tokenprefix }}...</code>,
created {{ formatDate(key.createddate) }}.
<template v-if="key.lastusedat">
Last used {{ formatDate(key.lastusedat) }}.
</template>
<template v-else>Not used yet.</template>
</p>
<p v-else class="field-hint">
No endpoint agent key is active. Generate one to let agents report
with a managed credential.
</p>
</div>
<div class="form-group">
<label>Bootstrap .env fallback</label>
<p class="field-hint">
<template v-if="envkeyfallback">
<strong>On.</strong> The bootstrap COLLECTOR_API_KEY still
authorizes endpoint reports. Generating a key turns this off so
the managed key becomes the only credential.
</template>
<template v-else>
<strong>Off.</strong> Only the managed key above authorizes
endpoint reports, so revoking it is an instant kill switch.
</template>
</p>
</div>
<div class="button-row">
<button class="btn btn-primary" @click="regenerate"
:disabled="busy">
{{ busy ? 'Working...' : (key ? 'Regenerate key' : 'Generate key') }}
</button>
<button v-if="key" class="btn btn-danger" @click="revoke"
:disabled="busy">
Revoke key
</button>
</div>
</template>
</div>
<div v-if="secret" class="card form-card secret-card">
<h4>Save this key now</h4>
<p class="field-hint">
It will not be shown again. Configure it as the agent
<code>X-API-Key</code> header value.
</p>
<p class="mono secret-value">{{ secret }}</p>
<button class="btn btn-secondary" @click="copySecret">
{{ copied ? 'Copied' : 'Copy to clipboard' }}
</button>
</div>
<div class="card form-card">
<h4>How it works</h4>
<ul class="field-hint info-list">
<li>
<strong>Generate</strong> mints a fresh key and, on the first
generation, turns off the bootstrap <code>.env</code> fallback so
the managed key becomes the only accepted credential. The full
secret is shown once - store it in the agent config as the
<code>X-API-Key</code> header.
</li>
<li>
<strong>Regenerate</strong> is a hard cutover: the previous key is
revoked in the same moment the new one is minted. Every deployed
agent must be reconfigured with the new key before it can report
again.
</li>
<li>
<strong>Revoke</strong> deactivates the key and leaves it off. All
endpoint-agent reporting stops until you generate a new key. This is
the kill switch.
</li>
</ul>
<h4>When to rotate</h4>
<ul class="field-hint info-list">
<li>
<strong>An installer or agent left the secure environment.</strong>
If a build carrying the key is copied off the shop floor - onto an
engineer's laptop, a USB stick, or a machine that leaves the site -
treat the key as compromised and regenerate immediately.
</li>
<li>
<strong>A device is lost, stolen, or decommissioned</strong> while
still holding the key, and you cannot confirm the key was wiped.
</li>
<li>
<strong>Staff offboarding.</strong> Someone who could read the key
from a config or build pipeline is leaving; rotate as part of the
offboarding checklist.
</li>
<li>
<strong>Scheduled hygiene.</strong> Rotate on a routine cadence
(for example quarterly) so a silently leaked key has a bounded
lifetime.
</li>
<li>
<strong>Suspicious traffic.</strong> The Fleet view shows reports
from a host you do not recognize - revoke now, investigate, then
regenerate and redeploy to known-good endpoints.
</li>
</ul>
</div>
</section>
<!-- ====================================================================
Build site agent (placeholder)
================================================================= -->
<section class="settings-section">
<h3>Build site agent</h3>
<div class="card form-card">
<p class="field-hint">
Produce an instance-locked agent executable stamped with this server's
URL, the endpoint agent key, and the reporting cadence, so it deploys
with no manual configuration on the client.
</p>
<p class="field-hint">
<strong>Coming soon.</strong> The build pipeline is not wired up yet -
the C# agent runtime is still under development. Generate a key above
and configure it manually as the agent <code>X-API-Key</code> for now.
</p>
<button class="btn btn-secondary" type="button" disabled>
Build agent (coming soon)
</button>
</div>
</section>
<!-- ====================================================================
Agent configuration (placeholder)
================================================================= -->
<section class="settings-section">
<h3>Agent configuration</h3>
<div class="card form-card">
<p class="field-hint">
Define what the agent collects and how often: reporting cadence,
monitored processes and services, the curated Windows event list, and
whether unknown hosts self-adopt on first contact. These settings are
stamped into the build above so a deployed agent needs no local edits.
</p>
<p class="field-hint">
<strong>Coming soon.</strong> These controls arrive with the C# agent
runtime. Until then the agent uses its built-in defaults.
</p>
<button class="btn btn-secondary" type="button" disabled>
Edit configuration (coming soon)
</button>
</div>
</section>
</div>
</template>
<script setup>
import { ref, onMounted } from 'vue'
import api from '@/api'
const loading = ref(true)
const busy = ref(false)
const key = ref(null)
const envkeyfallback = ref(true)
const secret = ref('')
const copied = ref(false)
const message = ref('')
const error = ref('')
function formatDate(value) {
if (!value) return ''
const parsed = new Date(value)
return Number.isNaN(parsed.getTime()) ? value : parsed.toLocaleString()
}
function applyStatus(data) {
key.value = data.key || null
envkeyfallback.value = !!data.envkeyfallback
}
onMounted(async () => {
try {
const response = await api.get('/endpoints/agentkey')
applyStatus(response.data.data || {})
} catch (loadError) {
error.value = loadError.response?.status === 403
? 'Only an administrator can manage the endpoint agent key'
: 'Could not load the endpoint agent key status'
console.error(loadError)
} finally {
loading.value = false
}
})
async function regenerate() {
if (key.value &&
!window.confirm('Regenerate the key? The current key stops working '
+ 'immediately and every agent must be reconfigured.')) {
return
}
busy.value = true
message.value = ''
error.value = ''
secret.value = ''
copied.value = false
try {
const response = await api.post('/endpoints/agentkey/regenerate')
const data = response.data.data || {}
applyStatus(data)
secret.value = data.secret || ''
message.value = 'Endpoint agent key generated.'
} catch (regenError) {
error.value = regenError.response?.data?.data?.error?.message
|| 'Could not generate the endpoint agent key'
} finally {
busy.value = false
}
}
async function revoke() {
if (!window.confirm('Revoke the key? All endpoint-agent reporting stops '
+ 'until a new key is generated.')) {
return
}
busy.value = true
message.value = ''
error.value = ''
secret.value = ''
try {
const response = await api.delete('/endpoints/agentkey')
applyStatus(response.data.data || {})
message.value = 'Endpoint agent key revoked.'
} catch (revokeError) {
error.value = revokeError.response?.data?.data?.error?.message
|| 'Could not revoke the endpoint agent key'
} finally {
busy.value = false
}
}
async function copySecret() {
try {
await navigator.clipboard.writeText(secret.value)
copied.value = true
} catch (copyError) {
console.error(copyError)
}
}
</script>
<style scoped>
.mono { font-family: monospace; word-break: break-all; }
.form-card h3 { margin-top: 0; }
.form-card h4 { margin-top: 1rem; margin-bottom: 0.25rem; }
.form-card h4:first-child { margin-top: 0; }
.button-row { display: flex; gap: 0.5rem; }
.secret-card { border: 1px solid var(--color-warning, #b8860b); }
.secret-value { padding: 0.5rem; background: var(--color-surface-2, #1e1e1e); }
.settings-section { margin-bottom: 2rem; }
.settings-section > h3 { margin-bottom: 0.25rem; }
.info-list { margin: 0.25rem 0 0; padding-left: 1.25rem; }
.info-list li { margin-bottom: 0.5rem; }
</style>

153
plugins/endpoints/frontend/views/EndpointsList.vue Normal file
View File

@@ -0,0 +1,153 @@
<template>
<div>
<div class="page-header">
<h2>Fleet</h2>
</div>
<!-- Filters -->
<div class="filters">
<input
v-model="search"
type="text"
class="form-control"
placeholder="Search endpoints..."
@input="debouncedSearch"
/>
</div>
<div class="card">
<div v-if="loading" class="loading">Loading...</div>
<template v-else>
<div class="table-container">
<table>
<thead>
<tr>
<th>Hostname</th>
<th>Logged-in User</th>
<th>Uptime</th>
<th>CPU</th>
<th>Memory</th>
<th>Last Reported</th>
</tr>
</thead>
<tbody>
<tr
v-for="item in endpoints"
:key="item.assetid"
class="clickable-row"
:class="{ disabled: !item.computerid }"
@click="openPc(item)"
>
<td>{{ item.hostname || '-' }}</td>
<td>{{ item.loggedinuser || '-' }}</td>
<td>{{ item.uptimedisplay || '-' }}</td>
<td>{{ formatPercent(item.cpupercent) }}</td>
<td>{{ formatPercent(item.memorypercent) }}</td>
<td>{{ formatTime(item.lastreportedat) }}</td>
</tr>
<tr v-if="endpoints.length === 0">
<td colspan="6" style="text-align: center; color: var(--text-light);">
No endpoints found
</td>
</tr>
</tbody>
</table>
</div>
<!-- Pagination -->
<PaginationBar
:page="page"
:totalPages="totalPages"
:perPage="perPage"
@update:page="goToPage"
@update:perPage="changePerPage"
/>
</template>
</div>
</div>
</template>
<script setup>
import { ref, onMounted } from 'vue'
import { useRouter } from 'vue-router'
import { endpointsApi } from '@/api'
import PaginationBar from '@/components/PaginationBar.vue'
import { useListQuery } from '@/composables/listQuery'
const router = useRouter()
const endpoints = ref([])
const loading = ref(true)
const { page, search, setPage, setSearch } = useListQuery({ onChange: loadEndpoints })
const totalPages = ref(1)
const perPage = ref(20)
let searchTimeout = null
onMounted(() => {
loadEndpoints()
})
async function loadEndpoints() {
loading.value = true
try {
const params = {
page: page.value,
perpage: perPage.value
}
if (search.value) params.search = search.value
const response = await endpointsApi.list(params)
endpoints.value = response.data.data || []
totalPages.value = response.data.meta?.pagination?.totalpages || response.data.meta?.pagination?.total_pages || 1
} catch (error) {
console.error('Error loading endpoints:', error)
} finally {
loading.value = false
}
}
// Interoperate with PC Details instead of a bespoke detail page: a row only
// navigates when it resolved to a computerid.
function openPc(item) {
if (item.computerid) {
router.push(`/pcs/${item.computerid}`)
}
}
function formatPercent(value) {
if (value === null || value === undefined) return '-'
return `${Math.round(value)}%`
}
function formatTime(value) {
if (!value) return '-'
return new Date(value).toLocaleString()
}
function debouncedSearch() {
clearTimeout(searchTimeout)
searchTimeout = setTimeout(() => {
setSearch(search.value)
loadEndpoints()
}, 300)
}
function goToPage(p) {
setPage(p)
loadEndpoints()
}
function changePerPage(newPerPage) {
perPage.value = newPerPage
setPage(1)
loadEndpoints()
}
</script>
<style scoped>
.clickable-row.disabled {
cursor: default;
opacity: 0.6;
}
</style>

21
plugins/endpoints/manifest.json Normal file
View File

@@ -0,0 +1,21 @@
{
"name": "endpoints",
"version": "0.1.0",
"description": "Endpoint RMM telemetry: metrics, services, inventory, and curated Windows event records reported by the ShopDB agent",
"author": "ShopDB Team",
"dependencies": ["computers"],
"core_version": ">=0.20.0,<1.0.0",
"api_prefix": "/api/endpoints",
"provides": {
"features": [
"endpoint_metrics",
"service_state",
"inventory_snapshot",
"event_records"
]
},
"settings": {
"report_interval_minutes": 15,
"quiet_report_hours": 24
}
}

15
plugins/endpoints/migrations/env.py Normal file
View File

@@ -0,0 +1,15 @@
"""Alembic environment for the endpoints plugin migration chain.
Delegates to the shared runner in shopdb.plugins.alembic_template, which filters
the metadata to this plugin's tables and drives Alembic against the per-plugin
version table alembic_version_endpoints. See ADR-008 for the ownership model.
This plugin is NEW: its 0001 baseline really CREATES its tables, because the
core chain never built them.
"""
import os
os.environ['PLUGIN_NAME'] = 'endpoints'
from shopdb.plugins.alembic_template import run_migrations # noqa: E402
run_migrations()

24
plugins/endpoints/migrations/script.py.mako Normal file
View File

@@ -0,0 +1,24 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
def upgrade():
${upgrades if upgrades else "pass"}
def downgrade():
${downgrades if downgrades else "pass"}

96
plugins/endpoints/migrations/versions/0001_endpoints_baseline.py Normal file
View File

@@ -0,0 +1,96 @@
"""endpoints plugin baseline (real create).
A NEW plugin: the core chain never knew about endpointstates or endpointevents,
so this per-plugin chain is their sole authoritative creator.
Emits explicit Alembic ops rather than using the create_plugin_tables helper:
the helper builds a per-plugin MetaData filtered to the plugin's own tables, so
the foreign key to the core assets table cannot resolve at CreateTable-compile
time (NoReferencedTableError). Tables inherit the connection's default charset,
matching how the core chain creates its own.
"""
from alembic import op
from shopdb.plugins.alembic_template import (
create_index_if_missing,
create_table_if_missing,
)
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = 'endpoints0001baseline'
down_revision = None
branch_labels = None
depends_on = None
def upgrade():
create_table_if_missing(
'endpointstates',
sa.Column('endpointstateid', sa.Integer(), nullable=False),
sa.Column('assetid', sa.Integer(), nullable=False),
sa.Column('hostname', sa.String(length=100), nullable=True),
sa.Column('loggedinuser', sa.String(length=100), nullable=True),
sa.Column('agentversion', sa.String(length=40), nullable=True),
sa.Column('lastboottime', sa.DateTime(), nullable=True),
sa.Column('uptimeseconds', sa.BigInteger(), nullable=True),
sa.Column('idleseconds', sa.BigInteger(), nullable=True),
sa.Column('cpupercent', sa.Float(), nullable=True),
sa.Column('memorypercent', sa.Float(), nullable=True),
sa.Column('servicesjson', sa.Text(length=16777215), nullable=True),
sa.Column('portsjson', sa.Text(length=16777215), nullable=True),
sa.Column('interfacesjson', sa.Text(length=16777215), nullable=True),
sa.Column('scheduledtasksjson', sa.Text(length=16777215),
nullable=True),
sa.Column('lastreportedat', sa.DateTime(), nullable=True),
sa.Column('createdat', sa.DateTime(), nullable=False),
sa.ForeignKeyConstraint(['assetid'], ['assets.assetid'],
ondelete='CASCADE'),
sa.PrimaryKeyConstraint('endpointstateid'),
sa.UniqueConstraint('assetid'),
)
create_index_if_missing('ix_endpointstates_assetid',
'endpointstates', ['assetid'])
create_index_if_missing('ix_endpointstates_lastreportedat',
'endpointstates', ['lastreportedat'])
create_table_if_missing(
'endpointevents',
sa.Column('endpointeventid', sa.Integer(), nullable=False),
sa.Column('assetid', sa.Integer(), nullable=False),
sa.Column('eventid', sa.Integer(), nullable=False),
sa.Column('eventchannel', sa.String(length=50), nullable=True),
sa.Column('eventsource', sa.String(length=255), nullable=True),
sa.Column('eventtime', sa.DateTime(), nullable=True),
sa.Column('eventmessage', sa.Text(), nullable=True),
sa.Column('payloadjson', sa.Text(length=16777215), nullable=True),
sa.Column('recordedat', sa.DateTime(), nullable=False),
sa.ForeignKeyConstraint(['assetid'], ['assets.assetid'],
ondelete='CASCADE'),
sa.PrimaryKeyConstraint('endpointeventid'),
)
create_index_if_missing('ix_endpointevents_assetid', 'endpointevents',
['assetid'])
create_index_if_missing('ix_endpointevents_eventid', 'endpointevents',
['eventid'])
create_index_if_missing('ix_endpointevents_eventtime', 'endpointevents',
['eventtime'])
# Alerting/history reads by "this endpoint's events over time", so the
# composite is the index that actually gets used.
create_index_if_missing('idx_endpointevents_assetevent', 'endpointevents',
['assetid', 'eventtime'])
def downgrade():
op.drop_index('idx_endpointevents_assetevent',
table_name='endpointevents')
op.drop_index('ix_endpointevents_eventtime', table_name='endpointevents')
op.drop_index('ix_endpointevents_eventid', table_name='endpointevents')
op.drop_index('ix_endpointevents_assetid', table_name='endpointevents')
op.drop_table('endpointevents')
op.drop_index('ix_endpointstates_lastreportedat',
table_name='endpointstates')
op.drop_index('ix_endpointstates_assetid', table_name='endpointstates')
op.drop_table('endpointstates')

84
plugins/endpoints/migrations/versions/0002_endpoint_commands.py Normal file
View File

@@ -0,0 +1,84 @@
"""endpoint commands queue + cached MTConnect state.
Adds the endpointcommands queue (one WebUI-issued, server-allowlisted action
per row, claimed and finalized by the agent long-poll) and two cached MTConnect
run-state columns on endpointstates so the WebUI toggle can show a real state
between command polls.
Idempotent: create_table_if_missing guards the new table, and the column adds
check the live schema first so a db.create_all() site (which already built the
columns) re-runs clean on strict MySQL 8.
"""
from alembic import op
from shopdb.plugins.alembic_template import (
create_index_if_missing,
create_table_if_missing,
)
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = 'endpoints0002commands'
down_revision = 'endpoints0001baseline'
branch_labels = None
depends_on = None
def _has_column(table_name, column_name):
from sqlalchemy import inspect
inspector = inspect(op.get_bind())
if not inspector.has_table(table_name):
return False
return any(col['name'] == column_name
for col in inspector.get_columns(table_name))
def upgrade():
if not _has_column('endpointstates', 'mtconnectrunning'):
op.add_column('endpointstates',
sa.Column('mtconnectrunning', sa.Boolean(),
nullable=True))
if not _has_column('endpointstates', 'mtconnectcheckedat'):
op.add_column('endpointstates',
sa.Column('mtconnectcheckedat', sa.DateTime(),
nullable=True))
create_table_if_missing(
'endpointcommands',
sa.Column('endpointcommandid', sa.Integer(), nullable=False),
sa.Column('assetid', sa.Integer(), nullable=False),
sa.Column('action', sa.String(length=64), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('requestedby', sa.Integer(), nullable=True),
sa.Column('resultmessage', sa.Text(), nullable=True),
sa.Column('resultjson', sa.Text(length=16777215), nullable=True),
sa.Column('createdat', sa.DateTime(), nullable=False),
sa.Column('claimedat', sa.DateTime(), nullable=True),
sa.Column('completedat', sa.DateTime(), nullable=True),
sa.ForeignKeyConstraint(['assetid'], ['assets.assetid'],
ondelete='CASCADE'),
sa.PrimaryKeyConstraint('endpointcommandid'),
)
create_index_if_missing('ix_endpointcommands_assetid',
'endpointcommands', ['assetid'])
create_index_if_missing('ix_endpointcommands_status',
'endpointcommands', ['status'])
create_index_if_missing('idx_endpointcommands_assetstatus',
'endpointcommands', ['assetid', 'status'])
def downgrade():
op.drop_index('idx_endpointcommands_assetstatus',
table_name='endpointcommands')
op.drop_index('ix_endpointcommands_status',
table_name='endpointcommands')
op.drop_index('ix_endpointcommands_assetid',
table_name='endpointcommands')
op.drop_table('endpointcommands')
if _has_column('endpointstates', 'mtconnectcheckedat'):
op.drop_column('endpointstates', 'mtconnectcheckedat')
if _has_column('endpointstates', 'mtconnectrunning'):
op.drop_column('endpointstates', 'mtconnectrunning')

50
plugins/endpoints/migrations/versions/0003_endpoint_command_target.py Normal file
View File

@@ -0,0 +1,50 @@
"""endpoint command target for generic service/process actions.
Adds nullable target and targetkind columns to endpointcommands so generic
service_* / process_* actions can carry their subject (a Windows service name or
a process image like 'DncMain.exe') over the wire. Fixed-target actions
(mtconnect_*) leave both null. This is the schema half of making the agent
data-driven: a site builds the agent once and every future policy/app is server
data, no agent redistribution.
Idempotent: each add checks the live schema first so a db.create_all() site
(which already built the columns) re-runs clean on strict MySQL 8.
"""
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = 'endpoints0003cmdtarget'
down_revision = 'endpoints0002commands'
branch_labels = None
depends_on = None
def _has_column(table_name, column_name):
from sqlalchemy import inspect
inspector = inspect(op.get_bind())
if not inspector.has_table(table_name):
return False
return any(col['name'] == column_name
for col in inspector.get_columns(table_name))
def upgrade():
if not _has_column('endpointcommands', 'target'):
op.add_column('endpointcommands',
sa.Column('target', sa.String(length=255),
nullable=True))
if not _has_column('endpointcommands', 'targetkind'):
op.add_column('endpointcommands',
sa.Column('targetkind', sa.String(length=16),
nullable=True))
def downgrade():
if _has_column('endpointcommands', 'targetkind'):
op.drop_column('endpointcommands', 'targetkind')
if _has_column('endpointcommands', 'target'):
op.drop_column('endpointcommands', 'target')

5
plugins/endpoints/models/__init__.py Normal file
View File

@@ -0,0 +1,5 @@
"""Endpoints plugin models."""
from .endpoint import EndpointState, EndpointEvent, EndpointCommand
__all__ = ['EndpointState', 'EndpointEvent', 'EndpointCommand']

248
plugins/endpoints/models/endpoint.py Normal file
View File

@@ -0,0 +1,248 @@
"""Endpoint RMM telemetry models.
Two shapes, one per concern:
EndpointState The cached CURRENT snapshot for an asset - one row per asset,
overwritten in place on every agent report. Holds the fast-
moving metrics (uptime, idle, last boot, logged-in user) and
the latest inventory snapshots (services/processes, open ports,
interfaces, scheduled tasks) as serialized JSON. This is the
cached-current-value half of the ledger pattern: it moves in
the SAME commit as any EndpointEvent rows the report carries.
EndpointEvent Append-only record of a curated Windows Event Log entry. The
agent only forwards a fixed catalog of IDs (logon/logoff,
privilege use, service install, unexpected shutdown, DCOM,
crash) so this stays a readable security/stability history
rather than a raw event firehose.
Both link to the core Asset via assetid (FK, ON DELETE CASCADE), so an endpoint
is the same physical thing as its Computer/Asset - no duplicate identity. The
agent reports a hostname; the plugin resolves it to the existing Computer.
"""
from datetime import datetime
from shopdb.api import db
def _utciso(value):
"""Naive-UTC datetime -> ISO string marked UTC, or None.
Every datetime this plugin stores is naive UTC. Serializing without the Z
leaves the receiver to guess, and JavaScript guesses browser-local.
"""
return value.isoformat() + 'Z' if value else None
class EndpointState(db.Model):
"""Latest reported state for a single endpoint (one row per asset)."""
__tablename__ = 'endpointstates'
endpointstateid = db.Column(db.Integer, primary_key=True)
# The asset this state belongs to. Unique: exactly one current snapshot per
# endpoint. Resolved from the reported hostname against the computers plugin.
assetid = db.Column(
db.Integer,
db.ForeignKey('assets.assetid', ondelete='CASCADE'),
unique=True,
nullable=False,
index=True,
)
# Read-only view of the owning asset. No cascade or backref: the asset side
# must not gain a dependency on this plugin.
asset = db.relationship('Asset', lazy='joined', viewonly=True)
hostname = db.Column(db.String(100), nullable=True)
loggedinuser = db.Column(db.String(100), nullable=True)
agentversion = db.Column(db.String(40), nullable=True)
# Fast-moving metrics
lastboottime = db.Column(db.DateTime, nullable=True)
uptimeseconds = db.Column(db.BigInteger, nullable=True)
idleseconds = db.Column(db.BigInteger, nullable=True)
cpupercent = db.Column(db.Float, nullable=True)
memorypercent = db.Column(db.Float, nullable=True)
# Latest inventory snapshots, serialized JSON (MEDIUMTEXT). Kept as the
# newest snapshot rather than one row per sample: the agent reports the full
# list each cycle and history of these lists is not what operators ask for.
servicesjson = db.Column(db.Text(length=16777215), nullable=True)
portsjson = db.Column(db.Text(length=16777215), nullable=True)
interfacesjson = db.Column(db.Text(length=16777215), nullable=True)
scheduledtasksjson = db.Column(db.Text(length=16777215), nullable=True)
# Last-known MTConnect agent run state, cached so the WebUI toggle can show
# a real state between command polls. Set by the agent's mtconnect_status
# result. Null = never reported (unknown).
mtconnectrunning = db.Column(db.Boolean, nullable=True)
mtconnectcheckedat = db.Column(db.DateTime, nullable=True)
# When the agent last successfully reported. There is no heartbeat separate
# from the report, so silence past the quiet-hours window is the only
# evidence an endpoint has stopped checking in.
lastreportedat = db.Column(db.DateTime, nullable=True, index=True)
createdat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
def __repr__(self):
return f"<EndpointState {self.hostname or self.assetid}>"
def to_dict(self):
return {
'endpointstateid': self.endpointstateid,
'assetid': self.assetid,
'hostname': self.hostname,
'loggedinuser': self.loggedinuser,
'agentversion': self.agentversion,
'lastboottime': _utciso(self.lastboottime),
'uptimeseconds': self.uptimeseconds,
'idleseconds': self.idleseconds,
'cpupercent': self.cpupercent,
'memorypercent': self.memorypercent,
'mtconnectrunning': self.mtconnectrunning,
'mtconnectcheckedat': _utciso(self.mtconnectcheckedat),
'lastreportedat': _utciso(self.lastreportedat),
'createdat': _utciso(self.createdat),
}
class EndpointCommand(db.Model):
"""A queued operator command targeting one endpoint agent.
The WebUI enqueues a named action (server allowlist only, never an arbitrary
command string) for an asset. The agent claims the next pending command on
its long-poll, executes it locally, and reports the outcome back, which
moves the row to a terminal status in the same request. This is a small
at-most-one-in-flight queue, not a general job system.
"""
__tablename__ = 'endpointcommands'
# Lifecycle: pending -> running -> succeeded|failed. A command is claimed
# (pending -> running) by exactly one agent poll, then finalized by its
# result post.
STATUS_PENDING = 'pending'
STATUS_RUNNING = 'running'
STATUS_SUCCEEDED = 'succeeded'
STATUS_FAILED = 'failed'
endpointcommandid = db.Column(db.Integer, primary_key=True)
assetid = db.Column(
db.Integer,
db.ForeignKey('assets.assetid', ondelete='CASCADE'),
nullable=False,
index=True,
)
asset = db.relationship('Asset', lazy='joined', viewonly=True)
# The allowlisted action name (for example 'mtconnect_start'). Validated
# against the server allowlist at enqueue time, so only known names land.
action = db.Column(db.String(64), nullable=False)
# Generic actions (service_* / process_*) carry their subject over the wire:
# target is a Windows service name or a process image (for example
# 'DncMain.exe'); targetkind is 'service' or 'process'. Fixed-target actions
# (mtconnect_*) leave both null. This is what lets a site build the agent
# once and drive every future policy/app purely from server data.
target = db.Column(db.String(255), nullable=True)
targetkind = db.Column(db.String(16), nullable=True)
status = db.Column(db.String(20), nullable=False,
default=STATUS_PENDING, index=True)
# Who queued it (userid) for audit; the agent never sees this.
requestedby = db.Column(db.Integer, nullable=True)
# Agent-reported outcome. resultjson holds structured detail (for example
# the observed MTConnect running state); resultmessage is a short summary.
resultmessage = db.Column(db.Text, nullable=True)
resultjson = db.Column(db.Text(length=16777215), nullable=True)
createdat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
claimedat = db.Column(db.DateTime, nullable=True)
completedat = db.Column(db.DateTime, nullable=True)
__table_args__ = (
db.Index('idx_endpointcommands_assetstatus', 'assetid', 'status'),
)
def __repr__(self):
return f"<EndpointCommand {self.action} asset={self.assetid} {self.status}>"
def to_dict(self):
return {
'endpointcommandid': self.endpointcommandid,
'assetid': self.assetid,
'action': self.action,
'target': self.target,
'targetkind': self.targetkind,
'status': self.status,
'requestedby': self.requestedby,
'resultmessage': self.resultmessage,
'createdat': _utciso(self.createdat),
'claimedat': _utciso(self.claimedat),
'completedat': _utciso(self.completedat),
}
class EndpointEvent(db.Model):
"""A single curated Windows Event Log record forwarded by the agent."""
__tablename__ = 'endpointevents'
endpointeventid = db.Column(db.Integer, primary_key=True)
assetid = db.Column(
db.Integer,
db.ForeignKey('assets.assetid', ondelete='CASCADE'),
nullable=False,
index=True,
)
asset = db.relationship('Asset', lazy='joined', viewonly=True)
# The Windows Event ID (for example 4625 failed logon, 41 unexpected
# shutdown, 1001 application crash). Indexed because alerting queries filter
# on it.
eventid = db.Column(db.Integer, nullable=False, index=True)
# Event Log channel the record came from: Security, System, Application.
eventchannel = db.Column(db.String(50), nullable=True)
eventsource = db.Column(db.String(255), nullable=True)
# When the event happened on the endpoint (from the record), not when it
# reached ShopDB.
eventtime = db.Column(db.DateTime, nullable=True, index=True)
eventmessage = db.Column(db.Text, nullable=True)
# Structured fields the agent extracted for this ID (for example the target
# account on a 4625). Advisory index only, not part of dedup.
payloadjson = db.Column(db.Text(length=16777215), nullable=True)
recordedat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
__table_args__ = (
db.Index('idx_endpointevents_assetevent', 'assetid', 'eventtime'),
)
def __repr__(self):
return f"<EndpointEvent {self.eventid} asset={self.assetid}>"
def to_dict(self):
return {
'endpointeventid': self.endpointeventid,
'assetid': self.assetid,
'eventid': self.eventid,
'eventchannel': self.eventchannel,
'eventsource': self.eventsource,
'eventtime': _utciso(self.eventtime),
'eventmessage': self.eventmessage,
'recordedat': _utciso(self.recordedat),
}

596
plugins/endpoints/plugin.py Normal file
View File

@@ -0,0 +1,596 @@
"""Endpoints plugin main class.
Endpoint RMM telemetry reported by the ShopDB agent. Accepts collector input
over HTTPS per ADR-006: the agent posts a per-host payload to
POST /api/collector/endpoints, the plugin resolves the reported hostname to an
existing computer asset, overwrites that asset's cached EndpointState, and
appends any curated Windows events the report carried - all in one commit, so
the cached snapshot and the events it summarizes never disagree.
Identity is the hostname, resolved against the computers plugin. An unresolved
hostname is reported as a ValueError (surfaced to the caller as a validation
error), never silently dropped: an unknown host is a real gap between the fleet
and ShopDB, the same discipline the backups collector uses for machine numbers.
"""
import json
import logging
from datetime import datetime
from pathlib import Path
from typing import Dict, List, Optional, Type
from flask import Flask, Blueprint
from shopdb.plugins.base import BasePlugin, PluginMeta
from shopdb.api import db
from .api import endpoints_bp
from .eventcatalog import EVENT_CATALOG
from .models import EndpointState, EndpointEvent, EndpointCommand
logger = logging.getLogger(__name__)
# The fixed catalog of Windows Event IDs the agent forwards. Derived from
# EVENT_CATALOG (eventcatalog.py) so the ingest filter and the display titles
# can never drift. IDs outside this set are ignored on ingest rather than
# stored, keeping endpointevents a readable security/stability history instead
# of a raw event firehose.
CURATED_EVENT_IDS = frozenset(EVENT_CATALOG)
def _parse_dt(value):
"""Best-effort parse of an ISO-8601 string to naive UTC, or None."""
if not value:
return None
if isinstance(value, datetime):
return value.replace(tzinfo=None)
text = str(value).strip().replace('Z', '+00:00')
try:
parsed = datetime.fromisoformat(text)
except ValueError:
return None
if parsed.tzinfo is not None:
parsed = parsed.astimezone(tz=None).replace(tzinfo=None)
return parsed
def _as_json(value):
"""Serialize a list/dict to JSON text, or None when nothing was sent."""
if value is None:
return None
return json.dumps(value, separators=(',', ':'), sort_keys=True)
class EndpointsPlugin(BasePlugin):
"""Endpoint RMM telemetry ingest plus read/dashboard surface."""
def __init__(self):
self._manifest = self._load_manifest()
def _load_manifest(self) -> Dict:
manifest_path = Path(__file__).parent / 'manifest.json'
if manifest_path.exists():
with open(manifest_path, 'r') as f:
return json.load(f)
return {}
@property
def meta(self) -> PluginMeta:
return PluginMeta(
name=self._manifest.get('name', 'endpoints'),
version=self._manifest.get('version', '0.1.0'),
description=self._manifest.get(
'description', 'Endpoint RMM telemetry'),
author=self._manifest.get('author', 'ShopDB Team'),
dependencies=self._manifest.get('dependencies', ['computers']),
core_version=self._manifest.get('core_version', '>=0.20.0,<1.0.0'),
api_prefix=self._manifest.get('api_prefix', '/api/endpoints'),
)
def get_blueprint(self) -> Optional[Blueprint]:
return endpoints_bp
def get_models(self) -> List[Type]:
return [EndpointState, EndpointEvent, EndpointCommand]
def get_permissions(self) -> List:
"""RBAC for the human-facing read routes.
View only. Ingest is not covered here: the collector path authorizes on
a collector.ingest-scoped service token, not a user JWT. There is no
write/delete permission because operators do not edit telemetry - the
agent is the sole author, and history is pruned by policy, not by hand.
"""
return [
('endpoints.view', 'View endpoint telemetry', 'endpoints'),
# Issue an allowlisted command to an endpoint agent (for example the
# MTConnect toggle). Separate from endpoints.view because acting on a
# shop-floor PC is a heavier grant than reading its telemetry.
('endpoints.command',
'Send allowlisted commands to endpoint agents', 'endpoints'),
# Elevated grant for destructive endpoint actions (system_restart).
# Required IN ADDITION to endpoints.command by the enqueue route, so a
# remote reboot is a strictly heavier grant than a routine toggle.
('endpoints.restart',
'Restart endpoint computers (destructive)', 'endpoints'),
# Collector service scope for the endpoint agent. A managed token
# scoped to ONLY this authorizes the endpoints collector and nothing
# else, so rotating/revoking it stops endpoint-agent traffic without
# touching computers/geenforce collectors. Minted from the settings
# UI (see get_settings_cards). Admins hold it via the catalog.
('collector.ingest.endpoints',
'Submit endpoint agent telemetry (fleet reporting)', 'collector'),
]
def get_settings_defaults(self) -> List[Dict]:
"""Settings owned by the endpoints agent-key kill switch.
`envkeyfallback` gates the bootstrap .env collector key: while true the
old COLLECTOR_API_KEY[_ENDPOINTS] still authorizes (back-compat, and the
only path before a UI key exists). The regenerate flow flips it false so
the .env key stops working and the managed token is the sole credential,
making revocation an instant kill switch. `agentkeyprovisioned` records
that a UI-managed key has been minted so the settings page can render
the rotate/revoke state instead of the first-run generate state.
"""
return [
{
'key': 'plugin.endpoints.envkeyfallback',
'value': 'true',
'valuetype': 'boolean',
'category': 'plugin.endpoints',
'description': 'Accept the bootstrap .env collector key for '
'endpoints. Turned off once a UI-managed agent '
'key is provisioned so revocation is a real kill '
'switch.',
},
{
'key': 'plugin.endpoints.agentkeyprovisioned',
'value': 'false',
'valuetype': 'boolean',
'category': 'plugin.endpoints',
'description': 'True once an operator has minted a UI-managed '
'endpoint agent key from the settings page.',
},
]
def get_settings_cards(self) -> List[Dict]:
"""The endpoints management card on the global settings page.
A settings card rather than a docs page because it is the operator's
control surface: manage the agent key (the kill switch), build the site
agent, and configure what it reports. Admin only - key management mints
a collector service credential.
"""
return [
{
'group': 'Endpoints',
'to': '/settings/endpoints/agentkey',
'icon': 'key',
'title': 'Endpoints',
'description': 'Manage the endpoint agent key, build the site '
'agent, and configure what it reports.',
'position': 27,
},
]
def get_navigation_items(self) -> List[Dict]:
"""Sidebar entry for the fleet list view.
Sits just after the PCs item (position 15) because the fleet view is
a telemetry lens over the same computers.
"""
return [
{
'name': 'Fleet',
'icon': 'activity',
'route': '/endpoints',
'position': 16,
'permission': 'endpoints.view',
'section': 'fleet',
},
]
def init_app(self, app: Flask, db_instance) -> None:
logger.info('Endpoints plugin initialized (v%s)', self.meta.version)
# ---- ADR-010 UI hooks -----------------------------------------------
def get_asset_panels(self) -> List[Dict]:
"""Per-PC detail panels, computer assets only.
Panels feeding the generic renderer: the current status snapshot
(keyvalue), monitored services and processes split into two list cards
that share one services snapshot (filtered by kind), the most recent
curated events (list), and the network interfaces (table). Each is a
single-asset indexed read, so the cost is the same profile as the
warranty/backups panels. Panels hide themselves when their endpoint
returns nothing.
"""
return [
{
'id': 'endpoints-status',
'title': 'Endpoint status',
'assettypes': ['computer'],
'endpoint': '/api/endpoints/asset/{assetid}/status',
'render': 'keyvalue',
'empty': 'No agent report yet.',
'position': 10,
},
{
'id': 'endpoints-tracked-services',
'title': 'Tracked Services',
'assettypes': ['computer'],
'endpoint': '/api/endpoints/asset/{assetid}/services',
'render': 'list',
'collapsible': True,
'filter': {'key': 'kind', 'equals': 'service'},
'status': {
'key': 'statecolor',
'levels': {
'green': 'ok',
'yellow': 'warning',
'red': 'critical',
},
},
'map': {
'title': 'name',
'badge': {'label': 'statelabel', 'color': 'statecolor'},
'action': 'action',
},
'commandendpoint': '/api/endpoints/{assetid}/commands',
'permission': 'endpoints.command',
'empty': 'No monitored services reported.',
'position': 12,
},
{
'id': 'endpoints-tracked-processes',
'title': 'Tracked Processes',
'assettypes': ['computer'],
'endpoint': '/api/endpoints/asset/{assetid}/services',
'render': 'list',
'collapsible': True,
'filter': {'key': 'kind', 'equals': 'process'},
'status': {
'key': 'statecolor',
'levels': {
'green': 'ok',
'yellow': 'warning',
'red': 'critical',
},
},
'map': {
'title': 'name',
'badge': {'label': 'statelabel', 'color': 'statecolor'},
'action': 'action',
},
'commandendpoint': '/api/endpoints/{assetid}/commands',
'permission': 'endpoints.command',
'empty': 'No monitored processes reported.',
'position': 13,
},
{
'id': 'endpoints-events',
'title': 'Recent events',
'assettypes': ['computer'],
'endpoint': '/api/endpoints/asset/{assetid}/recentevents',
'render': 'list',
'collapsible': True,
'status': {
'key': 'severity',
'levels': {
'info': 'ok',
'warning': 'warning',
'critical': 'critical',
},
},
'map': {
'title': 'title',
'badge': {
'label': 'severity',
'labelmap': {
'critical': 'Critical',
'warning': 'Warning',
'info': 'Info',
},
},
'meta': [
{'key': 'eventtime', 'label': 'At', 'format': 'date'},
],
},
'empty': 'No recent events.',
'collapsedDefault': True,
'position': 90,
},
{
'id': 'endpoints-interfaces',
'title': 'Network interfaces',
'assettypes': ['computer'],
'endpoint': '/api/endpoints/asset/{assetid}/interfaces',
'render': 'table',
'collapsible': True,
'columns': [
{'key': 'alias', 'label': 'Adapter'},
{'key': 'ipv4', 'label': 'IPv4'},
{'key': 'mac', 'label': 'MAC'},
{'key': 'gateway', 'label': 'Gateway'},
{'key': 'subnet', 'label': 'Subnet'},
{'key': 'dhcp', 'label': 'Assignment'},
{'key': 'role', 'label': 'Role'},
],
'empty': 'No network interfaces reported.',
'position': 16,
},
]
def get_dashboard_widgets(self) -> List[Dict]:
"""Fleet-wide exception cards. Run once on dashboard load, not per PC.
Both cards hide when empty and cap their rows, so a healthy fleet adds
nothing to the dashboard and an unhealthy one adds a bounded list.
"""
return [
{
'id': 'endpoints-quiet',
'viewall': '/pcs',
'title': 'Endpoints gone quiet',
'endpoint': '/api/endpoints/dashboard/quiet',
'render': 'exceptions',
'severity': 'warning',
'permission': 'endpoints.view',
'empty': 'hide',
'position': 22,
'map': {
'title': 'hostname',
'detail': '',
'meta': [{'key': 'quietdays', 'label': 'silent',
'suffix': ' days'}],
'link': '/pcs/{computerid}',
},
},
{
'id': 'endpoints-stability',
'viewall': '/pcs',
'title': 'Endpoints with crashes or shutdowns',
'endpoint': '/api/endpoints/dashboard/stability',
'render': 'exceptions',
'severity': 'critical',
'permission': 'endpoints.view',
'empty': 'hide',
'position': 16,
'map': {
'title': 'hostname',
'detail': '',
'meta': [{'key': 'eventcount', 'suffix': ' events (48h)'}],
'link': '/pcs/{computerid}',
},
},
]
# ---- ADR-006 collector contract -------------------------------------
def get_collector_schema(self) -> Optional[Dict]:
return {
'identityfield': 'hostname',
'fields': {
'type': 'object',
'required': ['hostname'],
'properties': {
'hostname': {
'type': 'string',
'description': 'Endpoint hostname. Resolved against '
'the computers plugin (computers.hostname).',
},
'adopt': {
'type': 'boolean',
'description': 'First-contact only. When true and the '
'hostname does not resolve, adoption is '
'delegated to the computers collector '
'(see DESIGN-DECISIONS-agent.md 1a). This '
'plugin never creates assets itself.',
},
'loggedinuser': {'type': 'string'},
'agentversion': {'type': 'string'},
'lastboottime': {'type': 'string', 'format': 'date-time'},
'uptimeseconds': {'type': 'integer'},
'idleseconds': {'type': 'integer'},
'cpupercent': {'type': 'number'},
'memorypercent': {'type': 'number'},
'services': {
'type': 'array',
'description': 'Monitored services/processes and their '
'run state (UDC, eDnc, Intune, ...).',
},
'ports': {
'type': 'array',
'description': 'Open listening ports.',
},
'interfaces': {
'type': 'array',
'description': 'Network interface information.',
},
'scheduledtasks': {
'type': 'array',
'description': 'Scheduled task inventory + last run.',
},
'events': {
'type': 'array',
'description': 'Curated Windows Event Log records. Only '
'IDs in the server catalog are stored.',
'items': {
'type': 'object',
'required': ['eventid'],
'properties': {
'eventid': {'type': 'integer'},
'eventchannel': {'type': 'string'},
'eventsource': {'type': 'string'},
'eventtime': {'type': 'string',
'format': 'date-time'},
'eventmessage': {'type': 'string'},
'fields': {'type': 'object'},
},
},
},
},
},
}
# PC-identifying and hardware fields forwarded to the computers collector,
# which owns the Asset/Computer row and renders the Hardware and Network
# cards. The endpoints plugin only caches telemetry, so these must be
# relayed or vendor/model/OS/IP never land on the PC (see
# DESIGN-DECISIONS-agent.md 1a).
_COMPUTER_HANDOFF_FIELDS = (
'loggedinuser', 'serialnumber', 'pctype', 'machinenumber',
'vendorname', 'modelnumber', 'osname', 'ipaddress', 'lastboottime',
)
def _forward_to_computers(self, hostname: str, payload: Dict):
"""Return the computers plugin and the PC subset to hand off, or None.
Both adoption and steady-state reports relay the same hardware subset,
so the Hardware and Network cards fill in on every report, not just at
first contact.
"""
from flask import current_app
manager = current_app.extensions.get('plugin_manager')
computers = manager.get_plugin('computers') if manager else None
if computers is None:
return None, None
subset = {'hostname': hostname}
for field in self._COMPUTER_HANDOFF_FIELDS:
if payload.get(field):
subset[field] = payload.get(field)
return computers, subset
def _adopt_computer(self, hostname: str, payload: Dict,
warnings: List[str]) -> None:
"""Delegate first-contact adoption to the computers collector.
This plugin never creates Asset or Computer rows itself (see
DESIGN-DECISIONS-agent.md 1a). The computers plugin is the sole
authority for that, so on an unknown hostname we forward the
PC-identifying subset of the payload to its collector. It runs inside
the same transaction, so a later failure rolls the adoption back.
"""
computers, adoptpayload = self._forward_to_computers(hostname, payload)
if computers is None:
raise ValueError(
'cannot adopt {!r}: computers plugin is not available'
.format(hostname))
computers.apply_collector_payload(adoptpayload)
db.session.flush()
warnings.append('adopted new computer for hostname: {!r}'.format(hostname))
def apply_collector_payload(self, payload: Dict) -> Dict:
"""Idempotently upsert one endpoint's state and append its events.
The cached EndpointState and any EndpointEvent rows are written in the
same transaction (the dispatcher owns the commit), so a reader never
sees a snapshot without the events that produced it.
"""
from plugins.computers.models import Computer
warnings: List[str] = []
hostname = (payload.get('hostname') or '').strip()
if not hostname:
raise ValueError('hostname is required')
computer = (Computer.query
.filter(db.func.lower(Computer.hostname) == hostname.lower())
.first())
if computer is None:
# First-contact adoption is delegated to the computers collector,
# which is the only authority allowed to create Asset + Computer
# rows (DESIGN-DECISIONS-agent.md 1a). This plugin never invents
# assets itself. Adoption runs in the dispatcher's transaction, so a
# later ValueError rolls the whole report back.
if payload.get('adopt'):
self._adopt_computer(hostname, payload, warnings)
computer = (Computer.query
.filter(db.func.lower(Computer.hostname)
== hostname.lower())
.first())
if computer is None:
raise ValueError(
'could not resolve hostname to a computer: {!r}'.format(hostname))
else:
# Already adopted: relay the hardware/identity subset on every
# report so the computers plugin keeps vendor/model/OS/IP current.
# Without this, Hardware and Network stay blank after adoption
# because this plugin only caches telemetry.
computers, subset = self._forward_to_computers(hostname, payload)
if computers is not None and len(subset) > 1:
computers.apply_collector_payload(subset)
db.session.flush()
assetid = computer.assetid
state = EndpointState.query.filter_by(assetid=assetid).first()
action = 'updated'
if state is None:
state = EndpointState(assetid=assetid)
db.session.add(state)
action = 'created'
state.hostname = hostname
if 'loggedinuser' in payload:
state.loggedinuser = payload.get('loggedinuser')
if 'agentversion' in payload:
state.agentversion = payload.get('agentversion')
if 'lastboottime' in payload:
state.lastboottime = _parse_dt(payload.get('lastboottime'))
if 'uptimeseconds' in payload:
state.uptimeseconds = payload.get('uptimeseconds')
if 'idleseconds' in payload:
state.idleseconds = payload.get('idleseconds')
if 'cpupercent' in payload:
state.cpupercent = payload.get('cpupercent')
if 'memorypercent' in payload:
state.memorypercent = payload.get('memorypercent')
if 'services' in payload:
state.servicesjson = _as_json(payload.get('services'))
if 'ports' in payload:
state.portsjson = _as_json(payload.get('ports'))
if 'interfaces' in payload:
state.interfacesjson = _as_json(payload.get('interfaces'))
if 'scheduledtasks' in payload:
state.scheduledtasksjson = _as_json(payload.get('scheduledtasks'))
state.lastreportedat = datetime.utcnow()
stored_events = 0
for record in (payload.get('events') or []):
try:
eventid = int(record.get('eventid'))
except (TypeError, ValueError):
warnings.append('skipped event with non-integer eventid')
continue
if eventid not in CURATED_EVENT_IDS:
# Outside the catalog: ignored on purpose, not an error.
continue
db.session.add(EndpointEvent(
assetid=assetid,
eventid=eventid,
eventchannel=record.get('eventchannel'),
eventsource=record.get('eventsource'),
eventtime=_parse_dt(record.get('eventtime')),
eventmessage=record.get('eventmessage'),
payloadjson=_as_json(record.get('fields')),
))
stored_events += 1
# flush, not commit: the collector dispatcher owns the transaction.
db.session.flush()
return {
'action': action,
'assetid': assetid,
'identityvalue': hostname,
'storedevents': stored_events,
'warnings': warnings,
}

171
plugins/endpoints/tests/test_plugin.py Normal file
View File

@@ -0,0 +1,171 @@
"""Endpoints plugin smoke + collector-contract tests.
Asserts the plugin loads cleanly, satisfies the framework contract, and that
its ADR-006 collector schema is well formed. Extend with DB-backed ingest tests
as the plugin grows.
"""
from flask import Blueprint
from plugins.endpoints.plugin import EndpointsPlugin, CURATED_EVENT_IDS
from plugins.endpoints.eventcatalog import EVENT_CATALOG, CRITICAL_EVENT_IDS
from plugins.endpoints.commands import (
COMMAND_ALLOWLIST, is_allowed_action, elevated_permission_for,
requires_target,
)
def test_endpoints_plugin_meta_is_valid():
"""meta returns a PluginMeta with the expected name and prefix."""
plugin = EndpointsPlugin()
assert plugin.meta.name == 'endpoints'
assert plugin.meta.api_prefix == '/api/endpoints'
assert 'computers' in plugin.meta.dependencies
def test_endpoints_plugin_get_blueprint_returns_blueprint():
"""get_blueprint returns a Flask Blueprint, not None."""
plugin = EndpointsPlugin()
assert isinstance(plugin.get_blueprint(), Blueprint)
def test_endpoints_plugin_get_models_returns_models():
"""get_models returns the two telemetry tables."""
plugin = EndpointsPlugin()
models = plugin.get_models()
assert len(models) >= 2
for model in models:
assert hasattr(model, '__tablename__')
def test_endpoints_collector_schema_identity_is_hostname():
"""ADR-006 schema declares hostname as the identity field."""
plugin = EndpointsPlugin()
schema = plugin.get_collector_schema()
assert schema is not None
assert schema['identityfield'] == 'hostname'
assert 'hostname' in schema['fields']['required']
def test_endpoints_curated_event_ids_include_core_security_ids():
"""The curated catalog covers the agreed logon/stability IDs."""
for eventid in (4624, 4625, 4634, 4672, 4697, 4698, 41, 4618, 55, 7011,
10016, 1001):
assert eventid in CURATED_EVENT_IDS
def test_curated_ids_match_the_event_catalog():
"""Ingest filter and display catalog cannot drift apart."""
assert CURATED_EVENT_IDS == frozenset(EVENT_CATALOG)
def test_critical_event_ids_are_a_subset_of_the_catalog():
"""Dashboard stability card only queries known critical IDs."""
assert set(CRITICAL_EVENT_IDS)
assert set(CRITICAL_EVENT_IDS).issubset(set(EVENT_CATALOG))
def test_endpoints_asset_panels_are_computer_scoped():
"""Every asset panel targets computer assets only, with an endpoint."""
plugin = EndpointsPlugin()
panels = plugin.get_asset_panels()
assert len(panels) == 5
for panel in panels:
assert panel['assettypes'] == ['computer']
assert panel['endpoint'].startswith('/api/endpoints/asset/{assetid}')
assert panel['render'] in ('keyvalue', 'list', 'table')
def test_services_and_processes_are_split_into_two_filtered_cards():
"""The single services card is now two cards filtered by row kind, both
reading the same services snapshot so a future top-20 collector is just
another filtered panel over the same feed."""
plugin = EndpointsPlugin()
panels = {p['id']: p for p in plugin.get_asset_panels()}
services = panels['endpoints-tracked-services']
processes = panels['endpoints-tracked-processes']
assert services['title'] == 'Tracked Services'
assert processes['title'] == 'Tracked Processes'
assert services['filter'] == {'key': 'kind', 'equals': 'service'}
assert processes['filter'] == {'key': 'kind', 'equals': 'process'}
# Both slice one endpoint feed.
assert services['endpoint'] == processes['endpoint']
assert services['endpoint'].endswith('/services')
# Command wiring carries over to both cards.
assert services['permission'] == 'endpoints.command'
assert processes['permission'] == 'endpoints.command'
def test_recent_events_card_is_last_and_starts_collapsed():
"""Events is the lowest-priority card: forced to the bottom of the column
and collapsed by default."""
plugin = EndpointsPlugin()
panels = plugin.get_asset_panels()
events = next(p for p in panels if p['id'] == 'endpoints-events')
assert events['collapsedDefault'] is True
assert events['position'] == max(p['position'] for p in panels)
def test_endpoints_dashboard_widgets_declared():
"""Quiet + stability exception cards are declared and hide when empty."""
plugin = EndpointsPlugin()
widgets = plugin.get_dashboard_widgets()
ids = {w['id'] for w in widgets}
assert {'endpoints-quiet', 'endpoints-stability'} <= ids
for widget in widgets:
assert widget['render'] == 'exceptions'
assert widget['empty'] == 'hide'
assert widget['permission'] == 'endpoints.view'
def test_endpoints_navigation_item_points_at_fleet_view():
"""The fleet list view is exposed as a single /endpoints nav item."""
plugin = EndpointsPlugin()
items = plugin.get_navigation_items()
routes = {item['route'] for item in items}
assert '/endpoints' in routes
def test_system_restart_is_allowlisted_and_policy_selectable():
"""Restart is a real allowlisted action, so the policy builder (which
reads COMMAND_ALLOWLIST) surfaces it as a selectable action."""
assert is_allowed_action('system_restart')
assert 'system_restart' in COMMAND_ALLOWLIST
assert COMMAND_ALLOWLIST['system_restart']['label'] == 'Restart computer'
def test_system_restart_requires_the_elevated_restart_permission():
"""Restart maps to endpoints.restart so the enqueue route can demand a
heavier grant than the routine endpoints.command toggle. Routine actions
carry no elevated requirement."""
assert elevated_permission_for('system_restart') == 'endpoints.restart'
assert elevated_permission_for('mtconnect_start') is None
def test_endpoints_declares_the_restart_permission():
"""The dedicated restart permission is declared for role-grid seeding."""
plugin = EndpointsPlugin()
permissions = {name for name, *_ in plugin.get_permissions()}
assert 'endpoints.restart' in permissions
assert 'endpoints.command' in permissions
def test_collect_now_is_allowlisted_targetless_and_routine():
"""collect_now forces an off-cadence telemetry report. It is allowlisted so
the enqueue route accepts it, carries no wire target (fixed action), and
needs no elevated grant beyond endpoints.command."""
assert is_allowed_action('collect_now')
assert 'collect_now' in COMMAND_ALLOWLIST
assert not requires_target('collect_now')
assert elevated_permission_for('collect_now') is None
def test_telemetrymeta_route_is_registered_for_freshness_gating():
"""The WebUI gates panel refreshes on lastreportedat advancing, so the
lightweight freshness probe must be exposed on the plugin blueprint."""
from flask import Flask
application = Flask(__name__)
application.register_blueprint(EndpointsPlugin().get_blueprint())
rules = {rule.rule for rule in application.url_map.iter_rules()}
assert any(rule.endswith('/asset/<int:asset_id>/telemetrymeta') for rule in rules)