ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge the wilm endpoint-monitor and channel-distribution work into main

Browse Source 
Six commits from wilm-import: the squashed import, route documentation, the
channel remap onto our tag-and-trunk model, the project map, parallel test
support, and the four smaller review findings.

Merged rather than cherry-picked: the published mirror rewrites hashes, so the
commits do not apply one at a time against our history even though the squashed
diff applies clean.

Scope: product

# Conflicts:
#	docs/PROJECT-MAP.md
This commit is contained in:
cproudlock
2026-09-02 15:19:55 -04:00
parent 30b7a5a8bf 720f4ddbb2
commit b5540e631c
81 changed files with 6581 additions and 186 deletions
Download Patch File Download Diff File Expand all files Collapse all files

93
.github/workflows/publish-images.yml vendored Normal file
View File

@@ -0,0 +1,93 @@
# Build and publish per-channel container images to GHCR.
#
# WHY this exists: pilot dev workstations sit behind an inspecting proxy
# (Zscaler) that resets npm's many small fetches mid-install, so building the
# image FROM SOURCE on a pilot box is unreliable. These runners
# (arc-runner-set) install npm cleanly - the CI `frontend` job already proves
# it - so we build the image ONCE here and pilots pull the finished image.
# A single image pull is one resumable HTTPS stream, far more robust through
# the proxy than hundreds of npm requests.
#
# Channels are mapped to how THIS repo actually ships, which is not how the
# branch this came from ships. Upstream, main is a release branch and dev-be is
# the trunk. Here main IS the trunk - it runs hundreds of commits past the last
# release - and releases are tags (ADR-007). Publishing main as :stable would
# put unreleased work behind a word that promises the opposite, to sites that
# have no way to know:
#
# tag v* -> ghcr.io/<owner>/shopdb-flask:stable (badge off) the release
# main -> ghcr.io/<owner>/shopdb-flask:edge (DEV badge) the trunk
#
# The image also carries a short-SHA tag for traceability and rollback.
#
# Automatic publishing is OFF until someone confirms the GHCR package exists
# under this owner and decides who may pull it. Re-enable by uncommenting the
# push: block below; workflow_dispatch works meanwhile.
name: publish-images
on:
# push:
# tags:
# - 'v*'
# branches:
# - main
# Manual rebuild/publish of the current ref's channel image.
workflow_dispatch:
# One in-flight publish per branch; a newer push supersedes an older one.
concurrency:
group: publish-images-${{ github.ref }}
cancel-in-progress: true
jobs:
build-and-push:
runs-on: arc-runner-set
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v5
# Lowercase the owner: GHCR repository paths must be lowercase, but
# github.repository_owner can carry uppercase (e.g. ge-aero is fine, but
# this keeps the workflow correct for any owner casing).
- name: Resolve image name and channel tag
id: meta
run: |
owner="$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')"
image="ghcr.io/${owner}/shopdb-flask"
# A tag is a release, anything else is the trunk. Deliberately not
# keyed on the branch name: a release here is a tag, so keying on
# "main" would label the trunk stable.
if [ "${{ github.ref_type }}" = "tag" ]; then
channel="stable"
else
channel="edge"
fi
echo "image=${image}" >> "$GITHUB_OUTPUT"
echo "channel=${channel}" >> "$GITHUB_OUTPUT"
echo "sha=$(git rev-parse --short HEAD)" >> "$GITHUB_OUTPUT"
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Buildx
uses: docker/setup-buildx-action@v3
- name: Build and push channel image
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: |
${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.channel }}
${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.channel }}-${{ steps.meta.outputs.sha }}
# Reuse the channel tag as a cache source/target so subsequent builds
# only redo layers that actually changed.
cache-from: type=registry,ref=${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.channel }}
cache-to: type=inline

4
.gitignore vendored
View File

@@ -132,9 +132,13 @@ docker_build.txt
docker_up.txt
gates_out.txt
pytest_targeted.txt
pt*.txt
alandry.py
kendry.py
# Visual Studio local editor state (not product source).
.vs/
# Scratch tooling under scripts/ (catalog diff/merge helpers, docx generator).
# Not product source; developer-local utilities.
scripts/.temp/

38
Dockerfile
View File

@@ -38,11 +38,40 @@ RUN if [ -f /tmp/corpca/corp-root-ca.pem ]; then \
cp /tmp/corpca/corp-root-ca.pem /usr/local/share/ca-certificates/corp-root-ca.crt && \
update-ca-certificates; \
fi
ENV NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/corp-root-ca.crt
# Copy only the manifests first so `npm ci` caches on dependency changes.
COPY frontend/package*.json ./frontend/
RUN cd frontend && npm ci
# Trust the corp CA for npm's TLS only when it was actually installed above. The
# guard keeps NODE_EXTRA_CA_CERTS unset on a clean checkout (no cert), so Node
# does not warn about a missing cert path; a proxy-inspected build still gets
# the cert because the file is present.
# Install the frontend dependencies. vite and the rest of the build toolchain
# live under devDependencies, so include them explicitly (--include=dev).
# Behind an inspecting proxy (Zscaler) each package fetch is slow (40-50s) and
# npm's default concurrency piles up parallel sockets that the proxy resets
# mid-stream, making npm abort with "Exit handler never called" and leaving an
# incomplete node_modules with no vite. Serializing to a single socket
# (maxsockets 1) lets every fetch complete; the retry loop and raised timeouts
# ride out any remaining transient reset, and the final test hard-verifies the
# toolchain landed so a truncated install fails loudly here rather than as a
# confusing "vite: not found" three steps later.
RUN if [ -f /usr/local/share/ca-certificates/corp-root-ca.crt ]; then \
export NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/corp-root-ca.crt; \
fi; \
cd frontend && \
npm config set fetch-retries 5 && \
npm config set fetch-retry-factor 2 && \
npm config set fetch-retry-mintimeout 20000 && \
npm config set fetch-retry-maxtimeout 120000 && \
npm config set fetch-timeout 600000 && \
npm config set maxsockets 1 && \
( npm ci --include=dev --no-audit --no-fund \
|| { echo "npm ci attempt 1 failed, retrying..."; sleep 5; npm ci --include=dev --no-audit --no-fund; } \
|| { echo "npm ci attempt 2 failed, retrying..."; sleep 10; npm ci --include=dev --no-audit --no-fund; } ) && \
test -x node_modules/.bin/vite || { \
echo "ERROR: node_modules/.bin/vite missing after npm ci (proxy likely truncated the install)."; \
exit 1; \
}
# The frontend keeps its repo-relative layout here, because npm prebuild runs
# `node ../scripts/stage-frontend.mjs`, which stages each plugin's frontend/ into
@@ -54,7 +83,10 @@ COPY scripts/stage-frontend.mjs ./scripts/
COPY plugins/ ./plugins/
COPY frontend/ ./frontend/
RUN cd frontend && npm run build
RUN if [ -f /usr/local/share/ca-certificates/corp-root-ca.crt ]; then \
export NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/corp-root-ca.crt; \
fi; \
cd frontend && npm run build
# Output lands in /build/frontend/dist (Vite default), copied into the final
# stage below.

58
agent/ShopDbEndpointAgent.Core/AgentConfig.cs
View File

@@ -34,14 +34,23 @@ public sealed class AgentConfig
[JsonPropertyName("monitors")] public MonitorConfig Monitors { get; set; } = new();
// Per-image launchable paths for the generic process_start action. The wire
// never carries a path (that would be a code-execution hole); Start sends
// only the image name and the agent maps it here to a known-good executable
// historically never carried a path; now a server-provisioned monitor MAY
// carry one, but the agent only accepts it after LaunchPathPolicy validates
// it against ApprovedLaunchRoots (see below). Start otherwise sends only the
// image name and the agent maps it here to a known-good executable
// (or a .bat launcher). Anything not in the resolved map fails closed. A
// site overrides or extends the shipped defaults in appsettings.json without
// a rebuild. Keyed by process image name (case-insensitive, .exe optional).
[JsonPropertyName("launchpaths")]
public Dictionary<string, string> LaunchPaths { get; set; } = new();
// Approved root directories a server-provisioned launch path must sit under
// before the agent will honor it. Empty = fall back to LaunchPathPolicy's
// shipped org defaults (Program Files roots). A site (or the future Site
// Agent Configuration tool) can widen or narrow this without a rebuild.
[JsonPropertyName("approvedlaunchroots")]
public List<string> ApprovedLaunchRoots { get; set; } = new();
// Shipped defaults for the org's common shop-floor apps. A site's
// appsettings.json LaunchPaths entries win over these on a per-image basis.
// MTConnect is intentionally absent here: it is a Windows service controlled
@@ -162,6 +171,42 @@ public sealed class MonitorConfig
[JsonPropertyName("services")] public MonitorTarget[] Services { get; set; }
= Array.Empty<MonitorTarget>();
// Union the local (appsettings.json) watch list with the server-resolved one,
// keyed case-insensitively by wire name so a target watched both locally and
// by policy is not enumerated twice. Server entries win on the label (App) so
// a policy display name overrides a stale local one, but a local-only target
// is preserved - the server list augments, it does not replace. Nulls on
// either side degrade to the other, which is what keeps monitoring alive when
// a fetch fails.
public static MonitorConfig Merge(MonitorConfig? local, ResolvedMonitors? server)
{
return new MonitorConfig
{
Processes = MergeTargets(local?.Processes, server?.Processes),
Services = MergeTargets(local?.Services, server?.Services),
};
}
private static MonitorTarget[] MergeTargets(
MonitorTarget[]? local, MonitorTarget[]? server)
{
var byName = new Dictionary<string, MonitorTarget>(
StringComparer.OrdinalIgnoreCase);
foreach (var target in local ?? Array.Empty<MonitorTarget>())
{
if (string.IsNullOrWhiteSpace(target?.Name))
continue;
byName[target.Name.Trim()] = target;
}
foreach (var target in server ?? Array.Empty<MonitorTarget>())
{
if (string.IsNullOrWhiteSpace(target?.Name))
continue;
byName[target.Name.Trim()] = target;
}
return byName.Values.ToArray();
}
}
public sealed class MonitorTarget
@@ -172,4 +217,13 @@ public sealed class MonitorTarget
// Human-facing application label, optional. Falls back to Name when empty.
[JsonPropertyName("app")] public string App { get; set; } = "";
// Optional server-provisioned launch path for a process monitor. Only set on
// process targets that a site leader configured a path for. The agent NEVER
// launches this blindly: LaunchPathPolicy validates it against an approved-
// root allowlist first, and a local appsettings.json launchpaths entry for
// the same image still wins. Null/empty means "no server path" - the agent
// falls back to its local launch map, and process_start fails closed when
// neither yields a validated path.
[JsonPropertyName("launchpath")] public string? LaunchPath { get; set; }
}

10
agent/ShopDbEndpointAgent.Core/AgentFaultMonitor.cs
View File

@@ -34,7 +34,15 @@ public sealed class AgentFaultMonitor
}
public void ReportFatal(string context, Exception error)
=> Report(EventLogEntryType.Error, 1000, $"{context}: {error}");
{
// File log first, always: the Event Log sink below is admin-gated (the
// source cannot be created without elevation), so an unelevated run such
// as the tray-launched --once/--report path would otherwise fail with no
// trace anywhere. The rolling file log is the one sink guaranteed to work
// in every context.
logger.LogError(error, "Fatal in {Context}", context);
Report(EventLogEntryType.Error, 1000, $"{context}: {error}");
}
private void OnUnhandledException(object sender, UnhandledExceptionEventArgs args)
{

83
agent/ShopDbEndpointAgent.Core/CollectorClient.cs
View File

@@ -89,6 +89,42 @@ public sealed class CollectorClient : IDisposable
return envelope.Data;
}
// Fetch the server-resolved watch list (policy-attached processes/services)
// for this hostname, GET /api/endpoints/monitors/resolved. Shares the same
// X-API-Key credential as telemetry. Best-effort: returns null on any
// failure so the caller falls back to the last-known/local list rather than
// blanking monitoring on a transient blip. A single attempt, no retry - the
// next telemetry cycle retries naturally, and a stale-but-usable watch list
// beats an empty one.
public async Task<ResolvedMonitors?> FetchResolvedMonitorsAsync(
string hostname, CancellationToken token)
{
var url = "api/endpoints/monitors/resolved?hostname="
+ Uri.EscapeDataString(hostname);
try
{
using var response = await http.GetAsync(url, token);
var body = await response.Content.ReadAsStringAsync(token);
if (!response.IsSuccessStatusCode)
{
logger.LogWarning(
"Resolved-monitors fetch returned HTTP {Status}: {Body}",
(int)response.StatusCode, body);
return null;
}
var envelope = JsonSerializer.Deserialize<ResolvedMonitorsEnvelope>(
body, JsonOptions);
return envelope?.Data;
}
catch (Exception error) when (
error is HttpRequestException or TaskCanceledException or JsonException
&& !token.IsCancellationRequested)
{
logger.LogWarning(error, "Resolved-monitors fetch failed; keeping current watch list");
return null;
}
}
private TimeSpan BackoffDelay(int attempt)
{
// 2^(attempt-1) seconds, capped, plus up to 1s jitter.
@@ -96,6 +132,53 @@ public sealed class CollectorClient : IDisposable
return TimeSpan.FromSeconds(seconds + jitter.NextDouble());
}
// Uploads one operator issue report (free-text reason/details plus an
// optional PNG screenshot) to POST /api/endpoints/operatorreports as
// multipart/form-data, authorized by the same X-API-Key collector key as
// telemetry. The server resolves this hostname to its asset. A single
// attempt: the tray shows the operator the outcome and can retry by hand,
// and a stuck retry loop holding a screenshot in memory is worse than a
// clear failure. Returns the parsed upload response (new row id) or throws.
public async Task<OperatorReportUploadResponse?> UploadOperatorReportAsync(
string reason, string details, string? screenshotPath, string hostname,
CancellationToken token)
{
using var form = new MultipartFormDataContent();
form.Add(new StringContent(reason ?? string.Empty), "reason");
form.Add(new StringContent(details ?? string.Empty), "details");
form.Add(new StringContent(hostname ?? string.Empty), "hostname");
StreamContent? imageContent = null;
FileStream? imageStream = null;
try
{
if (!string.IsNullOrWhiteSpace(screenshotPath) && File.Exists(screenshotPath))
{
imageStream = File.OpenRead(screenshotPath);
imageContent = new StreamContent(imageStream);
imageContent.Headers.ContentType =
new System.Net.Http.Headers.MediaTypeHeaderValue("image/png");
form.Add(imageContent, "screenshot",
Path.GetFileName(screenshotPath));
}
using var response = await http.PostAsync(
"api/endpoints/operatorreports", form, token);
var body = await response.Content.ReadAsStringAsync(token);
if (!response.IsSuccessStatusCode)
throw new CollectorException((int)response.StatusCode, body);
var envelope = JsonSerializer.Deserialize<OperatorReportUploadEnvelope>(
body, JsonOptions);
return envelope?.Data;
}
finally
{
imageContent?.Dispose();
imageStream?.Dispose();
}
}
public void Dispose() => http.Dispose();
}

16
agent/ShopDbEndpointAgent.Core/CommandClient.cs
View File

@@ -22,12 +22,26 @@ public sealed class CommandClient : IDisposable
};
private readonly HttpClient http;
private readonly SocketsHttpHandler handler;
private readonly ILogger<CommandClient> logger;
public CommandClient(AgentConfig config, ILogger<CommandClient> logger)
{
this.logger = logger;
http = new HttpClient
// The command channel long-polls: the server holds each claim open for a
// hold window and then, between windows, the pooled TLS connection sits
// idle. Gunicorn's keep-alive closes an idle socket after a couple of
// seconds, so a connection reused after that yields "unexpected EOF or 0
// bytes from the transport stream" on the next TLS handshake. Retire
// pooled connections faster than the server's keep-alive so a socket the
// server has already dropped is never handed back out; the lifetime cap
// is a hard backstop so no connection lingers indefinitely.
handler = new SocketsHttpHandler
{
PooledConnectionIdleTimeout = TimeSpan.FromSeconds(1),
PooledConnectionLifetime = TimeSpan.FromMinutes(2),
};
http = new HttpClient(handler, disposeHandler: true)
{
BaseAddress = new Uri(config.BaseUrl.TrimEnd('/') + "/"),
// Longer than the collector client: the claim request is held open

68
agent/ShopDbEndpointAgent.Core/CommandExecutor.cs
View File

@@ -56,9 +56,14 @@ public sealed class CommandExecutor
// Resolved per-image launchable paths for process_start (shipped defaults
// overlaid with any appsettings.json overrides). A target absent from this
// map cannot be started - process_start fails closed.
// map - and without a validated server-provisioned path - cannot be started,
// so process_start fails closed.
private readonly IReadOnlyDictionary<string, string> launchPaths;
// Validates server-provisioned launch paths (from policy-attached monitors)
// before they are honored. Local launchPaths always win over a server path.
private readonly LaunchPathPolicy launchPathPolicy;
// Held for update_app: the agent joins InstallPath to BaseUrl and downloads
// the pinned installer with its own collector credential (ApiKey), so the
// installer body never rides the command wire.
@@ -71,6 +76,7 @@ public sealed class CommandExecutor
this.startTimeout = startTimeout ?? TimeSpan.FromSeconds(30);
this.config = config ?? new AgentConfig();
this.launchPaths = this.config.ResolveLaunchPaths();
this.launchPathPolicy = new LaunchPathPolicy(this.config.ApprovedLaunchRoots);
}
public async Task<CommandResult> ExecuteAsync(
@@ -366,8 +372,8 @@ public sealed class CommandExecutor
if (bare.EndsWith(".exe", StringComparison.OrdinalIgnoreCase))
bare = bare[..^4];
if (!launchPaths.TryGetValue(bare, out var launchPath)
|| string.IsNullOrWhiteSpace(launchPath))
var launchPath = ResolveLaunchPath(bare, image);
if (string.IsNullOrWhiteSpace(launchPath))
{
logger.LogWarning(
"Rejected process_start for {Image}: no launchable path configured",
@@ -452,6 +458,62 @@ public sealed class CommandExecutor
}
}
// Resolves the launchable path for a process image. Precedence:
// 1. Local launch map (shipped defaults + appsettings.json). A site pin
// always wins so a server value never silently overrides it.
// 2. A server-provisioned path on the matching process monitor, but only
// after LaunchPathPolicy validates it (approved root, exists, launchable
// extension). A rejected server path is logged and dropped - the caller
// then fails closed exactly as before.
// Returns null when neither source yields a usable path.
private string? ResolveLaunchPath(string bareImage, string image)
{
if (launchPaths.TryGetValue(bareImage, out var localPath)
&& !string.IsNullOrWhiteSpace(localPath))
{
return localPath;
}
var serverPath = ServerLaunchPathFor(image, bareImage);
if (string.IsNullOrWhiteSpace(serverPath))
return null;
if (launchPathPolicy.TryValidate(serverPath, out var validated, out var reason))
return validated;
logger.LogWarning(
"Rejected server-provisioned launch path for {Image} ({Path}): {Reason}",
image, serverPath, reason);
return null;
}
// Finds the server-provisioned launch path carried on the process monitor
// whose wire name matches the image (case-insensitive, .exe optional).
private string? ServerLaunchPathFor(string image, string bareImage)
{
var processes = config.Monitors?.Processes;
if (processes is null)
return null;
foreach (var target in processes)
{
if (string.IsNullOrWhiteSpace(target?.Name)
|| string.IsNullOrWhiteSpace(target.LaunchPath))
continue;
var name = target.Name.Trim();
var bareName = name.EndsWith(".exe", StringComparison.OrdinalIgnoreCase)
? name[..^4]
: name;
if (string.Equals(name, image, StringComparison.OrdinalIgnoreCase)
|| string.Equals(bareName, bareImage, StringComparison.OrdinalIgnoreCase))
{
return target.LaunchPath;
}
}
return null;
}
private bool IsProcessRunning(string image)
{
var processes = FindProcessesByImage(image);

35
agent/ShopDbEndpointAgent.Core/Contracts.cs
View File

@@ -72,6 +72,20 @@ public sealed class CollectorEnvelope
[JsonPropertyName("data")] public CollectorResponse? Data { get; set; }
}
// Server response to an operator issue report upload. Carries the new row id so
// the agent can log it for correlation.
public sealed class OperatorReportUploadResponse
{
[JsonPropertyName("operatorreportid")] public int? OperatorReportId { get; set; }
[JsonPropertyName("assetid")] public int? AssetId { get; set; }
}
public sealed class OperatorReportUploadEnvelope
{
[JsonPropertyName("status")] public string Status { get; set; } = "";
[JsonPropertyName("data")] public OperatorReportUploadResponse? Data { get; set; }
}
// A command the server claimed for this endpoint on the long-poll. The
// allowlisted action name crosses the wire (see AGENT-CONTRACT.md section 7);
// generic actions (service_* / process_*) also carry the subject to act on -
@@ -147,3 +161,24 @@ public sealed class CommandResult
[JsonPropertyName("mtconnectrunning")] public bool? MtConnectRunning { get; set; }
[JsonPropertyName("detail")] public Dictionary<string, object>? Detail { get; set; }
}
// The server-resolved watch list for this endpoint (processes and services the
// site's policy attaches to the box's PC type), fetched each telemetry cycle so
// a policy-attached monitor is enumerated without editing appsettings.json on
// the box. Same {name, app} shape as the local MonitorConfig, so the merge is a
// direct list union. Empty lists mean "watch nothing extra beyond local".
public sealed class ResolvedMonitors
{
[JsonPropertyName("processes")] public MonitorTarget[] Processes { get; set; }
= Array.Empty<MonitorTarget>();
[JsonPropertyName("services")] public MonitorTarget[] Services { get; set; }
= Array.Empty<MonitorTarget>();
}
// Envelope for the resolved-monitors route: { status, data: { processes, services } }.
public sealed class ResolvedMonitorsEnvelope
{
[JsonPropertyName("status")] public string Status { get; set; } = "";
[JsonPropertyName("data")] public ResolvedMonitors? Data { get; set; }
}

123
agent/ShopDbEndpointAgent.Core/LaunchPathPolicy.cs Normal file
View File

@@ -0,0 +1,123 @@
using System.Text.Json.Serialization;
namespace ShopDbEndpointAgent.Core;
// Guards server-provisioned launch paths. Historically the command wire never
// carried a launch path at all - the agent mapped an image name to a known-good
// local path and anything else failed closed. Server-provisioned monitors now
// MAY carry a path a site leader set in the WebUI, which is a deliberate
// relaxation of that invariant. This policy is the replacement safeguard: a
// server path is only honored when it sits under an approved root, actually
// exists, and is an approved executable type. Everything else is rejected and
// the agent falls back to its local launch map (still fail closed).
//
// The approved-root list is org-opinionated with shipped defaults (the Windows
// Program Files roots), overridable per site via AgentConfig.ApprovedLaunchRoots
// so the future Site Agent Configuration tool can widen or narrow it without a
// rebuild. A local appsettings.json launchpaths entry still wins over any
// server path, so a site pin is never silently overridden.
public sealed class LaunchPathPolicy
{
// Only these extensions are launchable. .bat is included because some shop-
// floor apps ship a launcher script (see the eDNC/UDC run.bat pattern).
private static readonly string[] ApprovedExtensions =
{ ".exe", ".bat", ".cmd" };
// Shipped org defaults: the standard 64- and 32-bit install roots. Resolved
// from the environment so a non-C: system drive still matches.
public static IReadOnlyList<string> DefaultApprovedRoots { get; } =
BuildDefaultRoots();
private readonly IReadOnlyList<string> approvedRoots;
public LaunchPathPolicy(IEnumerable<string>? approvedRootsOverride = null)
{
var roots = (approvedRootsOverride ?? Enumerable.Empty<string>())
.Where(root => !string.IsNullOrWhiteSpace(root))
.Select(NormalizeRoot)
.ToList();
approvedRoots = roots.Count > 0 ? roots : DefaultApprovedRoots;
}
// Result of validating a candidate path: the normalized full path when it is
// acceptable, or a human-readable reason when it is rejected.
public bool TryValidate(string? candidate, out string fullPath, out string reason)
{
fullPath = "";
reason = "";
if (string.IsNullOrWhiteSpace(candidate))
{
reason = "empty path";
return false;
}
string resolved;
try
{
resolved = Path.GetFullPath(candidate.Trim());
}
catch (Exception ex)
{
reason = $"not a valid path: {ex.Message}";
return false;
}
var extension = Path.GetExtension(resolved);
if (!ApprovedExtensions.Contains(extension, StringComparer.OrdinalIgnoreCase))
{
reason = $"extension '{extension}' is not launchable";
return false;
}
if (!IsUnderApprovedRoot(resolved))
{
reason = "path is outside every approved launch root";
return false;
}
if (!File.Exists(resolved))
{
reason = "file does not exist";
return false;
}
fullPath = resolved;
return true;
}
private bool IsUnderApprovedRoot(string resolved)
{
foreach (var root in approvedRoots)
{
// Compare against root + separator so "C:\Program Files" does not
// also match "C:\Program FilesEvil\...". Case-insensitive on Windows.
var prefix = root.EndsWith(Path.DirectorySeparatorChar)
? root
: root + Path.DirectorySeparatorChar;
if (resolved.StartsWith(prefix, StringComparison.OrdinalIgnoreCase))
return true;
}
return false;
}
private static string NormalizeRoot(string root)
{
var full = Path.GetFullPath(root.Trim());
return full.TrimEnd(Path.DirectorySeparatorChar);
}
private static IReadOnlyList<string> BuildDefaultRoots()
{
var candidates = new[]
{
Environment.GetFolderPath(Environment.SpecialFolder.ProgramFiles),
Environment.GetFolderPath(Environment.SpecialFolder.ProgramFilesX86),
};
return candidates
.Where(path => !string.IsNullOrWhiteSpace(path))
.Select(NormalizeRoot)
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToList();
}
}

52
agent/ShopDbEndpointAgent.Core/OperatorReportPayload.cs Normal file
View File

@@ -0,0 +1,52 @@
using System.Text.Json;
using System.Text.Json.Serialization;
namespace ShopDbEndpointAgent.Core;
// The on-disk hand-off between the interactive tray (which captures the
// operator's screen and notes) and the headless service binary (which holds the
// collector key and performs the upload). The tray writes this JSON sidecar to a
// per-run temp folder and invokes `shopdbagent.exe --report <path>`; the agent
// reads it, uploads, and deletes the folder. Keeping the secret in the service
// preserves the "tray never holds a credential" rule already used by --once.
public sealed class OperatorReportPayload
{
[JsonPropertyName("reason")]
public string Reason { get; set; } = "Other (free text)";
[JsonPropertyName("details")]
public string Details { get; set; } = string.Empty;
// Absolute path to the captured PNG, or null when the operator declined a
// screenshot. Written by the tray into the interactive operator's
// %USERPROFILE%\ShopDBScreenshots folder and passed through as an absolute
// path; the agent reads it verbatim and never re-derives the location (as
// SYSTEM its own profile would resolve elsewhere).
[JsonPropertyName("screenshotpath")]
public string? ScreenshotPath { get; set; }
private static readonly JsonSerializerOptions Options = new()
{
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
WriteIndented = true,
};
public static void Write(string filePath, string reason, string details,
string? screenshotPath)
{
var payload = new OperatorReportPayload
{
Reason = reason,
Details = details,
ScreenshotPath = screenshotPath,
};
File.WriteAllText(filePath, JsonSerializer.Serialize(payload, Options));
}
public static OperatorReportPayload Read(string filePath)
{
var json = File.ReadAllText(filePath);
return JsonSerializer.Deserialize<OperatorReportPayload>(json, Options)
?? throw new InvalidDataException("empty or invalid report payload");
}
}

15
agent/ShopDbEndpointAgent.Core/TelemetryWorker.cs
View File

@@ -96,6 +96,21 @@ public sealed class TelemetryWorker : BackgroundService
// exact production collection path.
public async Task<CollectorResponse> RunOnceAsync(bool adopt, CancellationToken token)
{
// Refresh the watch list from server policy BEFORE collecting so a
// policy-attached monitor (added in the WebUI, never in appsettings.json)
// is enumerated and reported this cycle. Server entries merge over the
// local list; a failed fetch returns null and Merge keeps the local list,
// so monitoring never blanks on a transient blip.
var resolved = await client.FetchResolvedMonitorsAsync(
Environment.MachineName, token);
if (resolved is not null)
{
config.Monitors = MonitorConfig.Merge(config.Monitors, resolved);
logger.LogDebug(
"Watch list refreshed from policy: {Processes} process(es), {Services} service(s)",
config.Monitors.Processes.Length, config.Monitors.Services.Length);
}
var report = collector.Collect(agentVersion, adopt);
if (utilities is not null)

32
agent/ShopDbEndpointAgent.Tray/Program.cs
View File

@@ -1,6 +1,8 @@
using System.Threading;
namespace ShopDbEndpointAgent.Tray;
// Interactive user-session tray app (SCAFFOLD). Separate process from the
// Interactive user-session tray app (SCAFFOLD).
// headless Windows Service: the service owns telemetry, this owns the operator
// surface - a system-tray icon and the 5-click-max ServiceNow ticket flow
// (DESIGN-DECISIONS-agent.md section 6). This iteration wires the tray and a
@@ -8,9 +10,35 @@ namespace ShopDbEndpointAgent.Tray;
// is deferred until that decision is made server-side.
internal static class Program
{
// Machine-wide single-instance guard. The installer starts the tray from an
// HKLM Run value for every user at logon, and an operator can also launch it
// from the Start Menu shortcut, so a second copy is easy to spawn. A named
// mutex in the Global namespace keeps exactly one tray per session: a later
// launch that cannot own the mutex exits immediately instead of stacking a
// duplicate icon. Global\ scopes across sessions on the same machine.
private const string SingleInstanceMutexName = @"Global\ShopDbEndpointAgentTray";
[STAThread]
private static void Main()
private static void Main(string[] args)
{
// Headless icon export for the installer: `--export-icon <path>` writes
// the branded .ico and exits without starting the tray, so install.ps1
// can point the Start Menu shortcut at a real brand icon. Kept in this
// exe so the icon has a single source (TrayIcon) and no binary is
// checked into the repo.
if (args.Length >= 2 &&
string.Equals(args[0], "--export-icon", StringComparison.OrdinalIgnoreCase))
{
TrayIcon.Export(args[1]);
return;
}
using var singleInstance = new Mutex(true, SingleInstanceMutexName, out bool owned);
if (!owned)
{
return;
}
ApplicationConfiguration.Initialize();
Application.Run(new TrayContext());
}

34
agent/ShopDbEndpointAgent.Tray/ScreenCapture.cs Normal file
View File

@@ -0,0 +1,34 @@
using System.Drawing.Imaging;
namespace ShopDbEndpointAgent.Tray;
// Captures the full virtual desktop (all monitors) to a PNG on disk, porting
// the operator PowerShell proof of concept:
// CopyFromScreen over SystemInformation.VirtualScreen, then Bitmap.Save as
// PNG. Kept in the tray project because only the interactive tray session can
// see the operator's desktop; the headless service runs in session 0 and
// would capture a black frame.
internal static class ScreenCapture
{
// Grab every monitor in one shot. VirtualScreen spans the whole desktop so
// a multi-head shop-floor PC is captured intact rather than just the
// primary display. Returns the saved file path.
public static string CaptureToPng(string filePath)
{
var bounds = SystemInformation.VirtualScreen;
using var bitmap = new Bitmap(bounds.Width, bounds.Height,
PixelFormat.Format32bppArgb);
using (var graphics = Graphics.FromImage(bitmap))
{
graphics.CopyFromScreen(bounds.Left, bounds.Top, 0, 0,
bounds.Size, CopyPixelOperation.SourceCopy);
}
var directory = Path.GetDirectoryName(filePath);
if (!string.IsNullOrEmpty(directory))
Directory.CreateDirectory(directory);
bitmap.Save(filePath, ImageFormat.Png);
return filePath;
}
}

50
agent/ShopDbEndpointAgent.Tray/TicketForm.cs
View File

@@ -1,22 +1,20 @@
namespace ShopDbEndpointAgent.Tray;
// The 5-click-max issue reporter (SCAFFOLD). Target flow from
// DESIGN-DECISIONS-agent.md section 6:
// network precheck -> issue dropdown -> optional screenshot -> submit.
// This iteration renders the dropdown of fixed reasons and a submit button but
// does NOT talk to ServiceNow yet - the submission path (direct API vs proxied
// through a ShopDB server endpoint so ServiceNow creds stay server-side) is an
// open decision. Submit currently shows what WOULD be sent.
// The operator issue reporter. Collects a free-text description and an optional
// screenshot request, then hands the result back to TrayContext which performs
// the capture (after this dialog closes, so it never appears in the shot) and
// the upload via the service binary.
//
// Only "Other (free text)" is offered today: the fixed-reason routing from the
// original design is not wired to a ticketing backend, so a single free-text
// path is the one flow that works end to end. The dropdown is kept (disabled,
// preselected) so the richer reason list can return without reworking the form.
internal sealed class TicketForm : Form
{
// Fixed dropdown reasons from the source conversation (section 6).
// Only the free-text reason is functional. Kept as a one-item list so the
// control and its layout stay in place for the future reason routing.
private static readonly string[] Reasons =
{
"Can't logon to CSF",
"Data collections down",
"Can't download program",
"Machine communication failing",
"Network drives down",
"Other (free text)",
};
@@ -24,6 +22,11 @@ internal sealed class TicketForm : Form
private readonly TextBox detailBox;
private readonly CheckBox screenshotBox;
// Collected result, read by TrayContext after DialogResult.OK.
public string Reason => reasonBox.SelectedItem?.ToString() ?? "Other (free text)";
public string Details => detailBox.Text?.Trim() ?? string.Empty;
public bool IncludeScreenshot => screenshotBox.Checked;
public TicketForm()
{
Text = "Report an issue";
@@ -46,13 +49,16 @@ internal sealed class TicketForm : Form
Top = 40,
Width = 388,
DropDownStyle = ComboBoxStyle.DropDownList,
// Disabled until the fixed-reason routing exists; free text is the
// only functional path, so the selection is fixed on it.
Enabled = false,
};
reasonBox.Items.AddRange(Reasons);
reasonBox.SelectedIndex = 0;
var detailLabel = new Label
{
Text = "Details (optional)",
Text = "Details",
Left = 16,
Top = 76,
Width = 388,
@@ -72,6 +78,7 @@ internal sealed class TicketForm : Form
Left = 16,
Top = 188,
Width = 388,
Checked = true,
};
var submit = new Button
@@ -82,7 +89,6 @@ internal sealed class TicketForm : Form
Width = 76,
DialogResult = DialogResult.OK,
};
submit.Click += OnSubmit;
var cancel = new Button
{
@@ -101,18 +107,4 @@ internal sealed class TicketForm : Form
AcceptButton = submit;
CancelButton = cancel;
}
private void OnSubmit(object? sender, EventArgs e)
{
// Placeholder: show what would be submitted. The real path (screenshot
// capture to Pictures\Screenshots, then ServiceNow submit) lands once
// the direct-vs-proxied decision is made.
var summary =
$"Reason: {reasonBox.SelectedItem}\n" +
$"Details: {(string.IsNullOrWhiteSpace(detailBox.Text) ? "(none)" : detailBox.Text)}\n" +
$"Screenshot: {(screenshotBox.Checked ? "yes" : "no")}";
MessageBox.Show(
"Ticketing is not wired up yet. This would be submitted:\n\n" + summary,
"ShopDB agent (scaffold)", MessageBoxButtons.OK, MessageBoxIcon.Information);
}
}

136
agent/ShopDbEndpointAgent.Tray/TrayContext.cs
View File

@@ -1,4 +1,7 @@
using System.Diagnostics;
using System.Reflection;
using ShopDbEndpointAgent.Core;
namespace ShopDbEndpointAgent.Tray;
@@ -44,10 +47,102 @@ internal sealed class TrayContext : ApplicationContext
trayIcon.DoubleClick += OnSendStatus;
}
private void OnReportIssue(object? sender, EventArgs e)
private async void OnReportIssue(object? sender, EventArgs e)
{
using var dialog = new TicketForm();
dialog.ShowDialog();
string reason;
string details;
bool includeScreenshot;
using (var dialog = new TicketForm())
{
if (dialog.ShowDialog() != DialogResult.OK)
return;
reason = dialog.Reason;
details = dialog.Details;
includeScreenshot = dialog.IncludeScreenshot;
}
var agentExe = ResolveAgentExe();
if (agentExe is null)
{
ShowBalloon("Report an issue", "Could not find " + AgentExeName +
". Is the ShopDB Endpoint Agent installed?", ToolTipIcon.Error);
return;
}
// Capture AFTER the dialog closes so the report form is never in the
// shot. Stage into the interactive operator's own profile
// (%USERPROFILE%\ShopDBScreenshots), NOT %TEMP%: on this locked-down
// image the tray process's TEMP resolves to a different account's
// AppData that ShopFloor cannot write, and AppData/Temp are aggressively
// ACL'd/redirected. The profile root is owned by the operator and, by
// inherited ACL, readable by SYSTEM (the agent service) so the handoff
// works. Files are kept for the operator to show a DT/specialist later;
// pruning is the agent's future server-driven Capture Cleanup job.
var storeDir = OperatorReportStoreDir();
var stamp = DateTime.Now.ToString("yyyyMMdd-HHmmss");
string? payloadPath = null;
try
{
Directory.CreateDirectory(storeDir);
string? screenshotPath = null;
if (includeScreenshot)
{
screenshotPath = Path.Combine(storeDir, "screenshot-" + stamp + ".png");
ScreenCapture.CaptureToPng(screenshotPath);
}
payloadPath = Path.Combine(storeDir, "report-" + stamp + ".json");
OperatorReportPayload.Write(payloadPath, reason, details, screenshotPath);
ShowBalloon("Report an issue", "Sending your report to ShopDB...",
ToolTipIcon.Info);
int exitCode = await RunAgentAsync(agentExe, "--report \"" + payloadPath + "\"");
if (exitCode == 0)
ShowBalloon("Report an issue", "Your report was sent to ShopDB.",
ToolTipIcon.Info);
else
ShowBalloon("Report an issue",
"Report send failed (exit " + exitCode +
"). See the agent log under ProgramData\\ShopDB.",
ToolTipIcon.Warning);
}
catch (Exception error)
{
ShowBalloon("Report an issue", "Could not send the report: " +
error.Message, ToolTipIcon.Error);
}
finally
{
// Only the transient JSON sidecar is removed; the screenshot stays
// for the operator. The agent also deletes the sidecar on success,
// so this just covers the failure path.
if (payloadPath is not null)
TryDeleteFile(payloadPath);
}
}
// The operator-visible folder screenshots and the report sidecar are staged
// into, under the interactive user's profile. Resolved in the tray (running
// as that user) and passed to the agent as an absolute path; the agent must
// never re-derive it, since as SYSTEM its profile would resolve elsewhere.
private static string OperatorReportStoreDir()
=> Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.UserProfile),
"ShopDBScreenshots");
private static void TryDeleteFile(string filePath)
{
try
{
if (File.Exists(filePath))
File.Delete(filePath);
}
catch
{
// Best-effort; a leftover sidecar is harmless and the agent also
// removes it on a successful upload.
}
}
// Runs shopdbagent.exe --once out of process so the operator can push a
@@ -74,7 +169,7 @@ internal sealed class TrayContext : ApplicationContext
try
{
int exitCode = await RunAgentOnceAsync(agentExe);
int exitCode = await RunAgentAsync(agentExe, "--once");
if (exitCode == 0)
ShowBalloon("Send Status", "Status sent to ShopDB.",
ToolTipIcon.Info);
@@ -96,12 +191,12 @@ internal sealed class TrayContext : ApplicationContext
}
}
private static Task<int> RunAgentOnceAsync(string agentExe)
private static Task<int> RunAgentAsync(string agentExe, string arguments)
{
var startInfo = new ProcessStartInfo
{
FileName = agentExe,
Arguments = "--once",
Arguments = arguments,
WorkingDirectory = Path.GetDirectoryName(agentExe) ?? Environment.CurrentDirectory,
UseShellExecute = false,
CreateNoWindow = true,
@@ -181,9 +276,36 @@ internal sealed class TrayContext : ApplicationContext
=> MessageBox.Show(
"ShopDB Endpoint Agent tray.\n" +
"Telemetry runs as the ShopDB Endpoint Agent Windows service.\n" +
"Send Status now reports this PC to ShopDB on demand.",
"Send Status now reports this PC to ShopDB on demand.\n\n" +
"Tray version: " + TrayVersion() + "\n" +
"Agent version: " + AgentVersion(),
"About", MessageBoxButtons.OK, MessageBoxIcon.Information);
// The tray's own assembly version, the same value stamped from the csproj
// <Version> at build time.
private static string TrayVersion()
=> Assembly.GetExecutingAssembly().GetName().Version?.ToString() ?? "unknown";
// The installed agent service exe's file version, read from wherever the
// tray resolves it (beside the tray, then the default install dir). The
// service and tray can legitimately be different versions, so this is shown
// as its own line rather than assumed equal to the tray. Degrades to a
// clear note when the agent exe is not found or its version is unreadable.
private static string AgentVersion()
{
var agentExe = ResolveAgentExe();
if (agentExe is null)
return "not found";
try
{
return FileVersionInfo.GetVersionInfo(agentExe).FileVersion ?? "unknown";
}
catch (FileNotFoundException)
{
return "not found";
}
}
private void OnExit(object? sender, EventArgs e)
{
trayIcon.Visible = false;

16
agent/ShopDbEndpointAgent.Tray/TrayIcon.cs
View File

@@ -62,6 +62,22 @@ internal static class TrayIcon
}
}
// Writes the branded icon to an .ico file so a Start Menu / desktop shortcut
// can show the brand instead of the generic exe icon. Reuses the same GDI+
// drawing as the tray so there is still one source of brand truth and no
// checked-in binary. 64px suits shortcut/large-icon views; the OS downscales.
public static void Export(string path, int size = 64)
{
using var icon = Create(size);
var directory = Path.GetDirectoryName(path);
if (!string.IsNullOrEmpty(directory))
{
Directory.CreateDirectory(directory);
}
using var stream = new FileStream(path, FileMode.Create, FileAccess.Write);
icon.Save(stream);
}
private static GraphicsPath RoundedRect(RectangleF rect, float radius)
{
float diameter = radius * 2;

62
agent/ShopDbEndpointAgent/Program.cs
View File

@@ -10,6 +10,9 @@ using ShopDbEndpointAgent.Core;
// (default) run as a Windows Service (AddWindowsService)
// --console run interactively in the foreground for debugging
// --once collect and report exactly once, then exit (smoke test / cron)
// --report <path> upload one operator issue report (JSON sidecar written by
// the tray), then exit. Keeps the collector key in the
// service so the tray never holds a secret.
//
// Secrets are never read here: AgentConfigProvider layers the registry
// (HKLM\SOFTWARE\GE\ShopDB: CollectorKey, BaseUrl) over the shipped, non-secret
@@ -19,6 +22,7 @@ using ShopDbEndpointAgent.Core;
var runOnce = args.Any(a => string.Equals(a, "--once", StringComparison.OrdinalIgnoreCase));
var forceConsole = args.Any(a => string.Equals(a, "--console", StringComparison.OrdinalIgnoreCase));
var reportPayloadPath = ArgValue(args, "--report");
var agentVersion = Assembly.GetExecutingAssembly()
.GetName().Version?.ToString() ?? "0.0.0";
@@ -58,6 +62,9 @@ try
if (runOnce)
return await RunOnceAsync(config, agentVersion, logDirectory, faultMonitor);
if (!string.IsNullOrWhiteSpace(reportPayloadPath))
return await RunReportAsync(config, reportPayloadPath, logDirectory, faultMonitor);
await RunHostAsync(config, agentVersion, logDirectory, forceConsole);
return 0;
}
@@ -103,6 +110,61 @@ static async Task<int> RunOnceAsync(AgentConfig config, string agentVersion,
}
}
// Uploads one operator issue report described by a tray-written JSON sidecar,
// then deletes the payload folder. Runs out of process from the tray so the
// collector key stays in the service binary. The exit code is the tray's
// contract: 0 = uploaded, non-zero = failed (detail in the agent log).
static async Task<int> RunReportAsync(AgentConfig config, string payloadPath,
string logDirectory, AgentFaultMonitor faultMonitor)
{
using var services = BuildOnceServices(config, "0.0.0", logDirectory);
var client = services.GetRequiredService<CollectorClient>();
var logger = services.GetRequiredService<ILogger<CollectorClient>>();
try
{
var payload = OperatorReportPayload.Read(payloadPath);
var response = await client.UploadOperatorReportAsync(
payload.Reason, payload.Details, payload.ScreenshotPath,
Environment.MachineName, CancellationToken.None);
logger.LogInformation("Operator report uploaded: id={ReportId}",
response?.OperatorReportId);
return 0;
}
catch (Exception error)
{
faultMonitor.ReportFatal("--report", error);
return 1;
}
finally
{
// Remove only the transient JSON sidecar. The screenshot lives in the
// operator's %USERPROFILE%\ShopDBScreenshots folder and is deliberately
// KEPT so the operator can show it to a DT/specialist later; pruning it
// is the future server-driven Capture Cleanup job, not this path. (The
// sidecar previously sat in a throwaway temp folder we deleted whole;
// now it shares the operator's folder, so a recursive delete here would
// wipe saved screenshots.)
try
{
if (File.Exists(payloadPath))
File.Delete(payloadPath);
}
catch { /* best-effort; the tray also removes the sidecar */ }
}
}
// Read the value following a flag (e.g. --report C:\path\report.json). Returns
// null when the flag is absent or has no following token.
static string? ArgValue(string[] args, string flag)
{
for (int i = 0; i < args.Length - 1; i++)
{
if (string.Equals(args[i], flag, StringComparison.OrdinalIgnoreCase))
return args[i + 1];
}
return null;
}
static ServiceProvider BuildOnceServices(AgentConfig config, string agentVersion,
string logDirectory)
{

35
agent/ShopDbEndpointAgent/ShopDbEndpointAgent.csproj
View File

@@ -7,7 +7,38 @@
<Nullable>enable</Nullable>
<RootNamespace>ShopDbEndpointAgent</RootNamespace>
<AssemblyName>shopdbagent</AssemblyName>
<!-- 1.4.1: run_script/update_app stage into a fixed, Defender-excludable
<!-- 1.6.3: operator issue reports stage under the interactive operator's
%USERPROFILE%\ShopDBScreenshots (resolved in the ShopFloor tray session)
instead of %TEMP%, which on locked-down endpoints redirected to another
user's AppData\Local\Temp and left the SYSTEM-context report upload
reading an inaccessible path. Screenshots use flat timestamped names and
are KEPT after a successful upload for operator review; only the transient
JSON sidecar is pruned, so the report path no longer recursively deletes
the shared folder. AgentFaultMonitor.ReportFatal now mirrors fatal report
failures to the rolling file log so unelevated tray-launched runs surface
a cause instead of exiting 1 silently.
1.6.2: command channel retires pooled TLS connections faster than the
server keep-alive (SocketsHttpHandler PooledConnectionIdleTimeout 1s,
PooledConnectionLifetime 2m). Fixes intermittent "unexpected EOF or 0
bytes from the transport stream" on the long-poll claim when the pool
reused a socket the server had already closed between hold windows.
Telemetry was unaffected; this only touched CommandClient.
1.6.1: republished against the current source so both site packages
(Wilmington and West Jefferson) carry the same vetted build. No agent
behavior change; the server now computes warn-only drift from the
service/process state the agent already reports each cycle.
1.6.0: a policy-attached process monitor may carry a launch path set by
a site leader in the WebUI; it rides the /monitors/resolved payload and
the agent honors it only after LaunchPathPolicy validates it (approved
root, exists, launchable extension). A local appsettings.json launchpaths
entry still wins; an unvalidated path is dropped and process_start fails
closed.
1.5.0: the agent fetches its watch list from server policy each
telemetry cycle (GET /api/endpoints/monitors/resolved) and merges it
over local appsettings.json, so a policy-attached process/service
monitor is enumerated and reported without editing the box. Fail-closed:
a failed fetch keeps the local list.
1.4.1: run_script/update_app stage into a fixed, Defender-excludable
folder (C:\ProgramData\ShopDB\EndpointAgent) instead of the LocalSystem
temp path, and run_script user-context waits for exit and reports the
real exit code instead of fire-and-forget "launched".
@@ -19,7 +50,7 @@
(AgentConfig.LaunchPaths, shipped defaults overridable per site).
1.1.0: generic, data-driven service_*/process_* actions carry their
target over the wire, so new policies/apps need no agent rebuild. -->
<Version>1.4.1</Version>
<Version>1.6.3</Version>
<InvariantGlobalization>true</InvariantGlobalization>
<RuntimeIdentifier>win-x64</RuntimeIdentifier>
<SelfContained>true</SelfContained>

16
agent/ShopDbEndpointAgent/appsettings.json
View File

@@ -7,20 +7,10 @@
"utilitiespath": "",
"logpath": "",
"monitors": {
"processes": [
{ "name": "dncMain.exe", "app": "eDNC" },
{ "name": "udc.exe", "app": "Universal Data Collector" },
{ "name": "agent.exe", "app": "MTConnect" }
],
"services": [
{ "name": "MTConnect UDC Agent", "app": "MTConnect" }
]
"processes": [],
"services": []
},
"launchpaths": {
"DncMain.exe": "C:\\Program Files (x86)\\Dnc\\bin\\DncMain.exe",
"UDC.exe": "C:\\Program Files\\UDC\\UDC.exe",
"agent.exe": "C:\\ProgramData\\udc\\mtconnect_udc\\agent\\run.bat"
}
"launchpaths": {}
}

4
docker-compose.airgap.yml
View File

@@ -32,6 +32,10 @@ x-app-env: &app-env
LOG_LEVEL: ${LOG_LEVEL:-INFO}
ZABBIX_URL: ${ZABBIX_URL:-}
ZABBIX_TOKEN: ${ZABBIX_TOKEN:-}
# Channel label for a distributable installer. Empty on the stable (main)
# bundle, 'DEV' on the edge (dev-be) bundle, which lights up the sidebar
# badge. Sourced from .env, which build-channel-installer.ps1 pre-fills.
SHOPDB_ENV_LABEL: ${SHOPDB_ENV_LABEL:-}
services:
db:

27
docker-compose.channel.yml Normal file
View File

@@ -0,0 +1,27 @@
# Channel overlay for side-by-side stable / bleeding-edge stacks.
#
# The base docker-compose.yml is channel-agnostic. This overlay adds only the
# pieces that differ between a stable (main) and an edge (dev-be) stack so the
# two can run at once on one dev workstation without colliding:
#
# - SHOPDB_ENV_LABEL is passed into the api container. When non-empty the
# public settings endpoint surfaces it as env_label and the sidebar renders
# a red DEV badge. Stable leaves it unset, so nothing renders.
# - Ports fall back to the SAME defaults as the base file, so running this
# overlay alone changes nothing; the bootstrap script overrides API_PORT /
# MYSQL_PORT for the edge stack so both channels bind distinct host ports.
#
# The stacks are isolated by compose project name (-p shopdb-stable /
# -p shopdb-edge), which namespaces containers, networks, and the named
# volumes, so each channel keeps its own database and instance state.
#
# Usage (driven by scripts/dev-bootstrap.ps1, shown here for reference):
# docker compose -p shopdb-edge \
# -f docker-compose.yml -f docker-compose.channel.yml up -d
services:
api:
environment:
# Empty on stable. The edge stack sets SHOPDB_ENV_LABEL=DEV in its .env
# (written by the bootstrap script) to light up the badge.
SHOPDB_ENV_LABEL: ${SHOPDB_ENV_LABEL:-}

92
docker-compose.pull.yml Normal file
View File

@@ -0,0 +1,92 @@
# shopdb-flask PULL stack for dev workstations (stable / edge channels).
#
# WHY this exists: pilots behind an inspecting proxy cannot reliably BUILD the
# image from source (npm's many small fetches get reset mid-install). Instead
# CI (.github/workflows/publish-images.yml) builds the image on clean runners
# and publishes it to GHCR; this file just PULLS and RUNS it. One resumable
# image pull survives the proxy where hundreds of npm requests do not.
#
# Nothing is built here (no `build:`), mirroring docker-compose.airgap.yml:
# - api/migrate use `image:` (pulled from GHCR), never `build: .`.
# - NO ./plugins bind mount; the image carries every plugin baked in.
# - a one-shot `migrate` runs db upgrade + plugin upgrade-all + seed before
# api starts, so `up -d` alone brings up a working site.
#
# Driven by scripts/dev-bootstrap.ps1, which sets SHOPDB_IMAGE (the channel
# image ref), SHOPDB_ENV_LABEL, API_PORT and MYSQL_PORT per channel and picks a
# compose project name (-p shopdb-stable / -p shopdb-edge) so the two channels
# run side by side without colliding.
x-app-env: &app-env
FLASK_APP: wsgi.py
FLASK_ENV: production
DATABASE_URL: mysql+pymysql://shopdb:${MYSQL_PASSWORD}@db:3306/shopdb_flask?charset=utf8mb4
SECRET_KEY: ${SECRET_KEY:?SECRET_KEY must be set}
JWT_SECRET_KEY: ${JWT_SECRET_KEY:?JWT_SECRET_KEY must be set}
CORS_ORIGINS: ${CORS_ORIGINS:?CORS_ORIGINS must be set}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
# Empty on stable; 'DEV' on edge, which lights the sidebar badge.
SHOPDB_ENV_LABEL: ${SHOPDB_ENV_LABEL:-}
services:
db:
image: mysql:8.4
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:?MYSQL_ROOT_PASSWORD must be set}
MYSQL_DATABASE: shopdb_flask
MYSQL_USER: shopdb
MYSQL_PASSWORD: ${MYSQL_PASSWORD:?MYSQL_PASSWORD must be set}
volumes:
- db_data:/var/lib/mysql
ports:
- "127.0.0.1:${MYSQL_PORT:-3306}:3306"
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-u", "root", "-p${MYSQL_ROOT_PASSWORD}"]
interval: 10s
timeout: 5s
retries: 5
# One-shot schema + seed. Runs to completion and exits; api waits for it.
# Every step is idempotent, so it is safe to run on every `up`.
migrate:
image: ${SHOPDB_IMAGE:?SHOPDB_IMAGE must be set to the channel image ref}
pull_policy: always
restart: "no"
depends_on:
db:
condition: service_healthy
environment:
<<: *app-env
command:
- sh
- -c
- >
flask db upgrade &&
flask plugin upgrade-all &&
flask seed permissions &&
flask seed settings &&
flask seed reference-data
volumes:
- instance_data:/app/instance
api:
image: ${SHOPDB_IMAGE:?SHOPDB_IMAGE must be set to the channel image ref}
pull_policy: always
restart: unless-stopped
depends_on:
db:
condition: service_healthy
migrate:
condition: service_completed_successfully
environment:
<<: *app-env
ports:
- "${API_PORT:-5001}:5001"
volumes:
- instance_data:/app/instance
volumes:
db_data:
instance_data:

5
docker-compose.yml
View File

@@ -99,6 +99,11 @@ services:
retries: 3
volumes:
- ./plugins:/app/plugins:ro
# Standalone operator scripts shipped inside generated bundles (the
# endpoint-agent uninstaller). Read-only, same pattern as plugins: the
# site-agent packaging route reads uninstall-shopdbagent.ps1 from here at
# download time, so it must be visible in the container.
- ./scripts:/app/scripts:ro
# Self-signed TLS material for the dev/test https listener. Read-only;
# generated on the host under ./certs and never baked into the image.
- ./certs:/app/certs:ro

45
docs/DEV-BUILD-TEST.md Normal file
View File

@@ -0,0 +1,45 @@
# ShopDB dev build - quick test
Two dev builds you can stand up on a Windows box with Docker Desktop. You run
one PowerShell script: it checks your dependencies (offers to winget-install
anything missing), then PULLS a prebuilt image from GHCR and starts the stack in
Docker. Nothing is compiled on your machine - no Node, no npm, no source build -
so it works on a box that has only Docker installed.
## What you need
- Windows with Docker Desktop running.
- Git for Windows (`winget install Git.Git`).
- A GitHub login for this private repo/registry already stored on the box. If
you have cloned this repo before, or signed in to GitHub in Visual Studio or
VS Code, you already have it - the script uses that stored credential
automatically. If Docker cannot pull the image, sign in once with
`docker login ghcr.io` (use a GitHub username and a personal access token that
has `read:packages`), then re-run the launcher.
## Get the launcher
You will be sent `Start-ShopDBDev.ps1` directly. Save it anywhere (for example
your Downloads folder) and open PowerShell in that folder.
## Bleeding edge (main, the trunk)
powershell -ExecutionPolicy Bypass -File .\Start-ShopDBDev.ps1 -Channel edge
Comes up at https://localhost:5011. Red DEV badge in the sidebar. Pulls the
`edge` image.
## Stable (newest release tag)
powershell -ExecutionPolicy Bypass -File .\Start-ShopDBDev.ps1 -Channel stable
Comes up at https://localhost:5001. No DEV badge. Pulls the `stable` image.
## Notes
- Both stacks can run at the same time - they use different ports and separate
Docker volumes, so they do not collide and each keeps its own database.
- Each launch pulls the latest image for the channel, so re-running the launcher
is how you pick up a new build. No local rebuild step.
- First page load shows a self-signed cert warning (expected for a dev box).
- Stop a stack: `docker compose -p shopdb-stable down` (or `shopdb-edge`).

6
docs/PROJECT-MAP.md
View File

@@ -27,7 +27,7 @@ with `flask plugin upgrade-all`. Both are needed on a deploy.
| backups | `backups0003clearlastseen` |
| computers | `computers0004policies` |
| employees | `employees0002photo` |
| endpoints | `endpoints0005cmdpin` |
| endpoints | `endpoints0009opreports` |
| geenforce | `geenforce0004minlib` |
| guides | `guides0001baseline` |
| knowledgebase | `knowledgebase0001anchor` |
@@ -90,6 +90,6 @@ Manifest-less directories under `plugins/` are core frontend surface and always
## Size
- test functions defined: **1587** (parametrised cases collect higher)
- documented API paths: **346** (`docs/openapi.json`, regenerate with `scripts/gen_openapi.py`)
- test functions defined: **1619** (parametrised cases collect higher)
- documented API paths: **353** (`docs/openapi.json`, regenerate with `scripts/gen_openapi.py`)

0
docs/ShopDB-Planner-Tasks-v2.csv Normal file
View File

84
docs/api-inventory.json
View File

@@ -3773,10 +3773,10 @@
"endpoints": [
{
"method": "GET",
"path": "/api/endpoints",
"path": "/api/endpoints/",
"auth": "endpoints.view",
"params": "query: search (hostname substring)",
"purpose": "List endpoints reporting through the ShopDB agent, with last-seen and online state",
"purpose": "List endpoint states. Registered both with and without the trailing slash, so both forms are described here",
"example": "curl -H \"Authorization: Bearer $TOKEN\" http://localhost:5001/api/endpoints"
},
{
@@ -3922,6 +3922,86 @@
"params": "path: command_id; body: the command outcome (status, output)",
"purpose": "Agent reports a claimed command result. Finalizes the command row and, for MTConnect actions that observed run state, caches that on the EndpointState so the UI toggle reflects reality between polls - both in one commit. Collector-token auth, not JWT",
"example": "curl -X POST -H \"X-Collector-Key: $KEY\" -H 'Content-Type: application/json' -d '{\"status\":\"succeeded\",\"output\":\"ok\"}' http://localhost:5001/api/endpoints/commands/1/result"
},
{
"path": "/api/endpoints/monitors",
"method": "GET",
"auth": "jwt + require_permission endpoints.view",
"params": "none",
"purpose": "List every monitor definition, shipped and local, ordered shipped-first then by display name",
"example": "curl http://localhost:5001/api/endpoints/monitors -H \"Authorization: Bearer $TOK\""
},
{
"path": "/api/endpoints/monitors",
"method": "POST",
"auth": "jwt + require_role admin",
"params": "body: kind (process|service, required), displayname (required), processname (required for process), servicename (required for service), launchpath, datacollectorpath, logcollectorpath, isactive (default true)",
"purpose": "Create a local monitor definition; source is forced to 'local' so a caller cannot mint a shipped row",
"example": "curl -X POST http://localhost:5001/api/endpoints/monitors -H \"Authorization: Bearer $TOK\" -H 'Content-Type: application/json' -d '{\"kind\":\"service\",\"displayname\":\"Spooler\",\"servicename\":\"Spooler\"}'"
},
{
"path": "/api/endpoints/monitors/<monitor_id>",
"method": "PUT",
"auth": "jwt + require_role admin",
"params": "path: monitor_id; body: same fields as create, all optional",
"purpose": "Update a local monitor definition; 403 on a shipped row, which is product-owned and read-only",
"example": "curl -X PUT http://localhost:5001/api/endpoints/monitors/4 -H \"Authorization: Bearer $TOK\" -H 'Content-Type: application/json' -d '{\"isactive\":false}'"
},
{
"path": "/api/endpoints/monitors/<monitor_id>",
"method": "DELETE",
"auth": "jwt + require_role admin",
"params": "path: monitor_id",
"purpose": "Delete a local monitor definition; 403 on a shipped row",
"example": "curl -X DELETE http://localhost:5001/api/endpoints/monitors/4 -H \"Authorization: Bearer $TOK\""
},
{
"path": "/api/endpoints/monitors/resolve",
"method": "GET",
"auth": "jwt + require_permission endpoints.view",
"params": "query: pctype (required)",
"purpose": "Resolve the monitor set a PC type should watch: shipped defaults for 'Shopfloor PC' plus whatever active site policy attaches to that type. The human-facing view of what the agent will be told",
"example": "curl 'http://localhost:5001/api/endpoints/monitors/resolve?pctype=Shopfloor+PC' -H \"Authorization: Bearer $TOK\""
},
{
"path": "/api/endpoints/monitors/resolved",
"method": "GET",
"auth": "endpoints collector API key (X-API-Key/Bearer), not JWT",
"params": "query: hostname (required)",
"purpose": "Agent runtime watch-list. The agent calls this each telemetry cycle and receives {processes[], services[]} shaped like its own MonitorConfig, so a policy-attached monitor is watched without editing appsettings.json on the bay. Unknown hostname is a 404; a PC type with no monitors returns empty lists",
"example": "curl 'http://localhost:5001/api/endpoints/monitors/resolved?hostname=F7MHPF24' -H \"X-API-Key: $ENDPOINTS_KEY\""
},
{
"path": "/api/endpoints/operatorreports",
"method": "POST",
"auth": "endpoints collector API key (X-API-Key/Bearer), not JWT",
"params": "multipart/form-data: hostname (required), reason (default 'Other (free text)', truncated to 128), details, screenshot (png, 10 MB cap)",
"purpose": "Agent uploads one operator issue report captured from the tray. The hostname must resolve to a computer asset. The screenshot is stored under a server-generated uuid filename; a retention sweep runs opportunistically so the feed stays bounded without a scheduler",
"example": "curl -X POST http://localhost:5001/api/endpoints/operatorreports -H \"X-API-Key: $ENDPOINTS_KEY\" -F hostname=F7MHPF24 -F 'details=Machine will not home' -F screenshot=@shot.png"
},
{
"path": "/api/endpoints/panel/<asset_id>/operatorreports",
"method": "GET",
"auth": "jwt + require_permission endpoints.view",
"params": "path: asset_id",
"purpose": "Operator Reports asset panel: this endpoint's reports newest first, each carrying a hasimage flag so the viewer knows whether to request the screenshot",
"example": "curl http://localhost:5001/api/endpoints/panel/46/operatorreports -H \"Authorization: Bearer $TOK\""
},
{
"path": "/api/endpoints/operatorreports/<operatorreportid>/image",
"method": "GET",
"auth": "jwt + require_permission endpoints.view",
"params": "path: operatorreportid",
"purpose": "Serve one operator report's screenshot, hardened with the shared upload headers (CSP sandbox + nosniff); 404 when the report carries no image",
"example": "curl http://localhost:5001/api/endpoints/operatorreports/12/image -H \"Authorization: Bearer $TOK\" -o shot.png"
},
{
"method": "GET",
"path": "/api/endpoints",
"auth": "endpoints.view",
"params": "query: search (hostname substring)",
"purpose": "List endpoint states. Registered both with and without the trailing slash, so both forms are described here",
"example": "curl -H \"Authorization: Bearer $TOKEN\" http://localhost:5001/api/endpoints"
}
]
},

430
docs/openapi.json
View File

@@ -21826,8 +21826,8 @@
"tags": [
"plugin-endpoints"
],
"summary": "List endpoints reporting through the ShopDB agent, with last-seen and online state",
"description": "List endpoints reporting through the ShopDB agent, with last-seen and online state\n\n**Auth:** endpoints.view\n\n**Params:** query: search (hostname substring)\n\n**Example:**\n```\ncurl -H \"Authorization: Bearer $TOKEN\" http://localhost:5001/api/endpoints\n```",
"summary": "List endpoint states. Registered both with and without the trailing slash, so both forms are described here",
"description": "List endpoint states. Registered both with and without the trailing slash, so both forms are described here\n\n**Auth:** endpoints.view\n\n**Params:** query: search (hostname substring)\n\n**Example:**\n```\ncurl -H \"Authorization: Bearer $TOKEN\" http://localhost:5001/api/endpoints\n```",
"security": [
{
"bearerAuth": []
@@ -22718,6 +22718,432 @@
}
}
},
"/api/endpoints/monitors": {
"get": {
"tags": [
"plugin-endpoints"
],
"summary": "List every monitor definition, shipped and local, ordered shipped-first then by display name",
"description": "List every monitor definition, shipped and local, ordered shipped-first then by display name\n\n**Auth:** jwt + require_permission endpoints.view\n\n**Params:** none\n\n**Example:**\n```\ncurl http://localhost:5001/api/endpoints/monitors -H \"Authorization: Bearer $TOK\"\n```",
"security": [
{
"bearerAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
}
}
},
"post": {
"tags": [
"plugin-endpoints"
],
"summary": "Create a local monitor definition; source is forced to 'local' so a caller cannot mint a shipped row",
"description": "Create a local monitor definition; source is forced to 'local' so a caller cannot mint a shipped row\n\n**Auth:** jwt + require_role admin\n\n**Params:** body: kind (process|service, required), displayname (required), processname (required for process), servicename (required for service), launchpath, datacollectorpath, logcollectorpath, isactive (default true)\n\n**Example:**\n```\ncurl -X POST http://localhost:5001/api/endpoints/monitors -H \"Authorization: Bearer $TOK\" -H 'Content-Type: application/json' -d '{\"kind\":\"service\",\"displayname\":\"Spooler\",\"servicename\":\"Spooler\"}'\n```",
"security": [
{
"bearerAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
}
},
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"additionalProperties": true,
"description": "body: kind (process|service, required), displayname (required), processname (required for process), servicename (required for service), launchpath, datacollectorpath, logcollectorpath, isactive (default true)"
}
}
}
}
}
},
"/api/endpoints/monitors/{monitor_id}": {
"put": {
"tags": [
"plugin-endpoints"
],
"summary": "Update a local monitor definition; 403 on a shipped row, which is product-owned and read-only",
"description": "Update a local monitor definition; 403 on a shipped row, which is product-owned and read-only\n\n**Auth:** jwt + require_role admin\n\n**Params:** path: monitor_id; body: same fields as create, all optional\n\n**Example:**\n```\ncurl -X PUT http://localhost:5001/api/endpoints/monitors/4 -H \"Authorization: Bearer $TOK\" -H 'Content-Type: application/json' -d '{\"isactive\":false}'\n```",
"security": [
{
"bearerAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
},
"404": {
"description": "No such record."
}
},
"parameters": [
{
"name": "monitor_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"additionalProperties": true,
"description": "path: monitor_id; body: same fields as create, all optional"
}
}
}
}
},
"delete": {
"tags": [
"plugin-endpoints"
],
"summary": "Delete a local monitor definition; 403 on a shipped row",
"description": "Delete a local monitor definition; 403 on a shipped row\n\n**Auth:** jwt + require_role admin\n\n**Params:** path: monitor_id\n\n**Example:**\n```\ncurl -X DELETE http://localhost:5001/api/endpoints/monitors/4 -H \"Authorization: Bearer $TOK\"\n```",
"security": [
{
"bearerAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
},
"404": {
"description": "No such record."
}
},
"parameters": [
{
"name": "monitor_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
]
}
},
"/api/endpoints/monitors/resolve": {
"get": {
"tags": [
"plugin-endpoints"
],
"summary": "Resolve the monitor set a PC type should watch: shipped defaults for 'Shopfloor PC' plus whatever active site policy...",
"description": "Resolve the monitor set a PC type should watch: shipped defaults for 'Shopfloor PC' plus whatever active site policy attaches to that type. The human-facing view of what the agent will be told\n\n**Auth:** jwt + require_permission endpoints.view\n\n**Params:** query: pctype (required)\n\n**Example:**\n```\ncurl 'http://localhost:5001/api/endpoints/monitors/resolve?pctype=Shopfloor+PC' -H \"Authorization: Bearer $TOK\"\n```",
"security": [
{
"bearerAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
}
}
}
},
"/api/endpoints/monitors/resolved": {
"get": {
"tags": [
"plugin-endpoints"
],
"summary": "Agent runtime watch-list. The agent calls this each telemetry cycle and receives {processes[], services[]} shaped like...",
"description": "Agent runtime watch-list. The agent calls this each telemetry cycle and receives {processes[], services[]} shaped like its own MonitorConfig, so a policy-attached monitor is watched without editing appsettings.json on the bay. Unknown hostname is a 404; a PC type with no monitors returns empty lists\n\n**Auth:** endpoints collector API key (X-API-Key/Bearer), not JWT\n\n**Params:** query: hostname (required)\n\n**Example:**\n```\ncurl 'http://localhost:5001/api/endpoints/monitors/resolved?hostname=F7MHPF24' -H \"X-API-Key: $ENDPOINTS_KEY\"\n```",
"security": [
{
"apiKeyAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
}
}
}
},
"/api/endpoints/operatorreports": {
"post": {
"tags": [
"plugin-endpoints"
],
"summary": "Agent uploads one operator issue report captured from the tray. The hostname must resolve to a computer asset. The...",
"description": "Agent uploads one operator issue report captured from the tray. The hostname must resolve to a computer asset. The screenshot is stored under a server-generated uuid filename; a retention sweep runs opportunistically so the feed stays bounded without a scheduler\n\n**Auth:** endpoints collector API key (X-API-Key/Bearer), not JWT\n\n**Params:** multipart/form-data: hostname (required), reason (default 'Other (free text)', truncated to 128), details, screenshot (png, 10 MB cap)\n\n**Example:**\n```\ncurl -X POST http://localhost:5001/api/endpoints/operatorreports -H \"X-API-Key: $ENDPOINTS_KEY\" -F hostname=F7MHPF24 -F 'details=Machine will not home' -F screenshot=@shot.png\n```",
"security": [
{
"apiKeyAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
}
},
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"additionalProperties": true,
"description": "multipart/form-data: hostname (required), reason (default 'Other (free text)', truncated to 128), details, screenshot (png, 10 MB cap)"
}
}
}
}
}
},
"/api/endpoints/panel/{asset_id}/operatorreports": {
"get": {
"tags": [
"plugin-endpoints"
],
"summary": "Operator Reports asset panel: this endpoint's reports newest first, each carrying a hasimage flag so the viewer knows...",
"description": "Operator Reports asset panel: this endpoint's reports newest first, each carrying a hasimage flag so the viewer knows whether to request the screenshot\n\n**Auth:** jwt + require_permission endpoints.view\n\n**Params:** path: asset_id\n\n**Example:**\n```\ncurl http://localhost:5001/api/endpoints/panel/46/operatorreports -H \"Authorization: Bearer $TOK\"\n```",
"security": [
{
"bearerAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
},
"404": {
"description": "No such record."
}
},
"parameters": [
{
"name": "asset_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
]
}
},
"/api/endpoints/operatorreports/{operatorreportid}/image": {
"get": {
"tags": [
"plugin-endpoints"
],
"summary": "Serve one operator report's screenshot, hardened with the shared upload headers (CSP sandbox + nosniff); 404 when the...",
"description": "Serve one operator report's screenshot, hardened with the shared upload headers (CSP sandbox + nosniff); 404 when the report carries no image\n\n**Auth:** jwt + require_permission endpoints.view\n\n**Params:** path: operatorreportid\n\n**Example:**\n```\ncurl http://localhost:5001/api/endpoints/operatorreports/12/image -H \"Authorization: Bearer $TOK\" -o shot.png\n```",
"security": [
{
"bearerAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
},
"404": {
"description": "No such record."
}
},
"parameters": [
{
"name": "operatorreportid",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
]
}
},
"/api/scripts": {
"get": {
"tags": [

19
frontend/src/api/index.js
View File

@@ -178,6 +178,25 @@ export const endpointsApi = {
},
telemetryMeta(assetId) {
return api.get(`/endpoints/asset/${assetId}/telemetrymeta`)
},
// Monitor catalog: reusable process/service definitions. Shipped rows are
// product-owned and read-only; local rows are user-built on the Settings page.
monitors: {
list() {
return api.get('/endpoints/monitors')
},
create(data) {
return api.post('/endpoints/monitors', data)
},
update(monitorid, data) {
return api.put(`/endpoints/monitors/${monitorid}`, data)
},
remove(monitorid) {
return api.delete(`/endpoints/monitors/${monitorid}`)
},
resolve(pctype) {
return api.get('/endpoints/monitors/resolve', { params: { pctype } })
}
}
}

27
frontend/src/assets/style.css
View File

@@ -1068,17 +1068,28 @@ td.actions .btn + .btn {
}
/* Content Grid */
/* Balanced two-column card flow. Uses CSS multicol (not a hand-assigned
grid) so cards distribute by height and the columns stay even no matter
how many cards land on either side. display:contents flattens the two
.content-column wrappers so their cards flow directly into the columns,
which keeps the existing markup unchanged. */
/* Two-column card layout on an explicit CSS grid rather than multicol.
Multicol was abandoned because a child using column-span: all (the full
width Network Interfaces panel) overlaps the shorter column above it when
the two columns have unequal heights - a known multicol spanning bug. Grid
has no such flaw: short fact cards sit two-up and a full-width panel spans
both tracks cleanly via .pap-span. display:contents flattens the two
.content-column wrappers so their cards become direct grid items, which
keeps the existing markup unchanged. */
.content-grid {
column-count: 2;
column-gap: 25px;
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 25px;
align-items: start;
margin-bottom: 25px;
}
/* Grid supplies the gap; the per-card margin was a multicol-only crutch and
would otherwise double the vertical spacing. */
.content-grid > .section-card {
margin-bottom: 0;
}
.content-column {
display: contents;
}
@@ -1522,7 +1533,7 @@ td.actions .btn + .btn {
}
.content-grid {
column-count: 1;
grid-template-columns: 1fr;
}
.audit-footer {

290
frontend/src/components/PluginAssetPanels.vue
View File

@@ -143,21 +143,185 @@
<span v-for="(b, bi) in badges(panel)" :key="bi" class="status-badge"
:style="colorStyle(b.color)">{{ b.label }}</span>
</div>
<!-- operatorreports: a thumbnail grid of operator-submitted reports; a
card expands to a lightbox with the full screenshot and note, and
left/right arrows cycle through the reports. -->
<div v-else-if="panel.render === 'operatorreports'" class="pap-reports">
<template v-if="rows(panel).length">
<div class="pap-report-grid">
<button
v-for="(report, ri) in rows(panel)"
:key="report.operatorreportid"
type="button"
class="pap-report-card"
@click="openReport(panel, ri)"
>
<div class="pap-report-thumb">
<img
v-if="report.hasimage"
:src="reportImageUrl(panel, report)"
:alt="'Screenshot for report ' + report.operatorreportid"
loading="lazy"
/>
<span v-else class="pap-report-noimage">No screenshot</span>
</div>
<div class="pap-report-preview">
<span class="pap-report-text">{{ report.details || '(no note)' }}</span>
<span class="pap-report-when">{{ formatDate(report.createdat) }}</span>
</div>
</button>
</div>
</template>
<div v-else class="pap-empty">
<span class="muted">{{ panel.empty || 'No operator reports.' }}</span>
</div>
</div>
</div>
</div>
</template>
<!-- Lightbox: full screenshot and note for the open report, with left/right
cycling. Rendered once outside the panel loop so the overlay covers the
page rather than a single card. -->
<div
v-if="lightbox.panel"
ref="lightboxEl"
class="pap-lightbox"
role="dialog"
aria-modal="true"
tabindex="-1"
@click.self="closeReport"
@keydown.esc="closeReport"
@keydown.left="cycleReport(-1)"
@keydown.right="cycleReport(1)"
>
<button type="button" class="pap-lb-close" aria-label="Close" @click="closeReport">&times;</button>
<button
type="button"
class="pap-lb-nav pap-lb-prev"
aria-label="Previous report"
:disabled="rows(lightbox.panel).length < 2"
@click.stop="cycleReport(-1)"
>&#8249;</button>
<div class="pap-lb-body">
<div class="pap-lb-image">
<img
v-if="openReportRow && openReportRow.hasimage"
:src="reportImageUrl(lightbox.panel, openReportRow)"
:alt="'Screenshot for report ' + openReportRow.operatorreportid"
/>
<span v-else class="pap-report-noimage">No screenshot</span>
</div>
<div v-if="openReportRow" class="pap-lb-details">
<div class="pap-lb-meta">
<span>{{ openReportRow.reason }}</span>
<span>{{ formatDate(openReportRow.createdat) }}</span>
<span v-if="openReportRow.reportedhostname" class="mono">{{ openReportRow.reportedhostname }}</span>
</div>
<p class="pap-lb-note">{{ openReportRow.details || '(no note)' }}</p>
<div class="pap-lb-count">
{{ lightbox.index + 1 }} / {{ rows(lightbox.panel).length }}
</div>
</div>
</div>
<button
type="button"
class="pap-lb-nav pap-lb-next"
aria-label="Next report"
:disabled="rows(lightbox.panel).length < 2"
@click.stop="cycleReport(1)"
>&#8250;</button>
</div>
</template>
<script setup>
import { ref, computed, watch } from 'vue'
import { ref, computed, watch, onUnmounted, nextTick } from 'vue'
import api from '../api'
import { colorStyle } from '@/utils/colorStyle'
import {
toApiPath, rows, panelVisible, mapTitle, mapBadge, mapMeta, manageLink,
keyvalueFields, tableColumns, cell, badges, tabSections, rollupStatus,
rowAction, facets, facetedRows,
rowAction, facets, facetedRows, formatValue,
} from './pluginAssetPanels'
// Operator report screenshots are served by a JWT-gated route, so a bare
// <img src> (which sends no Authorization header) would 401. Fetch each image
// as a blob through the api instance and hand the <img> an object URL instead.
// Cache by report id so cycling the lightbox does not refetch, and revoke on
// unmount so the blobs are not leaked.
const reportImageUrls = ref({})
function formatDate(value) {
return formatValue(value, { format: 'date' })
}
function reportImageUrl(panel, report) {
const key = report.operatorreportid
if (reportImageUrls.value[key]) return reportImageUrls.value[key]
const base = panel.imagebase || '/api/endpoints/operatorreports'
const path = toApiPath(base + '/' + key + '/image', props.assetid)
api.get(path, { responseType: 'blob' })
.then((response) => {
reportImageUrls.value = {
...reportImageUrls.value,
[key]: URL.createObjectURL(response.data),
}
})
.catch(() => {})
return ''
}
// Release every cached object URL. Called on reload and unmount so a long-lived
// detail page does not leak blob handles as the operator revisits PCs.
function revokeReportImageUrls() {
Object.values(reportImageUrls.value).forEach((url) => {
try { URL.revokeObjectURL(url) } catch { /* already revoked */ }
})
reportImageUrls.value = {}
}
onUnmounted(revokeReportImageUrls)
// The open report lightbox: which panel and which row index. A null panel
// means closed.
const lightbox = ref({ panel: null, index: 0 })
const openReportRow = computed(() => {
if (!lightbox.value.panel) return null
return rows(lightbox.value.panel)[lightbox.value.index] || null
})
// The overlay owns Escape and the arrow keys, and a keydown handler on a div
// only fires while that div has focus. Nothing moved focus to it, so every
// keyboard binding on the lightbox was dead until the user happened to click
// the backdrop. Focus it once it has rendered, and hand focus back to the
// thumbnail that opened it on close, so keyboard users are not dropped at the
// top of the document.
const lightboxEl = ref(null)
let reopenFocus = null
function openReport(panel, index) {
reopenFocus = typeof document !== 'undefined' ? document.activeElement : null
lightbox.value = { panel, index }
nextTick(() => lightboxEl.value?.focus())
}
function closeReport() {
lightbox.value = { panel: null, index: 0 }
const returnTo = reopenFocus
reopenFocus = null
nextTick(() => returnTo?.focus?.())
}
function cycleReport(delta) {
if (!lightbox.value.panel) return
const total = rows(lightbox.value.panel).length
if (total < 2) return
const next = (lightbox.value.index + delta + total) % total
lightbox.value = { ...lightbox.value, index: next }
}
// Which filter chips are active, per panel id. Deliberately NOT reset when the
// data refreshes: a viewer who filtered to criticals and then hit refresh means
// to still be looking at criticals.
@@ -339,6 +503,8 @@ async function waitForFreshTelemetry(baseline, { attempts = 15, delayMs = 1000 }
// Fetch the declared panels for this asset, then each panel's data endpoint.
async function load() {
panels.value = []
revokeReportImageUrls()
closeReport()
if (!props.assetid) return
let declared
try {
@@ -447,13 +613,13 @@ defineExpose({ refresh: load, populatedGroups })
Network interfaces is the case that forced this: adapter, address, MAC,
speed and state at half page width either squeezed every cell to three
lines or handed the reader a horizontal scrollbar, and a scrollbar for
data that simply needs room is a layout failure, not a feature. column-span
is the multi-column layout's own answer, so the short fact cards stay two-up
around it.
data that simply needs room is a layout failure, not a feature. The parent
.content-grid is a CSS grid, so grid-column: 1 / -1 makes the panel span
every track while the short fact cards stay two-up around it.
The wrapper keeps overflow-x as a floor for the genuinely too-wide case on
a narrow viewport, where nothing else can help. */
.pap-span { column-span: all; }
.pap-span { grid-column: 1 / -1; }
.pap-table-wrap { overflow-x: auto; max-width: 100%; }
.pap-table { width: 100%; border-collapse: collapse; font-size: 0.85rem; }
.pap-table th, .pap-table td { text-align: left; padding: 0.4rem 0.6rem; border-bottom: 1px solid var(--border); }
@@ -476,4 +642,116 @@ defineExpose({ refresh: load, populatedGroups })
}
.mono { font-family: monospace; }
.muted { color: var(--text-light); }
/* Operator reports: a responsive thumbnail grid, each card a screenshot over a
one-line note preview. Clicking a card opens the lightbox. */
.pap-report-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(9rem, 1fr));
gap: 0.75rem;
}
.pap-report-card {
display: flex;
flex-direction: column;
padding: 0;
border: 1px solid var(--border);
border-radius: 6px;
background: var(--bg);
color: var(--text);
cursor: pointer;
overflow: hidden;
text-align: left;
}
.pap-report-card:hover { border-color: var(--primary); }
.pap-report-thumb {
aspect-ratio: 16 / 10;
display: flex;
align-items: center;
justify-content: center;
background: var(--bg-alt, #1b1b1b);
overflow: hidden;
}
.pap-report-thumb img { width: 100%; height: 100%; object-fit: cover; }
.pap-report-noimage { font-size: 0.78rem; color: var(--text-light); }
.pap-report-preview {
display: flex;
flex-direction: column;
gap: 0.2rem;
padding: 0.45rem 0.55rem;
}
.pap-report-text {
font-size: 0.82rem;
color: var(--text);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.pap-report-when { font-size: 0.72rem; color: var(--text-light); }
/* Lightbox: a page-covering overlay with the full screenshot, the note, and
left/right cycling controls. */
.pap-lightbox {
position: fixed;
inset: 0;
z-index: 1000;
display: flex;
align-items: center;
justify-content: center;
gap: 0.5rem;
padding: 2rem;
background: rgba(0, 0, 0, 0.8);
}
.pap-lb-body {
display: flex;
flex-direction: column;
gap: 0.75rem;
max-width: 90vw;
max-height: 90vh;
background: var(--bg);
border: 1px solid var(--border);
border-radius: 8px;
padding: 1rem;
overflow: auto;
}
.pap-lb-image {
display: flex;
align-items: center;
justify-content: center;
min-height: 0;
}
.pap-lb-image img { max-width: 100%; max-height: 70vh; object-fit: contain; }
.pap-lb-details { display: flex; flex-direction: column; gap: 0.4rem; }
.pap-lb-meta {
display: flex;
flex-wrap: wrap;
gap: 0.75rem;
font-size: 0.8rem;
color: var(--text-light);
}
.pap-lb-note { margin: 0; color: var(--text); white-space: pre-wrap; overflow-wrap: anywhere; }
.pap-lb-count { font-size: 0.78rem; color: var(--text-light); font-variant-numeric: tabular-nums; }
.pap-lb-close {
position: absolute;
top: 1rem;
right: 1rem;
background: none;
border: none;
color: #fff;
font-size: 1.8rem;
line-height: 1;
cursor: pointer;
}
.pap-lb-nav {
flex: 0 0 auto;
background: rgba(255, 255, 255, 0.1);
border: none;
color: #fff;
font-size: 2rem;
line-height: 1;
padding: 0.5rem 0.9rem;
border-radius: 6px;
cursor: pointer;
}
.pap-lb-nav:hover:not(:disabled) { background: rgba(255, 255, 255, 0.25); }
.pap-lb-nav:disabled { opacity: 0.3; cursor: default; }
</style>

2
frontend/src/components/pluginAssetPanels.js
View File

@@ -33,6 +33,7 @@ export function rows(panel) {
if (Array.isArray(data)) list = data
else if (data && Array.isArray(data.rows)) list = data.rows
else if (data && Array.isArray(data.items)) list = data.items
else if (data && Array.isArray(data.reports)) list = data.reports
else return []
// Optional panel-level row filter so two panels can share one endpoint feed
// and each show a slice of it (for example Tracked Services vs Tracked
@@ -258,5 +259,6 @@ export function panelVisible(panel) {
if (panel.render === 'keyvalue') return keyvalueFields(panel).length > 0
if (panel.render === 'table') return rows(panel).length > 0 || !!panel.empty
if (panel.render === 'badge') return badges(panel).length > 0
if (panel.render === 'operatorreports') return rows(panel).length > 0 || !!panel.empty
return false
}

8
frontend/src/utils/siteSettings.js
View File

@@ -37,6 +37,14 @@ export async function getFacilityName() {
return getSetting('facility_name', 'ShopDB')
}
// Environment channel label (e.g. 'DEV') injected by the backend from
// SHOPDB_ENV_LABEL. Empty on stable/main deployments, which renders no badge.
// Drives the red DEV tag in the sidebar so a bleeding-edge container is
// obviously distinct from a stable one running side by side.
export async function getEnvLabel() {
return getSetting('env_label', '')
}
// Main site logo (sidebar, login, dashboard header). Fallback = shipped GE mark.
export async function getSiteLogo() {
return withBase(await getSetting('site_logo', '/ge-aerospace-logo.svg'))

22
frontend/src/views/AppLayout.vue
View File

@@ -4,6 +4,9 @@
<div class="sidebar-header">
<img :src="siteLogo" alt="Site logo" class="sidebar-logo" />
<h1>{{ facilityName }}</h1>
<!-- Only present when the container sets SHOPDB_ENV_LABEL (edge/dev
channel). Stable leaves it empty, so nothing renders. --><span
v-if="envLabel" class="env-badge">{{ envLabel }}</span>
<!-- Under the site name, where the classic site put it. -->
<div class="fiscal-week">
<span class="fiscal-week-label">Fiscal Week</span>
@@ -122,7 +125,7 @@ import { useAuthStore } from '../stores/auth'
import { currentTheme, toggleTheme } from '../stores/theme'
import { dashboardApi, notificationsApi } from '../api'
import { fiscalWeek } from '@/utils/fiscalWeek'
import { getFacilityName, getSiteLogo, getServicenowUrls } from '../utils/siteSettings'
import { getFacilityName, getSiteLogo, getServicenowUrls, getEnvLabel } from '../utils/siteSettings'
import { withBase } from '../utils/basePath'
const router = useRouter()
@@ -155,6 +158,7 @@ const cansearch = ref(false)
const navItems = ref([])
const activeNotifications = ref([])
const facilityName = ref('ShopDB')
const envLabel = ref('')
// A tab left open overnight would otherwise show last week's number until
// somebody reloaded. Recomputed on a timer rather than only at mount.
@@ -286,6 +290,7 @@ onMounted(async () => {
fiscalWeekTimer = setInterval(() => { currentFiscalWeek.value = fiscalWeek() }, 30 * 60 * 1000)
getFacilityName().then(name => { facilityName.value = name })
getEnvLabel().then(label => { envLabel.value = label })
getSiteLogo().then(logo => { siteLogo.value = logo })
getServicenowUrls().then(config => { servicenowConfig.value = config })
@@ -352,6 +357,21 @@ onUnmounted(() => {
</script>
<style scoped>
/* Bleeding-edge channel badge. Loud red on purpose so a dev container is never
mistaken for stable when the two run side by side. Only rendered when the
backend reports a non-empty env_label (SHOPDB_ENV_LABEL). */
.env-badge {
display: inline-block;
margin-top: 0.35rem;
padding: 0.1rem 0.5rem;
border-radius: 3px;
background: #c62828;
color: #fff;
font-size: 0.62rem;
font-weight: 700;
letter-spacing: 0.12em;
text-transform: uppercase;
}
/* Fiscal week, under the site name, where the classic site put it. The number
carries the weight; the label is small because it never changes. */
.fiscal-week {

38
plugins/computers/api/policy_routes.py
View File

@@ -40,6 +40,23 @@ def _known_actions():
return set(COMMAND_ALLOWLIST.keys())
def _is_valid_action(name):
"""True when an action name is either an allowlisted command or a monitor
attachment (monitor:<id>) pointing at an existing monitor definition.
Monitor actions are intentionally kept out of the command allowlist so they
are never enqueued as commands; they are only valid as policy attachments.
"""
if name in _known_actions():
return True
from plugins.endpoints.commands import monitor_id_from_action
from plugins.endpoints.models import EndpointMonitor
monitorid = monitor_id_from_action(name)
if monitorid is None:
return False
return db.session.get(EndpointMonitor, monitorid) is not None
def _configpolicy_dict(policy):
data = policy.to_dict()
data['actions'] = [a.to_dict() for a in policy.actions]
@@ -89,11 +106,15 @@ def _apply_actions(policy, actions):
Returns an error_response on failure, else None.
"""
known = _known_actions()
policy.actions.clear()
# Flush the deletes before re-appending. The (configurationpolicyid, action)
# unique constraint collides if SQLAlchemy orders a re-added action's INSERT
# before the old row's DELETE in the same flush, which surfaces as a generic
# "value already in use" IntegrityError on an otherwise valid edit.
db.session.flush()
for row in actions or []:
name = (row.get('action') or '').strip()
if name not in known:
if not _is_valid_action(name):
return error_response(ErrorCodes.VALIDATION_ERROR,
f'Unknown action: {name!r}')
mode = (row.get('mode') or ACTION_MODE_GATE).strip()
@@ -259,6 +280,10 @@ def get_business_policy(policy_id: int):
def _apply_items(policy, items):
"""Replace a business policy's configuration-policy items, validating each."""
policy.items.clear()
# Flush the deletes before re-appending so the (businesspolicyid,
# configurationpolicyid) unique constraint does not collide with a re-added
# item whose INSERT would otherwise race ahead of the old row's DELETE.
db.session.flush()
seen = set()
for order, row in enumerate(items or []):
cpid = row.get('configurationpolicyid')
@@ -375,14 +400,19 @@ def delete_business_policy(policy_id: int):
@computers_bp.route('/policies/actions', methods=['GET'])
@require_permission('computers.view')
def list_policy_actions():
"""The allowlisted action names + labels a policy may reference."""
"""The allowlisted action names + labels a policy may reference.
Returns command actions tagged kind='command'. Monitor attachments are not
listed here: the UI builds that branch from the endpoints monitor catalog
(endpointsApi.monitors) so newly created monitors appear without a reload.
"""
gate = _feature_gate()
if gate:
return gate
from plugins.endpoints.commands import COMMAND_ALLOWLIST
return success_response([
{'action': name, 'label': meta.get('label'),
'description': meta.get('description')}
'description': meta.get('description'), 'kind': 'command'}
for name, meta in COMMAND_ALLOWLIST.items()
])

4
plugins/computers/frontend/routes.js
View File

@@ -70,8 +70,8 @@ export default [
meta: { requiresAuth: true, requiresAdmin: true, plugin: 'computers' }
},
{
path: 'settings/business-policies',
name: 'business-policies',
path: 'settings/site-policies',
name: 'site-policies',
component: () => import('./views/BusinessPoliciesList.vue'),
meta: { requiresAuth: true, requiresAdmin: true, plugin: 'computers' }
}

265
plugins/computers/frontend/views/ActionPicker.vue Normal file
View File

@@ -0,0 +1,265 @@
<template>
<div class="action-picker" @keydown.esc="closeDropdown">
<input
ref="inputEl"
type="text"
class="form-control"
:placeholder="placeholder"
v-model="query"
@focus="openDropdown"
@input="openDropdown"
/>
<div v-if="open" class="action-dropdown">
<!-- Command level: commands + a Monitor branch entry -->
<template v-if="level === 'command'">
<div
v-for="opt in filteredCommands"
:key="opt.action"
class="action-option"
@mousedown.prevent="selectCommand(opt)"
>
{{ opt.label || opt.action }}
</div>
<div
class="action-option action-branch"
@mousedown.prevent="enterMonitorLevel"
>
Monitor...
</div>
<div v-if="filteredCommands.length === 0" class="action-empty">No matching actions</div>
</template>
<!-- Monitor level: two create entries pinned on top, then existing monitors -->
<template v-else-if="level === 'monitor'">
<div class="action-option action-create" @mousedown.prevent="startCreate('process')">
&lt;Create Process Monitor&gt;
</div>
<div class="action-option action-create" @mousedown.prevent="startCreate('service')">
&lt;Create Service Monitor&gt;
</div>
<div class="action-divider"></div>
<div
v-for="m in filteredMonitors"
:key="m.endpointmonitorid"
class="action-option"
@mousedown.prevent="selectMonitor(m)"
>
{{ m.displayname }} <span class="monitor-kind">({{ m.kind }})</span>
</div>
<div v-if="filteredMonitors.length === 0" class="action-empty">No matching monitors</div>
</template>
<!-- Create level: inline text field to name the new process/service monitor -->
<template v-else>
<div class="action-create-form" @mousedown.prevent>
<label class="action-create-label">
{{ createkind === 'service' ? 'Service name to monitor' : 'Process name to monitor' }}
</label>
<input
ref="createInputEl"
type="text"
class="form-control"
:placeholder="createkind === 'service' ? 'e.g. Spooler' : 'e.g. agent.exe'"
v-model="createname"
@keydown.enter.prevent="submitCreate"
@keydown.esc.prevent="cancelCreate"
/>
<template v-if="createkind === 'process'">
<label class="action-create-label">
Launch path (optional)
</label>
<input
type="text"
class="form-control"
placeholder="e.g. C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe"
v-model="createlaunchpath"
@keydown.enter.prevent="submitCreate"
@keydown.esc.prevent="cancelCreate"
/>
</template>
<div class="action-create-actions">
<button type="button" class="btn btn-primary btn-sm" :disabled="!createname.trim() || creating" @mousedown.prevent="submitCreate">
Create
</button>
<button type="button" class="btn btn-secondary btn-sm" @mousedown.prevent="cancelCreate">
Cancel
</button>
</div>
</div>
</template>
</div>
</div>
</template>
<script setup>
import { ref, computed, watch, nextTick } from 'vue'
import { endpointsApi } from '@/api'
import { apiError } from '@/utils/apiError'
import { useToast } from '@/composables/toast'
const props = defineProps({
modelValue: { type: String, default: '' },
commandActions: { type: Array, default: () => [] },
monitors: { type: Array, default: () => [] }
})
const emit = defineEmits(['update:modelValue', 'monitor-created'])
const toast = useToast()
const inputEl = ref(null)
const createInputEl = ref(null)
const open = ref(false)
// 'command' -> top level, 'monitor' -> monitor list, 'create' -> inline name form
const level = ref('command')
const query = ref('')
const createkind = ref('process')
const createname = ref('')
const createlaunchpath = ref('')
const creating = ref(false)
// Show the chosen action's friendly label in the input when not actively typing.
watch(() => props.modelValue, syncLabel, { immediate: true })
watch(() => props.monitors, () => { if (!open.value) syncLabel() })
function syncLabel() {
query.value = labelFor(props.modelValue)
}
function labelFor(action) {
if (!action) return ''
if (action.startsWith('monitor:')) {
const id = Number(action.slice('monitor:'.length))
const monitor = props.monitors.find(m => m.endpointmonitorid === id)
return monitor ? `Monitor: ${monitor.displayname} (${monitor.kind})` : action
}
const command = props.commandActions.find(c => c.action === action)
return command ? (command.label || command.action) : action
}
const placeholder = computed(() =>
level.value === 'monitor' ? 'Search monitors...' : 'Search actions...')
const filteredCommands = computed(() => {
const term = query.value.trim().toLowerCase()
if (!term) return props.commandActions
return props.commandActions.filter(c =>
(c.label || c.action).toLowerCase().includes(term))
})
const filteredMonitors = computed(() => {
const term = query.value.trim().toLowerCase()
if (!term) return props.monitors
return props.monitors.filter(m => m.displayname.toLowerCase().includes(term))
})
function openDropdown() {
open.value = true
}
function closeDropdown() {
open.value = false
level.value = 'command'
resetCreate()
syncLabel()
}
function enterMonitorLevel() {
level.value = 'monitor'
query.value = ''
inputEl.value?.focus()
}
function selectCommand(opt) {
emit('update:modelValue', opt.action)
open.value = false
level.value = 'command'
query.value = opt.label || opt.action
}
function selectMonitor(monitor) {
emit('update:modelValue', `monitor:${monitor.endpointmonitorid}`)
open.value = false
level.value = 'command'
query.value = `Monitor: ${monitor.displayname} (${monitor.kind})`
}
async function submitCreate() {
const displayname = createname.value.trim()
if (!displayname || creating.value) return
const kind = createkind.value
const payload = {
kind,
displayname,
processname: kind === 'process' ? displayname : '',
servicename: kind === 'service' ? displayname : '',
isactive: true
}
if (kind === 'process' && createlaunchpath.value.trim()) {
payload.launchpath = createlaunchpath.value.trim()
}
creating.value = true
try {
const response = await endpointsApi.monitors.create(payload)
const created = response.data.data
emit('monitor-created', created)
emit('update:modelValue', `monitor:${created.endpointmonitorid}`)
open.value = false
level.value = 'command'
resetCreate()
query.value = `Monitor: ${created.displayname} (${created.kind})`
} catch (err) {
toast.error(apiError(err, 'Failed to create monitor'))
} finally {
creating.value = false
}
}
function startCreate(kind) {
createkind.value = kind
createname.value = ''
createlaunchpath.value = ''
level.value = 'create'
nextTick(() => createInputEl.value?.focus())
}
function resetCreate() {
createname.value = ''
createlaunchpath.value = ''
creating.value = false
}
function cancelCreate() {
resetCreate()
level.value = 'monitor'
nextTick(() => inputEl.value?.focus())
}
</script>
<style scoped>
.action-picker { position: relative; flex: 1; min-width: 220px; }
.action-dropdown {
position: absolute;
z-index: 20;
left: 0;
right: 0;
margin-top: 2px;
max-height: 320px;
overflow-y: auto;
background: var(--bg-card-solid);
color: var(--text);
border: 1px solid var(--border);
border-radius: 4px;
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.35);
}
.action-option { padding: 8px 12px; cursor: pointer; color: var(--text); }
.action-option:hover { background: var(--bg); }
.action-branch { font-style: italic; }
.action-create { font-weight: 600; color: var(--primary); }
.action-divider { height: 1px; background: var(--border); margin: 2px 0; }
.action-empty { padding: 6px 10px; color: var(--text-light); }
.monitor-kind { color: var(--text-light); }
.action-create-form { padding: 10px; display: flex; flex-direction: column; gap: 8px; }
.action-create-label { font-size: 12px; color: var(--text-light); }
.action-create-actions { display: flex; gap: 8px; }
</style>

12
plugins/computers/frontend/views/BusinessPoliciesList.vue
View File

@@ -1,8 +1,8 @@
<template>
<div>
<div class="page-header">
<h2>Business Policies</h2>
<button class="btn btn-primary" @click="openModal()">+ Add Business Policy</button>
<h2>Site Policies</h2>
<button class="btn btn-primary" @click="openModal()">+ Add Site Policy</button>
</div>
<div class="card">
@@ -42,7 +42,7 @@
</td>
</tr>
<tr v-if="policies.length === 0">
<td colspan="6" style="text-align:center; color: var(--text-light);">No business policies</td>
<td colspan="6" style="text-align:center; color: var(--text-light);">No site policies</td>
</tr>
</tbody>
</table>
@@ -53,7 +53,7 @@
<div v-if="showModal" class="modal-overlay">
<div class="modal modal-lg">
<div class="modal-header">
<h3>{{ editing ? 'Edit Business Policy' : 'Add Business Policy' }}</h3>
<h3>{{ editing ? 'Edit Site Policy' : 'Add Site Policy' }}</h3>
</div>
<form @submit.prevent="save">
<div class="modal-body">
@@ -196,14 +196,14 @@ async function save() {
closeModal()
loadData()
} catch (err) {
error.value = apiError(err, 'Failed to save business policy')
error.value = apiError(err, 'Failed to save site policy')
} finally {
saving.value = false
}
}
async function remove(p) {
if (!confirm(`Delete business policy "${p.name}"?`)) return
if (!confirm(`Delete site policy "${p.name}"?`)) return
try {
await computersApi.policies.business.remove(p.businesspolicyid)
loadData()

137
plugins/computers/frontend/views/ConfigurationPoliciesList.vue
View File

@@ -25,7 +25,7 @@
<td class="cell-truncate" :title="p.description">{{ p.description || '-' }}</td>
<td>
<span v-for="a in p.actions" :key="a.action" class="badge" style="margin-right:4px;">
{{ a.action }} ({{ a.mode }})
{{ actionLabel(a.action) }} ({{ a.mode }})
</span>
<span v-if="!p.actions || p.actions.length === 0" style="color: var(--text-light);">none</span>
</td>
@@ -74,17 +74,23 @@
<div class="form-group">
<label>Actions</label>
<div v-for="(a, index) in form.actions" :key="index" class="action-row">
<select v-model="a.action" class="form-control">
<option :value="''" disabled>Select action...</option>
<option v-for="opt in actionCatalog" :key="opt.action" :value="opt.action">
{{ opt.label || opt.action }}
</option>
</select>
<ActionPicker
v-model="a.action"
:command-actions="actionCatalog"
:monitors="monitors"
@monitor-created="onMonitorCreated"
/>
<select v-model="a.mode" class="form-control mode-select">
<option value="gate">Allow (gate)</option>
<option value="desired">Desired state</option>
</select>
<button type="button" class="btn btn-danger btn-sm" @click="form.actions.splice(index, 1)">Remove</button>
<p v-if="showDesiredWarning(a)" class="desired-warning">
Desired state keeps this app alive: on agent check-in the
server will try to START the process if a launch path is
mapped. Use gate to only report status with Start/Stop in the
Monitor UI.
</p>
</div>
<button type="button" class="btn btn-secondary btn-sm" @click="addAction">+ Add action</button>
</div>
@@ -103,10 +109,11 @@
<script setup>
import { ref, onMounted } from 'vue'
import { computersApi, applicationsApi } from '@/api'
import { computersApi, applicationsApi, endpointsApi } from '@/api'
import { useToast } from '@/composables/toast'
import { apiError } from '@/utils/apiError'
import ScopeNodeEditor from './ScopeNodeEditor.vue'
import ActionPicker from './ActionPicker.vue'
const toast = useToast()
@@ -114,6 +121,7 @@ const policies = ref([])
const pctypes = ref([])
const apps = ref([])
const actionCatalog = ref([])
const monitors = ref([])
const loading = ref(true)
const showModal = ref(false)
@@ -132,21 +140,60 @@ onMounted(() => loadData())
async function loadData() {
loading.value = true
try {
const [pol, types, appList, actions] = await Promise.all([
// allSettled, not all: one failing endpoint must not blank the others.
// A rejected Promise.all left PC types and apps empty when only monitors
// 500'd, which looked like data loss.
const [pol, types, appList, actions, monitorList] = await Promise.allSettled([
computersApi.policies.configuration.list(),
computersApi.types.list({ perpage: 200 }),
applicationsApi.listAll({ perpage: 200 }),
computersApi.policies.actions()
computersApi.policies.actions(),
endpointsApi.monitors.list()
])
policies.value = pol.data.data || []
pctypes.value = types.data.data || []
// applicationsApi.listAll resolves to a plain array (fetchAllPages), not an
// axios response, so read it directly. Reading .data on the array throws and
// rejects the whole Promise.all, blanking every dropdown.
apps.value = Array.isArray(appList) ? appList : (appList.data?.data || appList.data || [])
actionCatalog.value = actions.data.data || []
} catch (err) {
console.error('Error loading configuration policies:', err)
const failures = []
if (pol.status === 'fulfilled') {
policies.value = pol.value.data.data || []
} else {
failures.push('policies')
console.error('Error loading policies:', pol.reason)
}
if (types.status === 'fulfilled') {
pctypes.value = types.value.data.data || []
} else {
failures.push('PC types')
console.error('Error loading PC types:', types.reason)
}
if (appList.status === 'fulfilled') {
// applicationsApi.listAll resolves to a plain array (fetchAllPages), not
// an axios response, so read it directly.
const value = appList.value
apps.value = Array.isArray(value) ? value : (value.data?.data || value.data || [])
} else {
failures.push('applications')
console.error('Error loading applications:', appList.reason)
}
if (actions.status === 'fulfilled') {
actionCatalog.value = actions.value.data.data || []
} else {
failures.push('actions')
console.error('Error loading actions:', actions.reason)
}
if (monitorList.status === 'fulfilled') {
monitors.value = monitorList.value.data.data || []
} else {
failures.push('monitors')
console.error('Error loading monitors:', monitorList.reason)
}
if (failures.length) {
toast.error(`Failed to load: ${failures.join(', ')}. Some options may be unavailable.`)
}
} finally {
loading.value = false
}
@@ -177,6 +224,30 @@ function addAction() {
form.value.actions.push({ action: '', mode: 'gate' })
}
function onMonitorCreated(monitor) {
// Add the new definition to the local catalog so its label resolves and it
// is searchable on other action rows without a full reload.
if (monitor && !monitors.value.some(m => m.endpointmonitorid === monitor.endpointmonitorid)) {
monitors.value.push(monitor)
}
}
function actionLabel(action) {
if (!action) return action
if (action.startsWith('monitor:')) {
const id = Number(action.slice('monitor:'.length))
const monitor = monitors.value.find(m => m.endpointmonitorid === id)
return monitor ? `Monitor: ${monitor.displayname}` : action
}
const command = actionCatalog.value.find(c => c.action === action)
return command ? (command.label || command.action) : action
}
function showDesiredWarning(action) {
return !!action && action.mode === 'desired'
&& !!action.action && action.action.startsWith('monitor:')
}
async function save() {
saving.value = true
error.value = ''
@@ -213,11 +284,35 @@ async function remove(p) {
</script>
<style scoped>
.modal-lg {
max-width: 760px;
}
.modal-lg .modal-body {
max-height: 70vh;
overflow-y: auto;
}
.modal-lg :deep(.form-group) {
margin-bottom: 1.1rem;
}
.modal-lg :deep(.form-control) {
font-size: 14px;
padding: 0.55rem 0.7rem;
}
.scope-builder {
padding: 10px;
border: 1px solid var(--border-color, #444);
border: 1px solid var(--border);
border-radius: 4px;
}
.action-row { display: flex; gap: 8px; align-items: center; margin-bottom: 6px; }
.action-row { display: flex; flex-wrap: wrap; gap: 8px; align-items: center; margin-bottom: 6px; }
.mode-select { max-width: 160px; }
.desired-warning {
flex-basis: 100%;
margin: 2px 0 0;
padding: 6px 8px;
font-size: 0.85em;
color: var(--warning-text, #8a6d3b);
background: var(--warning-bg, rgba(240, 173, 78, 0.12));
border: 1px solid var(--warning-border, rgba(240, 173, 78, 0.4));
border-radius: 4px;
}
</style>

2
plugins/computers/models/__init__.py
View File

@@ -13,6 +13,7 @@ from .policy import (
BusinessPolicy,
BusinessPolicyItem,
POLICY_SOURCE_LOCAL,
POLICY_SOURCE_SHIPPED,
ACTION_MODE_DESIRED,
ACTION_MODE_GATE,
)
@@ -28,6 +29,7 @@ __all__ = [
'BusinessPolicy',
'BusinessPolicyItem',
'POLICY_SOURCE_LOCAL',
'POLICY_SOURCE_SHIPPED',
'ACTION_MODE_DESIRED',
'ACTION_MODE_GATE',
]

6
plugins/computers/models/policy.py
View File

@@ -27,6 +27,12 @@ from .jsontext import JSONText
# owner. Kept a plain string, not an enum column, so adding a source is data.
POLICY_SOURCE_LOCAL = 'local'
# Product-owned rows we ship (the shopfloor app monitor bundle). Same read-only
# treatment as any non-local source: the CRUD API refuses to edit or delete
# them, the seeder reconciles them. Distinct from 'local' so a site's own
# policies are never confused with what the product shipped.
POLICY_SOURCE_SHIPPED = 'shipped'
class ConfigurationPolicy(BaseModel):
"""A single reusable policy: a scope plus the actions it applies.

6
plugins/computers/plugin.py
View File

@@ -559,9 +559,9 @@ class ComputersPlugin(BasePlugin):
},
{
'group': 'Computers',
'to': '/settings/business-policies',
'to': '/settings/site-policies',
'icon': 'layers',
'title': 'Business policies',
'title': 'Site policies',
'description': 'Ordered collections of configuration policies '
'assigned by priority',
'position': 28,
@@ -633,7 +633,7 @@ class ComputersPlugin(BasePlugin):
'valuetype': 'boolean',
'category': 'computers',
'description': 'Master switch for RMM-style Configuration and '
'Business Policies. Off by default: models exist '
'Site Policies. Off by default: models exist '
'but the CRUD routes, WebUI, and endpoint '
'enforcement stay inert so the whole feature can '
'be rolled back without a schema change.',

58
plugins/computers/services/policyresolver.py
View File

@@ -166,3 +166,61 @@ def effective_policy_for(assetid):
business_policies = BusinessPolicy.query.filter_by(isactive=True).all()
return build_effective_policy(computer, business_policies)
def _monitor_action_ids(effective):
"""Extract attached monitor ids from an EffectivePolicy's action names.
Monitor attachment rides the policy layer as an action named
'monitor:<endpointmonitorid>' (see plugins/endpoints/commands.py). This
reads those ids back out of a resolved policy so a caller can turn them into
concrete monitor definitions.
"""
ids = set()
for name in effective.allowed_actions():
if isinstance(name, str) and name.startswith('monitor:'):
raw = name[len('monitor:'):]
try:
ids.add(int(raw))
except (TypeError, ValueError):
continue
return ids
def monitor_ids_for_asset(assetid):
"""Set of endpointmonitorids attached to an endpoint by effective policy."""
return _monitor_action_ids(effective_policy_for(assetid))
def monitor_ids_for_pctype(computertypeid):
"""Set of endpointmonitorids attached to a PC type by effective policy.
Folds active Business Policies against a synthetic context carrying only the
PC type (no installed apps), which is what the agent runtime-fetch path needs
when it resolves its monitor set by type rather than by a specific asset.
Returns an empty set when the policy feature is off.
"""
from ..policyflag import policy_flag_enabled
if not policy_flag_enabled():
return set()
from ..models import BusinessPolicy
context = {'computertypeid': computertypeid, 'appids': set()}
business_policies = BusinessPolicy.query.filter_by(isactive=True).all()
resolved = {}
for businesspolicyid, config_policy in _iter_matching_configpolicies(
business_policies, context):
for action_row in config_policy.actions:
if not action_row.isactive:
continue
resolved.setdefault(action_row.action, True)
ids = set()
for name in resolved:
if isinstance(name, str) and name.startswith('monitor:'):
raw = name[len('monitor:'):]
try:
ids.add(int(raw))
except (TypeError, ValueError):
continue
return ids

2
plugins/endpoints/api/__init__.py
View File

@@ -5,5 +5,7 @@ from .routes import endpoints_bp
from . import adoption # noqa: F401
# Attaches the site agent generator packaging routes to endpoints_bp.
from . import siteagent # noqa: F401
# Attaches the monitor catalog CRUD and resolve routes to endpoints_bp.
from . import monitors # noqa: F401
__all__ = ['endpoints_bp']

204
plugins/endpoints/api/monitors.py Normal file
View File

@@ -0,0 +1,204 @@
"""Endpoint monitor definitions API (the monitor catalog CRUD + resolver).
A monitor definition is the reusable "what" an operator builds on the Settings
page: a named process or service with a bespoke display label and optional
launch/collector paths. Attaching a monitor to a PC type is a site-policy
concern (see plugins/computers policies); this surface owns the definitions
themselves plus the resolve-by-pctype read the agent uses at runtime.
Shipped rows (source='shipped') are product-owned: readable by everyone with
the view permission but not editable or deletable through this API.
The resolve-by-pctype read returns the monitor set a deployed agent should watch
for its PC type: the shipped defaults for 'Shopfloor PC' unioned with whatever
active site policies attach to that type. Shipped monitors are hardcoded to
'Shopfloor PC' as a proof of concept until they are authored through the policy
route like any user-built monitor.
"""
from flask import request
from flask_jwt_extended import current_user
from shopdb.api import (
db, success_response, error_response, ErrorCodes, require_permission,
require_role,
)
from ..models import EndpointMonitor
from .routes import endpoints_bp
# Shipped monitors default to this PC type until they are authored as policies.
SHIPPED_DEFAULT_PCTYPE = 'Shopfloor PC'
_VALID_KINDS = ('process', 'service')
def _validate_monitor_payload(body, kind):
"""Return an error message for an invalid create/update body, or None."""
displayname = (body.get('displayname') or '').strip()
if not displayname:
return 'displayname is required'
if kind not in _VALID_KINDS:
return "kind must be 'process' or 'service'"
if kind == 'process' and not (body.get('processname') or '').strip():
return 'processname is required for a process monitor'
if kind == 'service' and not (body.get('servicename') or '').strip():
return 'servicename is required for a service monitor'
return None
@endpoints_bp.route('/monitors', methods=['GET'])
@require_permission('endpoints.view')
def list_monitors():
"""List monitor definitions (shipped and local)."""
monitors = EndpointMonitor.query.order_by(
EndpointMonitor.source.desc(),
EndpointMonitor.displayname.asc()).all()
return success_response([monitor.to_dict() for monitor in monitors])
@endpoints_bp.route('/monitors', methods=['POST'])
@require_role('admin')
def create_monitor():
"""Create a user-defined (local) monitor definition."""
body = request.get_json(silent=True) or {}
kind = (body.get('kind') or '').strip().lower()
problem = _validate_monitor_payload(body, kind)
if problem:
return error_response(ErrorCodes.VALIDATION_ERROR, problem, http_code=400)
monitor = EndpointMonitor(
kind=kind,
displayname=(body.get('displayname') or '').strip(),
processname=(body.get('processname') or '').strip() or None,
servicename=(body.get('servicename') or '').strip() or None,
launchpath=(body.get('launchpath') or '').strip() or None,
datacollectorpath=(body.get('datacollectorpath') or '').strip() or None,
logcollectorpath=(body.get('logcollectorpath') or '').strip() or None,
source='local',
isactive=bool(body.get('isactive', True)),
)
db.session.add(monitor)
db.session.commit()
return success_response(monitor.to_dict(), message='Monitor created',
http_code=201)
@endpoints_bp.route('/monitors/<int:monitor_id>', methods=['PUT'])
@require_role('admin')
def update_monitor(monitor_id):
"""Update a local monitor definition. Shipped rows are read-only."""
monitor = db.session.get(EndpointMonitor, monitor_id)
if monitor is None:
return error_response(ErrorCodes.NOT_FOUND, 'Monitor not found',
http_code=404)
if monitor.source != 'local':
return error_response(ErrorCodes.FORBIDDEN,
'Shipped monitors cannot be edited', http_code=403)
body = request.get_json(silent=True) or {}
kind = (body.get('kind') or monitor.kind or '').strip().lower()
problem = _validate_monitor_payload(
{
'displayname': body.get('displayname', monitor.displayname),
'processname': body.get('processname', monitor.processname),
'servicename': body.get('servicename', monitor.servicename),
},
kind,
)
if problem:
return error_response(ErrorCodes.VALIDATION_ERROR, problem, http_code=400)
monitor.kind = kind
monitor.displayname = (body.get('displayname', monitor.displayname) or '').strip()
monitor.processname = (body.get('processname', monitor.processname) or '').strip() or None
monitor.servicename = (body.get('servicename', monitor.servicename) or '').strip() or None
monitor.launchpath = (body.get('launchpath', monitor.launchpath) or '').strip() or None
monitor.datacollectorpath = (body.get('datacollectorpath', monitor.datacollectorpath) or '').strip() or None
monitor.logcollectorpath = (body.get('logcollectorpath', monitor.logcollectorpath) or '').strip() or None
if 'isactive' in body:
monitor.isactive = bool(body.get('isactive'))
db.session.commit()
return success_response(monitor.to_dict(), message='Monitor updated')
@endpoints_bp.route('/monitors/<int:monitor_id>', methods=['DELETE'])
@require_role('admin')
def delete_monitor(monitor_id):
"""Delete a local monitor definition. Shipped rows are protected."""
monitor = db.session.get(EndpointMonitor, monitor_id)
if monitor is None:
return error_response(ErrorCodes.NOT_FOUND, 'Monitor not found',
http_code=404)
if monitor.source != 'local':
return error_response(ErrorCodes.FORBIDDEN,
'Shipped monitors cannot be deleted', http_code=403)
db.session.delete(monitor)
db.session.commit()
return success_response(None, message='Monitor deleted')
def _resolve_pctype_monitors(pctype):
"""Monitor definitions a PC type should watch: shipped defaults + policy.
Shipped monitors are attached to SHIPPED_DEFAULT_PCTYPE as a code rule;
policy-attached monitors come from the computers policy resolver by PC type.
Returns a list of EndpointMonitor rows (active only), de-duped by id.
"""
selected = {}
if pctype == SHIPPED_DEFAULT_PCTYPE:
for monitor in EndpointMonitor.query.filter_by(
source='shipped', isactive=True).all():
selected[monitor.endpointmonitorid] = monitor
computertypeid = _computertypeid_for(pctype)
if computertypeid is not None:
policy_ids = _policy_monitor_ids(computertypeid)
if policy_ids:
for monitor in EndpointMonitor.query.filter(
EndpointMonitor.endpointmonitorid.in_(policy_ids),
EndpointMonitor.isactive.is_(True)).all():
selected[monitor.endpointmonitorid] = monitor
return list(selected.values())
def _computertypeid_for(pctype):
"""Resolve a PC type name to its computertypeid, or None."""
try:
from plugins.computers.models import ComputerType
except Exception:
return None
row = ComputerType.query.filter_by(computertype=pctype).first()
return row.computertypeid if row else None
def _policy_monitor_ids(computertypeid):
"""Monitor ids attached to a PC type by effective policy, or empty set."""
try:
from plugins.computers.services.policyresolver import monitor_ids_for_pctype
except Exception:
return set()
return monitor_ids_for_pctype(computertypeid)
@endpoints_bp.route('/monitors/resolve', methods=['GET'])
@require_permission('endpoints.view')
def resolve_monitors():
"""Resolve the monitor set for a PC type (agent runtime fetch).
Query param pctype names the computer type. Returns the definitions the
agent should watch: shipped defaults for 'Shopfloor PC' plus any monitors
active site policies attach to that type.
"""
pctype = (request.args.get('pctype') or '').strip()
if not pctype:
return error_response(ErrorCodes.VALIDATION_ERROR,
'pctype is required', http_code=400)
monitors = _resolve_pctype_monitors(pctype)
return success_response({
'pctype': pctype,
'monitors': [monitor.to_dict() for monitor in monitors],
})

444
plugins/endpoints/api/routes.py
View File

@@ -25,7 +25,10 @@ from shopdb.api import (
Application, AppVersion, Script,
)
from ..models import EndpointState, EndpointEvent, EndpointCommand, EndpointApplication
from ..models import (
EndpointState, EndpointEvent, EndpointCommand, EndpointApplication,
OperatorReport,
)
from ..commands import COMMAND_ALLOWLIST, is_allowed_action, ALLOWED_ACTIONS_FOR, row_actions_for, requires_target, action_targetkind, elevated_permission_for
from ..eventcatalog import CRITICAL_EVENT_IDS, event_severity, event_title
@@ -36,11 +39,12 @@ def _naive_utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
# A watched service or process reports a STATE STRING, not a boolean, and the
# vocabulary varies by source. Both the services panel and the fleet health
# rollup have to agree on what counts as running, so the rule lives here once -
# reading it wrong makes every service look stopped.
_RUNNING_STATES = ('running', 'started', 'up', 'ok', 'active')
# What counts as RUNNING is defined once, by the reconciler, and imported here.
# The panel and the reconciler read the same agent snapshot, so a disagreement
# would paint a service green and flag it as drift on the same page. Importing
# api -> services is the direction the plugin layout already runs in; the
# reconciler's own import of this module is deferred, so there is no cycle.
from ..services.reconciler import RUNNING_STATES as _RUNNING_STATES
def _service_running(entry):
@@ -87,7 +91,8 @@ def _format_uptime(seconds):
# Endpoint state
# =============================================================================
@endpoints_bp.route('', methods=['GET'])
@endpoints_bp.route('', methods=['GET'], strict_slashes=False)
@endpoints_bp.route('/', methods=['GET'], strict_slashes=False)
@require_permission('endpoints.fleet')
def list_endpoints():
"""List reported endpoint states.
@@ -396,6 +401,24 @@ def list_endpoint_events(asset_id: int):
# Asset panels (ADR-010) - shaped for the generic PluginAssetPanels renderer
# =============================================================================
def _drift_for_state(state):
"""Parse the cached warn-only reconciler drift list off an EndpointState.
driftjson is written on every check-in by the endpoints ingest: the monitors
site policy expects running that the latest snapshot did not report running.
Null means never computed; an empty list means computed with no drift.
Tolerant of a malformed blob so a bad cache never breaks the panel.
"""
raw = getattr(state, 'driftjson', None) if state else None
if not raw:
return []
try:
parsed = json.loads(raw)
except (TypeError, ValueError):
return []
return parsed if isinstance(parsed, list) else []
@endpoints_bp.route('/asset/<int:asset_id>/status', methods=['GET'])
@require_permission('endpoints.view')
def panel_status(asset_id: int):
@@ -425,6 +448,19 @@ def panel_status(asset_id: int):
state.lastreportedat.isoformat() + 'Z' if state.lastreportedat else None),
]
fields = [f for f in fields if f['value'] not in (None, '')]
# Warn-only reconciler summary: monitors site policy expects to be running
# that the last check-in did not report as running. Surfaced as a status
# field so an operator sees the drift on the panel; nothing is auto-started.
drift = _drift_for_state(state)
if drift:
names = ', '.join(entry.get('displayname') or entry.get('name')
for entry in drift)
fields.append(field(
'Needs attention',
'{} monitor(s) expected running but not reported: {}'.format(
len(drift), names)))
return success_response({'fields': fields})
@@ -446,28 +482,100 @@ def panel_telemetry_meta(asset_id: int):
return success_response({'lastreportedat': stamp})
def _pctype_for_asset(asset_id):
"""Resolve an asset's PC type name, or None.
Endpoints stays optional-on-computers: a missing plugin, a non-computer
asset, or an unset type all resolve to None so the caller falls back to
reported-only behaviour.
"""
try:
from plugins.computers.models import Computer, ComputerType
except Exception:
return None
row = (db.session.query(ComputerType.computertype)
.join(Computer,
Computer.computertypeid == ComputerType.computertypeid)
.filter(Computer.assetid == asset_id)
.first())
return row[0] if row else None
def _resolved_monitors_for_asset(asset_id):
"""Monitor definitions attached to this asset's PC type by policy/shipped.
Reuses the endpoints monitor resolver (shipped defaults plus policy-attached
monitors). Returns a list of EndpointMonitor rows, or an empty list when the
PC type is unknown or nothing resolves.
"""
pctype = _pctype_for_asset(asset_id)
if not pctype:
return []
try:
from .monitors import _resolve_pctype_monitors
return _resolve_pctype_monitors(pctype) or []
except Exception:
return []
def _row_action_dict(app, kind, name, running):
"""Build the inline Start/Stop action map for a services/processes row."""
targets = row_actions_for(app, kind, name)
if not targets:
return None
return {
'running': running,
'start': targets['start'],
'stop': targets['stop'],
'status': targets['status'],
'target': targets['target'],
'targetkind': targets['targetkind'],
}
@endpoints_bp.route('/asset/<int:asset_id>/services', methods=['GET'])
@require_permission('endpoints.view')
def panel_services(asset_id: int):
"""List panel: monitored services/processes with a run-state badge per row.
Reshapes the latest servicesjson snapshot into rows the list renderer maps
to name + status badge. Tolerant of a missing or unexpected snapshot shape:
an unknown state renders as an Unknown badge rather than failing.
to name + status badge, then MERGES the monitors that site policy (and the
shipped defaults) attach to this asset's PC type. A policy-attached monitor
the agent has not yet reported surfaces as an Unknown row with a Start
control, so an operator sees the intent immediately rather than waiting for
the next agent snapshot. Tolerant of a missing snapshot or plugin: an
unknown state renders as an Unknown badge rather than failing.
"""
state = EndpointState.query.filter_by(assetid=asset_id).first()
if not state or not state.servicesjson:
return success_response([])
maycommand = caller_holds('endpoints.command')
try:
services = json.loads(state.servicesjson)
except (TypeError, ValueError):
return success_response([])
services = []
if state and state.servicesjson:
try:
services = json.loads(state.servicesjson) or []
except (TypeError, ValueError):
services = []
rows = []
for entry in (services or []):
# Reported identities, keyed (kind, lowercased name), so a policy monitor
# the agent already reports is not duplicated as a synthetic row.
reported = set()
# Warn-only reconciler drift, keyed (kind, lowercased name), so a row that is
# expected running but not reported running can be flagged for the operator.
# kind may be None on an older snapshot, so match on name too.
drift_keys = set()
for entry in _drift_for_state(state):
name = (entry.get('name') or '').strip().lower()
if not name:
continue
drift_keys.add(((entry.get('kind') or '').lower() or None, name))
drift_keys.add((None, name))
def _is_drift(kind, name):
key = (name or '').strip().lower()
return (kind, key) in drift_keys or (None, key) in drift_keys
for entry in services:
if isinstance(entry, dict):
name = entry.get('name') or entry.get('service') or 'Unknown'
raw = (entry.get('state') or entry.get('status') or '').strip().lower()
@@ -476,9 +584,10 @@ def panel_services(asset_id: int):
else:
name, raw, app, kind = str(entry), '', None, None
running = _service_running(entry)
normkind = (kind or '').lower() or None
row = {
'name': name,
'kind': (kind or '').lower() or None,
'kind': normkind,
'statelabel': 'Running' if running else (raw.title() or 'Unknown'),
# Amber for paused and the four Windows transitional states. A
# service stuck in Start Pending is failing to start on a retry
@@ -493,17 +602,42 @@ def panel_services(asset_id: int):
# emission never consulted the caller, so a read-only viewer was shown
# controls on a shop-floor PC that 403 on click - wrong signal, and it
# gets worse the moment guests can read this panel.
targets = row_actions_for(app, kind, name) if maycommand else None
if targets:
row['action'] = {
'running': running,
'start': targets['start'],
'stop': targets['stop'],
'status': targets['status'],
'target': targets['target'],
'targetkind': targets['targetkind'],
}
action = _row_action_dict(app, kind, name, running) if maycommand else None
if action:
row['action'] = action
if not running and _is_drift(normkind, name):
row['drift'] = True
rows.append(row)
reported.add((normkind, name.strip().lower()))
# Merge policy/shipped monitors this PC type should watch. A monitor already
# reported by the agent stays as its live row; one not yet seen is appended
# as an Unknown row so the operator sees the intended coverage and can Start
# it. The monitor's displayname is the operator label (e.g. 'msedge.exe'
# -> 'Edge'); its process/service name is the wire target for actions.
for monitor in _resolved_monitors_for_asset(asset_id):
kind = (monitor.kind or '').lower() or None
osname = (monitor.processname if kind == 'process'
else monitor.servicename)
osname = (osname or '').strip()
if not osname:
continue
if (kind, osname.lower()) in reported:
continue
row = {
'name': monitor.displayname or osname,
'kind': kind,
'statelabel': 'Unknown',
'statecolor': 'gray',
}
action = _row_action_dict(None, kind, osname, False) if maycommand else None
if action:
row['action'] = action
if _is_drift(kind, osname):
row['drift'] = True
rows.append(row)
reported.add((kind, osname.lower()))
return success_response(rows)
@@ -1335,6 +1469,64 @@ def enqueue_endpoint_command(asset_id: int):
message='Command queued', http_code=201)
@endpoints_bp.route('/monitors/resolved', methods=['GET'])
def resolved_monitors_for_agent():
"""Agent watch list: the monitors this endpoint should report on.
Collector-token auth (not JWT), same as the command channel. The agent
calls this each telemetry cycle with its hostname and receives the process
and service monitors that site policy (and the shipped defaults) attach to
this endpoint's PC type, so a policy-attached monitor is enumerated and
reported without editing appsettings.json on the box. The shape mirrors the
agent's MonitorConfig (processes[]/services[] of {name, app}) so the agent
can bind it directly: name is the wire identity (process image or service
key), app is the operator label. An unknown hostname or a PC type with no
monitors returns empty lists, which the agent merges as "watch nothing extra".
"""
error = _collector_auth_or_error()
if error is not None:
return error
hostname = (request.args.get('hostname') or '').strip()
if not hostname:
return error_response(ErrorCodes.VALIDATION_ERROR,
'hostname is required')
asset_id = _resolve_asset_for_hostname(hostname)
if asset_id is None:
return error_response(ErrorCodes.NOT_FOUND,
f'Unknown hostname: {hostname!r}', http_code=404)
processes = []
services = []
for monitor in _resolved_monitors_for_asset(asset_id):
kind = (monitor.kind or '').lower()
if kind == 'process':
wirename = (monitor.processname or '').strip()
bucket = processes
elif kind == 'service':
wirename = (monitor.servicename or '').strip()
bucket = services
else:
continue
if not wirename:
continue
entry = {
'name': wirename,
'app': (monitor.displayname or wirename),
}
# Only process monitors carry a launch path, and only when set. The
# agent never trusts it blindly: it validates the path against an
# approved-root allowlist before it becomes launchable (fail closed).
if kind == 'process':
launchpath = (monitor.launchpath or '').strip()
if launchpath:
entry['launchpath'] = launchpath
bucket.append(entry)
return success_response({'processes': processes, 'services': services})
@endpoints_bp.route('/commands/next', methods=['GET'])
def claim_next_command():
"""Agent long-poll: claim the next pending command for a hostname.
@@ -1578,3 +1770,199 @@ def download_script_body(script_id: int):
return send_upload(script_dir, script.filename, as_attachment=True)
# Operator issue reports (tray screenshot + free-text notes)
# ---------------------------------------------------------------------------
# Retention groundwork: how many days an operator report (and its screenshot
# file) is kept before the purge sweep removes it. Read from the Setting store
# so Agent Configuration can expose a toggle later without a code change; the
# default is 7 days. A value of 0 or less means "keep forever" (purge disabled).
OPERATOR_REPORT_RETENTION_SETTING = 'endpoints.operatorreport.retentiondays'
OPERATOR_REPORT_RETENTION_DEFAULT_DAYS = 7
# Screenshots are small PNGs from a virtual-screen grab. Cap the upload so a
# malformed or hostile agent cannot fill the disk with one request.
OPERATOR_REPORT_MAX_IMAGE_BYTES = 10 * 1024 * 1024
# The multipart envelope carries the other form fields and the part boundaries
# too, so content_length is legitimately larger than the image. Allow for that
# rather than rejecting a screenshot that is just under the cap.
_OPERATOR_REPORT_FORM_SLACK = 64 * 1024
# PNG signature. The route stores and serves the file as image/png, so this is
# the one thing it accepts.
_PNG_MAGIC = b'\x89PNG\r\n\x1a\n'
def _operator_report_dir():
return os.path.join(current_app.instance_path, 'operatorreports')
def _operator_report_retention_days():
"""Configured retention window in whole days (default 7, <=0 disables)."""
raw = Setting.get(OPERATOR_REPORT_RETENTION_SETTING)
if raw is None:
return OPERATOR_REPORT_RETENTION_DEFAULT_DAYS
try:
return int(raw)
except (TypeError, ValueError):
return OPERATOR_REPORT_RETENTION_DEFAULT_DAYS
def _purge_expired_operator_reports():
"""Delete operator reports older than the retention window, files and all.
Best-effort and idempotent: called opportunistically on upload so the feed
stays bounded without a scheduler. A retention of <=0 disables the sweep.
"""
days = _operator_report_retention_days()
if days <= 0:
return
cutoff = _naive_utcnow() - timedelta(days=days)
stale = OperatorReport.query.filter(OperatorReport.createdat < cutoff).all()
if not stale:
return
report_dir = _operator_report_dir()
for report in stale:
if report.imagefilename:
path = os.path.join(report_dir, report.imagefilename)
try:
os.remove(path)
except OSError:
pass
db.session.delete(report)
db.session.commit()
@endpoints_bp.route('/operatorreports', methods=['POST'])
def upload_operator_report():
"""Agent uploads one operator issue report (collector-token auth).
The tray captures the operator's free-text note and an optional screenshot,
then hands them to the service, which posts here as multipart/form-data on
the same endpoints collector token used for telemetry. The reported hostname
resolves to an asset; an unresolved hostname is a validation error. The
screenshot bytes are written to the operator-reports uploads dir and the row
holds only metadata plus the stored filename. One retention sweep runs
opportunistically so the feed stays bounded.
"""
import uuid
error = _collector_auth_or_error()
if error is not None:
return error
hostname = (request.form.get('hostname') or '').strip()
if not hostname:
return error_response(ErrorCodes.VALIDATION_ERROR,
'hostname is required')
asset_id = _resolve_asset_for_hostname(hostname)
if asset_id is None:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'could not resolve hostname to a computer: {!r}'.format(hostname))
reason = (request.form.get('reason') or 'Other (free text)').strip()[:128]
details = (request.form.get('details') or '').strip() or None
imagefilename = None
imagemimetype = None
imagebytesize = None
# Refuse an oversized body BEFORE reading it. The cap used to be checked
# after upload.read(), which pulls the whole thing into memory first - the
# check could not prevent what it was there to prevent. content_length is
# caller-supplied so it is a cheap early out, not the enforcement; the read
# below is still bounded.
declared = request.content_length or 0
if declared > OPERATOR_REPORT_MAX_IMAGE_BYTES + _OPERATOR_REPORT_FORM_SLACK:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'screenshot exceeds the {} byte limit'.format(
OPERATOR_REPORT_MAX_IMAGE_BYTES))
upload = request.files.get('screenshot')
if upload is not None and upload.filename:
# Read one byte past the cap: enough to know it was exceeded, without
# taking the whole of a body that lied about its length.
data = upload.read(OPERATOR_REPORT_MAX_IMAGE_BYTES + 1)
if len(data) > OPERATOR_REPORT_MAX_IMAGE_BYTES:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'screenshot exceeds the {} byte limit'.format(
OPERATOR_REPORT_MAX_IMAGE_BYTES))
if data and not data.startswith(_PNG_MAGIC):
# The stored file is served back as image/png. Storing whatever
# arrived under that name meant the extension and the bytes could
# disagree; the upload headers make the document case inert, but
# nothing is gained by keeping a file we already know is not what it
# claims.
return error_response(ErrorCodes.VALIDATION_ERROR,
'screenshot must be a PNG')
if data:
imagefilename = 'report-{}.png'.format(uuid.uuid4().hex)
imagemimetype = 'image/png'
imagebytesize = len(data)
report_dir = _operator_report_dir()
os.makedirs(report_dir, exist_ok=True)
with open(os.path.join(report_dir, imagefilename), 'wb') as handle:
handle.write(data)
report = OperatorReport(
assetid=asset_id,
reason=reason,
details=details,
imagefilename=imagefilename,
imagemimetype=imagemimetype,
imagebytesize=imagebytesize,
reportedhostname=hostname,
)
db.session.add(report)
db.session.commit()
# Keep the feed bounded without a scheduler. Best-effort: a purge failure
# must not fail the upload the operator already completed.
try:
_purge_expired_operator_reports()
except Exception:
db.session.rollback()
return success_response(report.to_dict(),
message='Operator report received', http_code=201)
@endpoints_bp.route('/panel/<int:asset_id>/operatorreports', methods=['GET'])
@require_permission('endpoints.view')
def panel_operator_reports(asset_id: int):
"""Operator Reports panel: this PC's recent operator issue reports.
Newest first. Each row carries a hasimage flag so the viewer knows whether
to request the screenshot from the serve route below. Runs a retention
sweep opportunistically so a viewed PC does not accumulate stale rows.
"""
try:
_purge_expired_operator_reports()
except Exception:
db.session.rollback()
reports = (OperatorReport.query
.filter_by(assetid=asset_id)
.order_by(OperatorReport.createdat.desc())
.all())
return success_response({'reports': [r.to_dict() for r in reports]})
@endpoints_bp.route('/operatorreports/<int:operatorreportid>/image',
methods=['GET'])
@require_permission('endpoints.view')
def serve_operator_report_image(operatorreportid: int):
"""Serve one operator report's screenshot (user JWT + endpoints.view)."""
from shopdb.api import send_upload
report = db.session.get(OperatorReport, operatorreportid)
if report is None or not report.imagefilename:
return error_response(ErrorCodes.NOT_FOUND,
'No screenshot for this report', http_code=404)
return send_upload(_operator_report_dir(), report.imagefilename)

140
plugins/endpoints/api/siteagent.py
View File

@@ -41,6 +41,14 @@ from .routes import endpoints_bp
STAGED_AGENT_RELPATH = os.path.join('siteagent', 'shopdbagent.exe')
STAGED_TRAY_RELPATH = os.path.join('siteagent', 'shopdb-endpoint-tray.exe')
# The standalone uninstaller lives in the repo tree (not staged per-instance),
# so it ships in every bundle. Resolve it relative to this file:
# plugins/endpoints/api/siteagent.py -> repo root is three levels up.
UNINSTALLER_PATH = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(
os.path.abspath(__file__))))),
'scripts', 'uninstall-shopdbagent.ps1')
# Public TLS certificate the agent must trust when this deployment terminates
# TLS itself with a self-signed or otherwise untrusted certificate. We can only
# auto-bundle when THIS app owns the cert file (the dev topology, where gunicorn
@@ -144,6 +152,12 @@ FILES
appsettings.json Non-secret runtime config (cadence, monitors,
launch paths). No credentials.
install.ps1 Provisions the credentials and installs the service.
uninstall-shopdbagent.ps1
Standalone uninstaller. Run elevated to reverse the
install: stops and deletes the service, kills agent
and tray processes, removes the tray autostart Run
key, the Start Menu shortcut, and the credential
hive, and deletes the install directory.
server.crt Server TLS certificate (present only for self-signed
or internal-CA deployments, either auto-detected or
uploaded by the operator). install.ps1 imports it
@@ -153,10 +167,16 @@ FILES
USAGE
1. Copy this whole folder to the target endpoint.
2. Right-click install.ps1 and Run with PowerShell as administrator. It
writes the server URL and collector key to HKLM\\SOFTWARE\\GE\\ShopDB,
copies the agent into Program Files, and starts the service.
writes the server URL and collector key to HKLM\\SOFTWARE\\GE\\ShopDB,
copies the agent into Program Files, and starts the service. It also
registers the tray to autostart for every user at logon and adds a
"ShopDB Agent" Start Menu shortcut for manual relaunch.
SECURITY
UNINSTALL
Run uninstall-shopdbagent.ps1 from an elevated PowerShell to fully remove the
agent, tray autostart, Start Menu shortcut, and credentials.
SECURITY
install.ps1 carries a live collector key in its default -CollectorKey value.
Treat this package as a secret. The key is not recoverable from the ShopDB
server after it was generated; regenerate the key in the UI to revoke it.
@@ -248,7 +268,7 @@ Write-Host "ServiceName : $ServiceName"
# 0. Stop and remove any previous install so the file copy is a clean overwrite.
$existing = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
if ($existing) {{
Write-Host "[0/4] Removing existing service $ServiceName ..."
Write-Host "[0/6] Removing existing service $ServiceName ..."
if ($existing.Status -ne "Stopped") {{
try {{ Stop-Service -Name $ServiceName -Force -ErrorAction Stop }}
catch {{ Write-Warning "could not stop service: $($_.Exception.Message)" }}
@@ -264,7 +284,7 @@ foreach ($name in @("shopdbagent", $trayBase)) {{
}}
# 1. Provision registry credentials (both 64-bit and 32-bit views).
Write-Host "[1/4] Provisioning registry HKLM:\\SOFTWARE\\GE\\ShopDB (Registry64 + Registry32) ..."
Write-Host "[1/6] Provisioning registry HKLM:\\SOFTWARE\\GE\\ShopDB (Registry64 + Registry32) ..."
Set-HklmValueBothViews -SubKey "SOFTWARE\\GE\\ShopDB" -Name "BaseUrl" -Value $BaseUrl
Set-HklmValueBothViews -SubKey "SOFTWARE\\GE\\ShopDB" -Name "CollectorKey" -Value $CollectorKey
@@ -288,7 +308,7 @@ if (-not [string]::IsNullOrWhiteSpace($CertFileName)) {{
}}
# 2. Copy agent files to the install directory.
Write-Host "[2/4] Copying agent files to $InstallDir ..."
Write-Host "[2/6] Copying agent files to $InstallDir ..."
New-Item -ItemType Directory -Force -Path $InstallDir | Out-Null
Get-ChildItem -Path $sourceDir -File | ForEach-Object {{
try {{ Unblock-File -Path $_.FullName -ErrorAction Stop }} catch {{ }}
@@ -306,17 +326,118 @@ if (-not $NoTray) {{
$traySource = Join-Path $sourceDir $TrayExeName
if (Test-Path $traySource) {{
Copy-Item -Path $traySource -Destination $InstallDir -Force
$installedTray = Join-Path $InstallDir $TrayExeName
# Autostart the tray for every user at logon via the HKLM Run key (both
# 64-bit and 32-bit views, matching the credential hive). This is the
# reliable launch guarantee: a plain Start Menu shortcut does not
# autostart, and the SYSTEM service cannot cleanly spawn into the
# interactive session. The tray itself is single-instance, so a stray
# second launch is harmless.
Set-HklmValueBothViews -SubKey "SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Run" -Name "ShopDbEndpointTray" -Value ('"' + $installedTray + '"')
# Export the branded icon from the tray exe so the shortcut shows the
# ShopDB brand instead of the generic exe icon. The tray draws its icon
# at runtime (no embedded Win32 icon), so we ask it to write an .ico into
# the install dir and point the shortcut at it.
$iconPath = Join-Path $InstallDir "shopdb-agent.ico"
try {{ & $installedTray --export-icon $iconPath | Out-Null }}
catch {{ Write-Warning "could not export shortcut icon: $($_.Exception.Message)" }}
# Start Menu shortcut so operators can relaunch the tray on demand. This
# is discoverability only, not autostart. Written to the machine-wide
# (All Users) Programs folder.
try {{
$commonPrograms = [Environment]::GetFolderPath([Environment+SpecialFolder]::CommonPrograms)
$shortcutPath = Join-Path $commonPrograms "ShopDB Agent.lnk"
$shell = New-Object -ComObject WScript.Shell
$shortcut = $shell.CreateShortcut($shortcutPath)
$shortcut.TargetPath = $installedTray
$shortcut.WorkingDirectory = $InstallDir
$shortcut.Description = "ShopDB Endpoint Agent tray"
if (Test-Path $iconPath) {{ $shortcut.IconLocation = "$iconPath,0" }}
$shortcut.Save()
Write-Host " created Start Menu shortcut: $shortcutPath"
}}
catch {{ Write-Warning "could not create Start Menu shortcut: $($_.Exception.Message)" }}
# Launch the tray once now so it appears without waiting for the next
# logon. Single-instance guard makes a duplicate launch a no-op.
try {{ Start-Process -FilePath $installedTray -WorkingDirectory $InstallDir }}
catch {{ Write-Warning "could not start tray: $($_.Exception.Message)" }}
}}
}}
# 3. Install and start the service. The agent supports its own service install.
Write-Host "[3/4] Installing and starting the service ..."
Write-Host "[3/6] Installing and starting the service ..."
$installedAgent = Join-Path $InstallDir "shopdbagent.exe"
$binPath = '"' + $installedAgent + '"'
sc.exe create $ServiceName binPath= $binPath start= auto DisplayName= "ShopDB Endpoint Agent" | Out-Null
sc.exe start $ServiceName | Out-Null
Write-Host "[4/4] Done." -ForegroundColor Green
# 4. Confirm the service (and tray, when installed) actually reached a running
# state before we destroy the installer. sc.exe start returns before the service
# is up, and a failed start would otherwise leave a half-installed endpoint with
# the secret-bearing install.ps1 already deleted. Poll with a short backoff and
# retry the start a few times; only proceed to cleanup once healthy.
Write-Host "[4/6] Verifying agent runtime ..."
$serviceOk = $false
for ($attempt = 1; $attempt -le 5; $attempt++) {{
Start-Sleep -Seconds 3
$svc = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
if ($svc -and $svc.Status -eq "Running") {{ $serviceOk = $true; break }}
Write-Host " service not running yet (attempt $attempt/5); retrying start ..."
try {{ Start-Service -Name $ServiceName -ErrorAction Stop }} catch {{ }}
}}
if (-not $serviceOk) {{
throw "Service $ServiceName did not reach Running after 5 attempts. Leaving install.ps1 in place for diagnosis; re-run once resolved."
}}
Write-Host " service is running." -ForegroundColor Green
$trayBase2 = [System.IO.Path]::GetFileNameWithoutExtension($TrayExeName)
if (-not $NoTray -and (Test-Path (Join-Path $InstallDir $TrayExeName))) {{
$trayOk = $false
for ($attempt = 1; $attempt -le 5; $attempt++) {{
if (Get-Process -Name $trayBase2 -ErrorAction SilentlyContinue) {{ $trayOk = $true; break }}
Write-Host " tray not running yet (attempt $attempt/5); relaunching ..."
try {{ Start-Process -FilePath (Join-Path $InstallDir $TrayExeName) -WorkingDirectory $InstallDir }} catch {{ }}
Start-Sleep -Seconds 2
}}
if ($trayOk) {{ Write-Host " tray is running." -ForegroundColor Green }}
else {{ Write-Warning "tray did not confirm running; it will retry at next logon via the Run key." }}
}}
# 5. Preserve the install artifacts (minus the secret-bearing install.ps1) under
# ProgramData so the uninstaller and appsettings/cert are available later, then
# scrub install.ps1 from wherever it was run. install.ps1 carries the live
# collector key in its default parameter, so it must not linger on the endpoint.
Write-Host "[5/6] Archiving install artifacts and removing install.ps1 ..."
$dataDir = Join-Path $env:ProgramData "ShopDB\\EndpointAgent"
New-Item -ItemType Directory -Force -Path $dataDir | Out-Null
foreach ($name in @("uninstall-shopdbagent.ps1", "appsettings.json", "README.txt", $CertFileName)) {{
if ([string]::IsNullOrWhiteSpace($name)) {{ continue }}
$src = Join-Path $sourceDir $name
if (Test-Path $src) {{
try {{ Copy-Item -Path $src -Destination $dataDir -Force }}
catch {{ Write-Warning "could not archive $name : $($_.Exception.Message)" }}
}}
}}
$uninstaller = Join-Path $dataDir "uninstall-shopdbagent.ps1"
$installSelf = Join-Path $sourceDir "install.ps1"
if (Test-Path $installSelf) {{
try {{ Remove-Item -Path $installSelf -Force -ErrorAction Stop; Write-Host " removed install.ps1 (carried the collector key)." }}
catch {{ Write-Warning "could not remove install.ps1 - delete it manually, it contains the collector key: $($_.Exception.Message)" }}
}}
Write-Host "[6/6] Done." -ForegroundColor Green
if (Test-Path $uninstaller) {{
Write-Host "Uninstaller: $uninstaller" -ForegroundColor Cyan
Write-Host " To remove the agent, run that script from an elevated PowerShell."
}}
else {{
Write-Warning "uninstaller was not bundled; fetch uninstall-shopdbagent.ps1 from the ShopDB repo scripts/ folder if needed."
}}
""".format(
baseurl=baseurl_literal,
collectorkey=collectorkey_literal,
@@ -421,6 +542,9 @@ def package_siteagent():
cadenceseconds, certfilename))
archive.writestr('README.txt', _README)
if os.path.isfile(UNINSTALLER_PATH):
archive.write(UNINSTALLER_PATH, 'uninstall-shopdbagent.ps1')
agent = _staged_path(STAGED_AGENT_RELPATH)
if os.path.isfile(agent):
archive.write(agent, 'shopdbagent.exe')

68
plugins/endpoints/commands.py
View File

@@ -185,6 +185,36 @@ def requires_target(action):
return action_targetkind(action) is not None
# Monitor attachment as a policy action. A monitor definition is bound to a PC
# type by carrying a ConfigurationPolicyAction whose name is 'monitor:<id>'.
# Distinct names let one policy attach several monitors and fold naturally
# (first-match-wins per name), and they are validated in the policy layer but
# never enqueued as commands because ALLOWED_ACTIONS_FOR intersects with
# COMMAND_ALLOWLIST, which carries no monitor: entries.
MONITOR_ACTION_PREFIX = 'monitor:'
def monitor_action_name(endpointmonitorid):
"""Build the policy action name that attaches a monitor definition."""
return f'{MONITOR_ACTION_PREFIX}{endpointmonitorid}'
def is_monitor_action(action):
"""True when action names a monitor attachment (monitor:<id>)."""
return isinstance(action, str) and action.startswith(MONITOR_ACTION_PREFIX)
def monitor_id_from_action(action):
"""The monitor id an action attaches, or None when it is not a monitor action."""
if not is_monitor_action(action):
return None
raw = action[len(MONITOR_ACTION_PREFIX):]
try:
return int(raw)
except (TypeError, ValueError):
return None
def ALLOWED_ACTIONS_FOR(pctype=None, assetid=None):
"""The action names an endpoint may run.
@@ -211,6 +241,44 @@ def ALLOWED_ACTIONS_FOR(pctype=None, assetid=None):
# behavior). This is intentionally permissive; deny-by-default would be
# a separate, explicit setting.
return base
# Attaching a monitor expresses intent to watch a process/service AND to
# control it: the panel renders that row's Start/Stop/Status buttons, so the
# gate must permit the matching generic primitives or every click 409s
# "not permitted by policy". Expand each monitor:<id> to its kind's control
# actions before intersecting.
allowed = allowed | _monitor_control_actions(allowed)
# Never widen past the code-level allowlist: policy can only gate within it.
return base & allowed
def _monitor_control_actions(action_names):
"""Control primitives granted by the monitor: attachments in a policy.
A process monitor grants process_start/stop/status; a service monitor grants
service_start/stop/status. The monitor's kind is read from the catalog, so a
definition's kind is the single source of truth for what its row can do.
"""
ids = set()
for name in action_names:
if isinstance(name, str) and name.startswith(MONITOR_ACTION_PREFIX):
monitor_id = monitor_id_from_action(name)
if monitor_id is not None:
ids.add(monitor_id)
if not ids:
return set()
try:
from .models import EndpointMonitor
except Exception:
return set()
granted = set()
for monitor in EndpointMonitor.query.filter(
EndpointMonitor.endpointmonitorid.in_(ids)).all():
kind = (monitor.kind or '').lower()
if kind == 'process':
granted |= {'process_start', 'process_stop', 'process_status'}
elif kind == 'service':
granted |= {'service_start', 'service_stop', 'service_status'}
return granted

6
plugins/endpoints/frontend/routes.js
View File

@@ -18,6 +18,12 @@ export default [
component: () => import('./views/AgentKeySettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true, plugin: 'endpoints' }
},
{
path: 'settings/endpoints/monitors',
name: 'endpoints-monitors-settings',
component: () => import('./views/MonitorsSettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true, plugin: 'endpoints' }
},
// TEMPORARY / DEV ONLY. A manual bench to exercise the update_app and
// run_script delivery path end to end while Cameron builds the real UI.
// Gated on endpoints.deploy inside the view; remove once the production

21
plugins/endpoints/frontend/views/AgentKeySettings.vue
View File

@@ -307,7 +307,7 @@
</section>
<!-- ====================================================================
Agent configuration (placeholder)
Agent configuration
================================================================= -->
<section class="settings-section">
<h3>Agent configuration</h3>
@@ -319,11 +319,16 @@
stamped into the build above so a deployed agent needs no local edits.
</p>
<p class="field-hint">
<strong>Coming soon.</strong> These controls arrive with the C# agent
runtime. Until then the agent uses its built-in defaults.
Monitored processes and services are defined as reusable monitors and
attached to PC types through site policies, so a deployed agent fetches
the right set for its type.
</p>
<button class="btn btn-secondary" type="button" disabled>
Edit configuration (coming soon)
<button
class="btn btn-secondary"
type="button"
@click="editConfiguration"
>
Edit configuration
</button>
</div>
</section>
@@ -332,8 +337,10 @@
<script setup>
import { ref, onMounted } from 'vue'
import { useRouter } from 'vue-router'
import api from '@/api'
const router = useRouter()
const loading = ref(true)
const busy = ref(false)
const key = ref(null)
@@ -540,6 +547,10 @@ async function buildSiteAgent() {
agentbusy.value = false
}
}
function editConfiguration() {
router.push({ name: 'endpoints-monitors-settings' })
}
</script>
<style scoped>

232
plugins/endpoints/frontend/views/MonitorsSettings.vue Normal file
View File

@@ -0,0 +1,232 @@
<template>
<div>
<div class="page-header">
<h2>Monitor Definitions</h2>
<button class="btn btn-primary" @click="openModal()">+ Add Monitor</button>
</div>
<p class="setting-description">
A monitor definition is a reusable process or service the agent can watch,
with a bespoke display label separate from the raw OS name. Shipped monitors
are product-owned and read-only. Attach monitors to PC types through site
policies so a deployed agent fetches the right set for its type.
</p>
<div class="card">
<div v-if="loading" class="loading">Loading...</div>
<template v-else>
<div class="table-container">
<table>
<thead>
<tr>
<th>Display name</th>
<th>Kind</th>
<th>OS identity</th>
<th>Launch path</th>
<th>Active</th>
<th>Source</th>
<th></th>
</tr>
</thead>
<tbody>
<tr v-for="m in monitors" :key="m.endpointmonitorid">
<td>{{ m.displayname }}</td>
<td>{{ m.kind }}</td>
<td class="cell-truncate" :title="osidentity(m)">{{ osidentity(m) || '-' }}</td>
<td class="cell-truncate" :title="m.launchpath">{{ m.launchpath || '-' }}</td>
<td>{{ m.isactive ? 'Yes' : 'No' }}</td>
<td>
<span class="badge" :class="m.source === 'local' ? 'badge-secondary' : 'badge-warning'">
{{ m.source }}
</span>
</td>
<td class="actions">
<button class="btn btn-secondary btn-sm" :disabled="m.source !== 'local'" @click="openModal(m)">Edit</button>
<button class="btn btn-danger btn-sm" :disabled="m.source !== 'local'" @click="remove(m)">Delete</button>
</td>
</tr>
<tr v-if="monitors.length === 0">
<td colspan="7" style="text-align:center; color: var(--text-light);">No monitor definitions</td>
</tr>
</tbody>
</table>
</div>
</template>
</div>
<div v-if="showModal" class="modal-overlay">
<div class="modal modal-lg">
<div class="modal-header">
<h3>{{ editing ? 'Edit Monitor' : 'Add Monitor' }}</h3>
</div>
<form @submit.prevent="save">
<div class="modal-body">
<div class="form-group">
<label>Display name *</label>
<input v-model="form.displayname" type="text" class="form-control" required />
</div>
<div class="form-group">
<label>Kind *</label>
<select v-model="form.kind" class="form-control">
<option value="process">process</option>
<option value="service">service</option>
</select>
</div>
<div v-if="form.kind === 'process'" class="form-group">
<label>Process name * <span class="hint">(executable the agent matches, e.g. agent.exe)</span></label>
<input v-model="form.processname" type="text" class="form-control" />
</div>
<div v-else class="form-group">
<label>Service name * <span class="hint">(Windows service name)</span></label>
<input v-model="form.servicename" type="text" class="form-control" />
</div>
<div class="form-group">
<label>Launch path <span class="hint">(optional; starts the app)</span></label>
<input v-model="form.launchpath" type="text" class="form-control" />
</div>
<div class="form-group">
<label>Data collector path <span class="hint">(optional)</span></label>
<input v-model="form.datacollectorpath" type="text" class="form-control" />
</div>
<div class="form-group">
<label>Log collector path <span class="hint">(optional)</span></label>
<input v-model="form.logcollectorpath" type="text" class="form-control" />
</div>
<div class="form-group">
<label>
<input v-model="form.isactive" type="checkbox" />
Active
</label>
</div>
<div v-if="error" class="error-message">{{ error }}</div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-secondary" @click="closeModal">Cancel</button>
<button type="submit" class="btn btn-primary" :disabled="saving">{{ saving ? 'Saving...' : 'Save' }}</button>
</div>
</form>
</div>
</div>
</div>
</template>
<script setup>
import { ref, onMounted } from 'vue'
import { endpointsApi } from '@/api'
import { useToast } from '@/composables/toast'
import { apiError } from '@/utils/apiError'
const toast = useToast()
const monitors = ref([])
const loading = ref(true)
const showModal = ref(false)
const editing = ref(null)
const saving = ref(false)
const error = ref('')
const form = ref(emptyForm())
function emptyForm() {
return {
kind: 'process',
displayname: '',
processname: '',
servicename: '',
launchpath: '',
datacollectorpath: '',
logcollectorpath: '',
isactive: true
}
}
function osidentity(m) {
return m.kind === 'service' ? m.servicename : m.processname
}
onMounted(() => loadData())
async function loadData() {
loading.value = true
try {
const response = await endpointsApi.monitors.list()
monitors.value = response.data.data || []
} catch (err) {
console.error('Error loading monitor definitions:', err)
} finally {
loading.value = false
}
}
function openModal(item = null) {
editing.value = item
error.value = ''
if (item) {
form.value = {
kind: item.kind || 'process',
displayname: item.displayname || '',
processname: item.processname || '',
servicename: item.servicename || '',
launchpath: item.launchpath || '',
datacollectorpath: item.datacollectorpath || '',
logcollectorpath: item.logcollectorpath || '',
isactive: item.isactive !== false
}
} else {
form.value = emptyForm()
}
showModal.value = true
}
function closeModal() { showModal.value = false; editing.value = null }
async function save() {
saving.value = true
error.value = ''
try {
const payload = {
kind: form.value.kind,
displayname: form.value.displayname,
processname: form.value.processname,
servicename: form.value.servicename,
launchpath: form.value.launchpath,
datacollectorpath: form.value.datacollectorpath,
logcollectorpath: form.value.logcollectorpath,
isactive: form.value.isactive
}
if (editing.value) {
await endpointsApi.monitors.update(editing.value.endpointmonitorid, payload)
} else {
await endpointsApi.monitors.create(payload)
}
closeModal()
loadData()
} catch (err) {
error.value = apiError(err, 'Failed to save monitor')
} finally {
saving.value = false
}
}
async function remove(m) {
if (!confirm(`Delete monitor "${m.displayname}"?`)) return
try {
await endpointsApi.monitors.remove(m.endpointmonitorid)
loadData()
} catch (err) {
toast.error(apiError(err, 'Failed to delete'))
}
}
</script>
<style scoped>
.cell-truncate {
max-width: 240px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.hint { color: var(--text-light); font-weight: normal; }
</style>

151
plugins/endpoints/migrations/versions/0006_endpoint_monitors.py Normal file
View File

@@ -0,0 +1,151 @@
"""Reusable endpoint monitor definitions (the monitor catalog).
Adds endpointmonitors: one row per process or service the agent can watch, with
a bespoke display label separate from the raw OS name and optional launch and
collector paths. A definition is the "what"; attaching it to a PC type is a
separate site-policy concern.
Seeds the four monitors we ship today, lifted from the agent's appsettings.json
so the catalog matches what a deployed agent already watches:
process dncMain.exe -> eDNC
process udc.exe -> Universal Data Collector
process agent.exe -> MTConnect
service MTConnect UDC Agent -> MTConnect
Shipped rows are product-owned (source='shipped'): the CRUD API refuses to edit
or delete them. Launch paths mirror the agent's launchpaths block; collector and
log paths are left null until site paths are supplied.
Idempotent: creation checks the live schema first so a db.create_all() site
(which already built the table) re-runs clean on strict MySQL 8. Seeding is a
separate check on row count, so a site that built the table without the shipped
rows still gets them.
"""
from alembic import op
import sqlalchemy as sa
from datetime import datetime
# revision identifiers, used by Alembic.
revision = 'endpoints0006monitors'
down_revision = 'endpoints0005cmdpin'
branch_labels = None
depends_on = None
def _has_table(table_name):
from sqlalchemy import inspect
return inspect(op.get_bind()).has_table(table_name)
def upgrade():
bind = op.get_bind()
if not _has_table('endpointmonitors'):
op.create_table(
'endpointmonitors',
sa.Column('endpointmonitorid', sa.Integer(), nullable=False),
sa.Column('kind', sa.String(length=16), nullable=False),
sa.Column('displayname', sa.String(length=128), nullable=False),
sa.Column('processname', sa.String(length=255), nullable=True),
sa.Column('servicename', sa.String(length=255), nullable=True),
sa.Column('launchpath', sa.String(length=512), nullable=True),
sa.Column('datacollectorpath', sa.String(length=512), nullable=True),
sa.Column('logcollectorpath', sa.String(length=512), nullable=True),
sa.Column('source', sa.String(length=16), nullable=False,
server_default='local'),
sa.Column('isactive', sa.Boolean(), nullable=False,
server_default=sa.true()),
sa.Column('createdat', sa.DateTime(), nullable=False),
sa.PrimaryKeyConstraint('endpointmonitorid'),
sa.UniqueConstraint('kind', 'displayname',
name='uq_endpointmonitors_kinddisplay'),
)
op.create_index('idx_endpointmonitors_kind', 'endpointmonitors', ['kind'])
op.create_index('idx_endpointmonitors_source', 'endpointmonitors', ['source'])
# Seed the shipped catalog independently of table creation so a
# db.create_all() site (which already built the table) still gets the rows.
existing = bind.execute(
sa.text('SELECT COUNT(*) FROM endpointmonitors')).scalar()
if existing:
return
monitors = sa.table(
'endpointmonitors',
sa.column('kind', sa.String),
sa.column('displayname', sa.String),
sa.column('processname', sa.String),
sa.column('servicename', sa.String),
sa.column('launchpath', sa.String),
sa.column('datacollectorpath', sa.String),
sa.column('logcollectorpath', sa.String),
sa.column('source', sa.String),
sa.column('isactive', sa.Boolean),
sa.column('createdat', sa.DateTime),
)
now = datetime.utcnow()
op.bulk_insert(monitors, [
{
'kind': 'process',
'displayname': 'eDNC',
'processname': 'dncMain.exe',
'servicename': None,
'launchpath': r'C:\Program Files (x86)\Dnc\bin\DncMain.exe',
'datacollectorpath': None,
'logcollectorpath': None,
'source': 'shipped',
'isactive': True,
'createdat': now,
},
{
'kind': 'process',
'displayname': 'Universal Data Collector',
'processname': 'udc.exe',
'servicename': None,
'launchpath': r'C:\Program Files\UDC\UDC.exe',
'datacollectorpath': None,
'logcollectorpath': None,
'source': 'shipped',
'isactive': True,
'createdat': now,
},
{
'kind': 'process',
'displayname': 'MTConnect',
'processname': 'agent.exe',
'servicename': None,
'launchpath': r'C:\ProgramData\udc\mtconnect_udc\agent\run.bat',
'datacollectorpath': None,
'logcollectorpath': None,
'source': 'shipped',
'isactive': True,
'createdat': now,
},
{
'kind': 'service',
'displayname': 'MTConnect',
'processname': None,
'servicename': 'MTConnect UDC Agent',
'launchpath': None,
'datacollectorpath': None,
'logcollectorpath': None,
'source': 'shipped',
'isactive': True,
'createdat': now,
},
])
def downgrade():
if not _has_table('endpointmonitors'):
return
op.drop_index('idx_endpointmonitors_source', table_name='endpointmonitors')
op.drop_index('idx_endpointmonitors_kind', table_name='endpointmonitors')
op.drop_table('endpointmonitors')

234
plugins/endpoints/migrations/versions/0007_shipped_monitor_policy.py Normal file
View File

@@ -0,0 +1,234 @@
"""Ship the shopfloor app monitors as a read-only Site Policy.
Wraps the four shipped endpointmonitors (eDNC, Universal Data Collector,
MTConnect process, MTConnect UDC Agent service) in a product-owned policy bundle
so the agent stops carrying them as hardcoded appsettings.json defaults and reads
its watch-list from the server instead.
Seeds two computers-plugin rows, both source='shipped' (read-only in the UI, the
CRUD API refuses to edit or delete them):
ConfigurationPolicy "Shopfloor App Monitors"
scope {"type":"true"} (every endpoint), one action per shipped monitor,
named 'monitor:<endpointmonitorid>', mode='gate', desiredstate=NULL.
Gate, not desired, on purpose. A monitor REPORTS a process/service state and
exposes Start/Stop in the Monitor UI; the state read happens on manual
refresh and on agent check-in, so attaching a monitor never generates fleet
traffic on its own and a policy edit does not push a collection. A 'desired'
running state is a separate, deliberate opt-in that keeps an app alive by
trying to START the process when a launch path is mapped - never something
the product ships switched on for every endpoint.
BusinessPolicy "RPCA Shopfloor App Monitors"
the site-level wrapper, carrying the Configuration Policy as its one item.
Monitors are looked up by identity (kind, displayname), never by a hardcoded id,
so the bundle survives id drift across installs.
Lives in the endpoints chain, after 0006, because the monitor rows it references
are seeded there. The computers policy tables are created by the core baseline
before per-plugin chains run; this migration still guards on their presence and
no-ops cleanly if they are absent, and is idempotent on the policy/business-policy
names so re-running never duplicates the bundle.
"""
from alembic import op
import sqlalchemy as sa
from datetime import datetime
# revision identifiers, used by Alembic.
revision = 'endpoints0007shippedpolicy'
down_revision = 'endpoints0006monitors'
branch_labels = None
depends_on = None
CONFIG_POLICY_NAME = 'Shopfloor App Monitors'
BUSINESS_POLICY_NAME = 'RPCA Shopfloor App Monitors'
# The shipped monitors this bundle attaches, by identity (kind, displayname).
_SHIPPED_MONITORS = (
('process', 'eDNC'),
('process', 'Universal Data Collector'),
('process', 'MTConnect'),
('service', 'MTConnect'),
)
def _tables(bind):
from sqlalchemy import inspect
return set(inspect(bind).get_table_names())
def upgrade():
bind = op.get_bind()
tables = _tables(bind)
# The bundle lives in the computers policy tables and references the
# endpoints monitor catalog. If either side is missing (a partial site),
# do nothing rather than fail - a later run reconciles once both exist.
required = {'endpointmonitors', 'configurationpolicies',
'configurationpolicyactions', 'businesspolicies',
'businesspolicyitems'}
if not required.issubset(tables):
return
# Idempotent: never seed the bundle twice.
existing_config = bind.execute(
sa.text('SELECT configurationpolicyid FROM configurationpolicies '
'WHERE name = :name'),
{'name': CONFIG_POLICY_NAME}).first()
if existing_config:
return
# Resolve the shipped monitor ids by identity. Skip any that a site has
# not got; attach whatever shipped monitors are present.
monitor_ids = []
for kind, displayname in _SHIPPED_MONITORS:
row = bind.execute(
sa.text('SELECT endpointmonitorid FROM endpointmonitors '
'WHERE kind = :kind AND displayname = :displayname'),
{'kind': kind, 'displayname': displayname}).first()
if row:
monitor_ids.append(row[0])
if not monitor_ids:
return
now = datetime.utcnow()
config_policies = sa.table(
'configurationpolicies',
sa.column('name', sa.String),
sa.column('description', sa.Text),
sa.column('scopejson', sa.Text),
sa.column('source', sa.String),
sa.column('createddate', sa.DateTime),
sa.column('modifieddate', sa.DateTime),
sa.column('isactive', sa.Boolean),
)
op.bulk_insert(config_policies, [{
'name': CONFIG_POLICY_NAME,
'description': 'Product-shipped watch-list for the shopfloor apps '
'(eDNC, Universal Data Collector, MTConnect). Read-only.',
'scopejson': '{"type": "true"}',
'source': 'shipped',
'createddate': now,
'modifieddate': now,
'isactive': True,
}])
config_policy_id = bind.execute(
sa.text('SELECT configurationpolicyid FROM configurationpolicies '
'WHERE name = :name'),
{'name': CONFIG_POLICY_NAME}).scalar()
actions = sa.table(
'configurationpolicyactions',
sa.column('configurationpolicyid', sa.Integer),
sa.column('action', sa.String),
sa.column('mode', sa.String),
sa.column('desiredstate', sa.Text),
sa.column('params', sa.Text),
sa.column('intervalminutes', sa.Integer),
sa.column('sortorder', sa.Integer),
sa.column('createddate', sa.DateTime),
sa.column('modifieddate', sa.DateTime),
sa.column('isactive', sa.Boolean),
)
op.bulk_insert(actions, [{
'configurationpolicyid': config_policy_id,
'action': f'monitor:{monitorid}',
'mode': 'gate',
'desiredstate': None,
'params': None,
'intervalminutes': None,
'sortorder': index,
'createddate': now,
'modifieddate': now,
'isactive': True,
} for index, monitorid in enumerate(monitor_ids)])
business_policies = sa.table(
'businesspolicies',
sa.column('name', sa.String),
sa.column('description', sa.Text),
sa.column('priority', sa.Integer),
sa.column('source', sa.String),
sa.column('createddate', sa.DateTime),
sa.column('modifieddate', sa.DateTime),
sa.column('isactive', sa.Boolean),
)
op.bulk_insert(business_policies, [{
'name': BUSINESS_POLICY_NAME,
'description': 'Product-shipped Site Policy that applies the shopfloor '
'app monitors to every endpoint. Read-only.',
'priority': 100,
'source': 'shipped',
'createddate': now,
'modifieddate': now,
'isactive': True,
}])
business_policy_id = bind.execute(
sa.text('SELECT businesspolicyid FROM businesspolicies '
'WHERE name = :name'),
{'name': BUSINESS_POLICY_NAME}).scalar()
items = sa.table(
'businesspolicyitems',
sa.column('businesspolicyid', sa.Integer),
sa.column('configurationpolicyid', sa.Integer),
sa.column('sortorder', sa.Integer),
sa.column('createddate', sa.DateTime),
sa.column('modifieddate', sa.DateTime),
sa.column('isactive', sa.Boolean),
)
op.bulk_insert(items, [{
'businesspolicyid': business_policy_id,
'configurationpolicyid': config_policy_id,
'sortorder': 0,
'createddate': now,
'modifieddate': now,
'isactive': True,
}])
def downgrade():
bind = op.get_bind()
tables = _tables(bind)
if 'businesspolicies' not in tables or 'configurationpolicies' not in tables:
return
# Clear children explicitly rather than lean on ON DELETE CASCADE, which a
# test DB (sqlite) does not enforce by default.
business_policy_id = bind.execute(
sa.text('SELECT businesspolicyid FROM businesspolicies '
'WHERE name = :name'),
{'name': BUSINESS_POLICY_NAME}).scalar()
config_policy_id = bind.execute(
sa.text('SELECT configurationpolicyid FROM configurationpolicies '
'WHERE name = :name'),
{'name': CONFIG_POLICY_NAME}).scalar()
if business_policy_id is not None:
bind.execute(
sa.text('DELETE FROM businesspolicyitems '
'WHERE businesspolicyid = :id'),
{'id': business_policy_id})
bind.execute(
sa.text('DELETE FROM businesspolicies WHERE businesspolicyid = :id'),
{'id': business_policy_id})
if config_policy_id is not None:
bind.execute(
sa.text('DELETE FROM configurationpolicyactions '
'WHERE configurationpolicyid = :id'),
{'id': config_policy_id})
bind.execute(
sa.text('DELETE FROM configurationpolicies '
'WHERE configurationpolicyid = :id'),
{'id': config_policy_id})

51
plugins/endpoints/migrations/versions/0008_endpoint_drift.py Normal file
View File

@@ -0,0 +1,51 @@
"""Cache the warn-only reconciler drift on the endpoint state.
Adds endpointstates.driftjson: a per-asset JSON snapshot of the monitors a site
policy (or the shipped bundle) expects to be running on an endpoint but which the
latest agent report does not show as running. It is computed server-side on every
agent check-in and overwritten in place, the same ledger pattern the other state
snapshots (servicesjson, portsjson) already use - one current row per asset, no
history. Warn-only: nothing here is auto-remediated, it exists so an operator can
see the drift and intervene manually.
Nullable so an endpoint that has never checked in since the column landed reads
as "never computed" rather than "no drift".
Idempotent: the column-add checks the live schema first, so a db.create_all()
site (which built the column from the model already) and a re-run both stay clean
on strict MySQL 8.
"""
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = 'endpoints0008drift'
down_revision = 'endpoints0007shippedpolicy'
branch_labels = None
depends_on = None
def _has_column(table_name, column_name):
from sqlalchemy import inspect
inspector = inspect(op.get_bind())
if not inspector.has_table(table_name):
return False
return any(col['name'] == column_name
for col in inspector.get_columns(table_name))
def upgrade():
if _has_column('endpointstates', 'driftjson'):
return
op.add_column(
'endpointstates',
sa.Column('driftjson', sa.Text(length=16777215), nullable=True))
def downgrade():
if not _has_column('endpointstates', 'driftjson'):
return
op.drop_column('endpointstates', 'driftjson')

63
plugins/endpoints/migrations/versions/0009_operator_reports.py Normal file
View File

@@ -0,0 +1,63 @@
"""Operator issue reports (tray screenshot + free-text notes).
Adds operatorreports: one row per operator-submitted issue report for a PC
asset. The screenshot bytes live on disk under the instance uploads dir; this
table holds only metadata plus the stored file name so the image can be served
back and purged on retention. Reports are transient - a retention window
(default 7 days) purges old rows and their files - so this is a rolling
operator-signal feed, not a permanent record.
Idempotent: creation checks the live schema first so a db.create_all() site
(which already built the table) re-runs clean on strict MySQL 8.
"""
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = 'endpoints0009opreports'
down_revision = 'endpoints0008drift'
branch_labels = None
depends_on = None
def _has_table(table_name):
from sqlalchemy import inspect
return inspect(op.get_bind()).has_table(table_name)
def upgrade():
if _has_table('operatorreports'):
return
op.create_table(
'operatorreports',
sa.Column('operatorreportid', sa.Integer(), nullable=False),
sa.Column('assetid', sa.Integer(), nullable=False),
sa.Column('reason', sa.String(length=128), nullable=False,
server_default='Other (free text)'),
sa.Column('details', sa.Text(), nullable=True),
sa.Column('imagefilename', sa.String(length=255), nullable=True),
sa.Column('imagemimetype', sa.String(length=64), nullable=True),
sa.Column('imagebytesize', sa.Integer(), nullable=True),
sa.Column('reportedhostname', sa.String(length=255), nullable=True),
sa.Column('createdat', sa.DateTime(), nullable=False),
sa.PrimaryKeyConstraint('operatorreportid'),
sa.ForeignKeyConstraint(
['assetid'], ['assets.assetid'],
name='fk_operatorreports_asset', ondelete='CASCADE'),
)
op.create_index('idx_operatorreports_asset', 'operatorreports', ['assetid'])
op.create_index('idx_operatorreports_createdat', 'operatorreports',
['createdat'])
def downgrade():
if not _has_table('operatorreports'):
return
op.drop_index('idx_operatorreports_createdat', table_name='operatorreports')
op.drop_index('idx_operatorreports_asset', table_name='operatorreports')
op.drop_table('operatorreports')

3
plugins/endpoints/models/__init__.py
View File

@@ -3,7 +3,10 @@
from .endpoint import (
EndpointState, EndpointEvent, EndpointCommand, EndpointApplication,
)
from .monitor import EndpointMonitor
from .operatorreport import OperatorReport
__all__ = [
'EndpointState', 'EndpointEvent', 'EndpointCommand', 'EndpointApplication',
'EndpointMonitor', 'OperatorReport',
]

7
plugins/endpoints/models/endpoint.py
View File

@@ -75,6 +75,13 @@ class EndpointState(db.Model):
interfacesjson = db.Column(db.Text(length=16777215), nullable=True)
scheduledtasksjson = db.Column(db.Text(length=16777215), nullable=True)
# Warn-only reconciler output, refreshed on every agent check-in in the same
# commit as servicesjson. Each entry names a policy/shipped-attached monitor
# the endpoint is expected to be running but the latest snapshot does not
# report as running, so an operator sees the drift and intervenes manually.
# This is warn-only: nothing here is auto-remediated. Null = never computed.
driftjson = db.Column(db.Text(length=16777215), nullable=True)
# Last-known MTConnect agent run state, cached so the WebUI toggle can show
# a real state between command polls. Set by the agent's mtconnect_status
# result. Null = never reported (unknown).

87
plugins/endpoints/models/monitor.py Normal file
View File

@@ -0,0 +1,87 @@
"""Endpoint monitor definitions (the reusable monitor catalog).
A monitor definition is the "what": a named process or service the agent should
watch, with a bespoke display label separate from the raw OS name, plus optional
launch and collector paths. Definitions are the noun that a configuration policy
action points at; attaching a monitor to a PC type is a policy concern, not a
property of the definition itself.
Two sources:
shipped Internal apps we ship (eDNC, MTConnect, UDC). Seeded by migration.
Read-only in the UI: name and identity are fixed by the product.
local User-defined monitors built on the Settings page.
A definition is watched on an endpoint when a site policy in effect for that
endpoint's PC type includes a monitor action pointing at it.
"""
from datetime import datetime
from shopdb.api import db
def _utciso(value):
"""Naive-UTC datetime -> ISO string marked UTC, or None."""
return value.isoformat() + 'Z' if value else None
class EndpointMonitor(db.Model):
"""One reusable process or service monitor definition."""
__tablename__ = 'endpointmonitors'
endpointmonitorid = db.Column(db.Integer, primary_key=True)
# 'process' or 'service'. Decides which identity column is authoritative and
# which start/stop/status actions the row exposes.
kind = db.Column(db.String(16), nullable=False)
# The bespoke label operators see (e.g. 'MTConnect'). Distinct from the raw
# OS name because agent.exe is not obviously MTConnect.
displayname = db.Column(db.String(128), nullable=False)
# OS identity. For a process monitor, processname is the executable the agent
# matches (e.g. 'agent.exe'); for a service monitor, servicename is the
# Windows service name (e.g. 'MTConnect UDC Agent'). Exactly one is the
# authoritative identity for a given kind; the other stays null.
processname = db.Column(db.String(255), nullable=True)
servicename = db.Column(db.String(255), nullable=True)
# Optional paths supplied per monitor. launchpath starts the app; the
# collector paths point the agent at where the app writes data and logs.
launchpath = db.Column(db.String(512), nullable=True)
datacollectorpath = db.Column(db.String(512), nullable=True)
logcollectorpath = db.Column(db.String(512), nullable=True)
# 'shipped' (product-owned, seeded) or 'local' (user-built). Shipped rows are
# not editable or deletable through the CRUD API.
source = db.Column(db.String(16), nullable=False, default='local')
# Soft on/off so a monitor can be retired without losing policy references.
isactive = db.Column(db.Boolean, nullable=False, default=True)
createdat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
__table_args__ = (
db.UniqueConstraint('kind', 'displayname', name='uq_endpointmonitors_kinddisplay'),
db.Index('idx_endpointmonitors_source', 'source'),
)
def __repr__(self):
return f"<EndpointMonitor {self.displayname!r} kind={self.kind} source={self.source}>"
def to_dict(self):
return {
'endpointmonitorid': self.endpointmonitorid,
'kind': self.kind,
'displayname': self.displayname,
'processname': self.processname,
'servicename': self.servicename,
'launchpath': self.launchpath,
'datacollectorpath': self.datacollectorpath,
'logcollectorpath': self.logcollectorpath,
'source': self.source,
'isactive': self.isactive,
'createdat': _utciso(self.createdat),
}

82
plugins/endpoints/models/operatorreport.py Normal file
View File

@@ -0,0 +1,82 @@
"""Operator issue reports (screenshot + free-text notes from the tray).
An operator on a shop-floor PC opens the tray "Report an issue" dialog, writes a
free-text description, and optionally attaches a screenshot of their screen. The
agent uploads that to ShopDB, where it becomes one OperatorReport row linked to
the reporting PC's asset. The screenshot bytes live on disk under the instance
uploads dir (like employee photos); this row holds only metadata plus the
relative file path so the image can be served back and purged on retention.
Reports are transient by design: a retention window (default 7 days) purges old
rows and their image files, so this is a rolling operator-signal feed, not a
permanent record.
"""
from datetime import datetime
from shopdb.api import db
def _utciso(value):
"""Naive-UTC datetime -> ISO string marked UTC, or None."""
return value.isoformat() + 'Z' if value else None
class OperatorReport(db.Model):
"""One operator-submitted issue report for a PC asset."""
__tablename__ = 'operatorreports'
operatorreportid = db.Column(db.Integer, primary_key=True)
# The PC this report is about. ON DELETE CASCADE: when the asset goes, its
# operator reports go with it.
assetid = db.Column(
db.Integer,
db.ForeignKey('assets.assetid', ondelete='CASCADE'),
nullable=False,
)
# The dropdown reason. Only 'Other (free text)' is functional today, but the
# column is kept so the richer reason routing can return without a migration.
reason = db.Column(db.String(128), nullable=False, default='Other (free text)')
# The operator's free-text description. Nullable: a screenshot alone is a
# valid report.
details = db.Column(db.Text, nullable=True)
# Screenshot metadata. imagefilename is the stored file name under the
# operator-reports uploads dir; null when the operator declined a screenshot.
imagefilename = db.Column(db.String(255), nullable=True)
imagemimetype = db.Column(db.String(64), nullable=True)
imagebytesize = db.Column(db.Integer, nullable=True)
# The hostname the agent reported at upload time, kept for traceability even
# if the asset link is later re-pointed.
reportedhostname = db.Column(db.String(255), nullable=True)
createdat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
__table_args__ = (
db.Index('idx_operatorreports_asset', 'assetid'),
db.Index('idx_operatorreports_createdat', 'createdat'),
)
def __repr__(self):
return f"<OperatorReport {self.operatorreportid} asset={self.assetid}>"
def has_image(self):
return bool(self.imagefilename)
def to_dict(self):
return {
'operatorreportid': self.operatorreportid,
'assetid': self.assetid,
'reason': self.reason,
'details': self.details,
'hasimage': self.has_image(),
'imagemimetype': self.imagemimetype,
'imagebytesize': self.imagebytesize,
'reportedhostname': self.reportedhostname,
'createdat': _utciso(self.createdat),
}

37
plugins/endpoints/plugin.py
View File

@@ -486,6 +486,23 @@ class EndpointsPlugin(BasePlugin):
# the installed-software inventory
'group': 'software',
},
{
# Operator-submitted issue reports: a free-text note and an
# optional screenshot the operator captured from the tray. Sits
# in the Monitors tab beside the other watch-and-observe panels.
# A custom render carries the thumbnail grid and the expanding
# lightbox with left/right cycling the generic modes cannot.
'id': 'endpoints-operator-reports',
'title': 'Operator Reports',
'assettypes': ['computer'],
'endpoint': '/api/endpoints/panel/{assetid}/operatorreports',
'render': 'operatorreports',
'imagebase': '/api/endpoints/operatorreports',
'collapsible': True,
'empty': 'No operator reports for this endpoint.',
'position': 18,
'group': 'monitors',
},
]
def get_dashboard_widgets(self) -> List[Dict]:
@@ -802,6 +819,26 @@ class EndpointsPlugin(BasePlugin):
state.lastreportedat = datetime.utcnow()
# Warn-only reconciler: on this check-in, compare the monitors site
# policy expects to be running against what the snapshot reports, and
# cache the drift on the state in THIS transaction (same ledger commit as
# servicesjson). No start/stop is issued - drift is surfaced for an
# operator to act on. Reads the freshly reported services when present,
# else the last-known snapshot, so a report that omits services still
# reconciles against what we last saw. Fail-open: a reconciler error must
# never fail an otherwise-good check-in, so drift is left unchanged.
try:
if 'services' in payload:
reported_services = payload.get('services') or []
else:
reported_services = json.loads(state.servicesjson or '[]')
from .services.reconciler import compute_drift
state.driftjson = _as_json(compute_drift(assetid, reported_services))
except Exception:
logger.exception(
'reconciler failed for asset %s; leaving drift unchanged',
assetid)
# Log any keys the agent sent that the ingest path does not consume yet.
# New agent telemetry (disks, bios, memory, ...) lands here first: it is
# not persisted until the server grows columns for it, but logging the

0
plugins/endpoints/services/__init__.py Normal file
View File

158
plugins/endpoints/services/reconciler.py Normal file
View File

@@ -0,0 +1,158 @@
"""Warn-only monitor reconciler.
The reconciler answers one question on every agent check-in: for the monitors a
site policy (or the shipped bundle) attaches to this endpoint, is each one the
endpoint is EXPECTED to be running actually reported as running in the latest
snapshot? Every monitor that is expected-running but not observed-running is
DRIFT, and drift is surfaced to an operator for MANUAL intervention.
This is warn-only. The reconciler NEVER starts, stops, or enqueues anything - it
compares observed state against expected state and records the difference. Whether
to later automate remediation (a keep-alive that starts a process when it drifts)
is a separate, deliberate opt-in the product does not ship switched on.
Two seams are kept deliberately narrow so this stays easy to change:
monitor_expects_running(monitor)
The single place that decides "should this monitor be up?". Today every
attached monitor is treated as expected-running (attachment implies intent
to run). When an observe-only monitor kind arrives - one that only collects
general state and must never be flagged as drift - this is the one function
that branches on it. Nothing else in the reconciler encodes that policy.
compute_drift(asset_id, reported_services)
Pure given its inputs: it resolves the attached monitors and diffs them
against the reported running set. It reads no request state and writes no
rows, so the ingest path can call it inside its own transaction and tests
can call it directly.
"""
# The running-state vocabulary a watched process/service can report. A watched
# service reports a STATE STRING, not a boolean, and the vocabulary varies by
# source, so what counts as running is a single rule rather than two.
#
# It lives here, and api/routes.py imports it, because the panel and this
# reconciler MUST agree: a service the panel paints green that this flags as
# drift is a contradiction shown to the same person on the same page. It was
# two copies with a comment asking a human to keep them in lockstep, which is
# not a mechanism.
RUNNING_STATES = ('running', 'started', 'up', 'ok', 'active')
def monitor_expects_running(monitor):
"""Whether an attached monitor is expected to be running on an endpoint.
The seam for the "should be up" decision. Today attachment implies intent to
run, so every active monitor returns True. A future observe-only monitor kind
(general state collection, never keep-alive) branches HERE and returns False
so it is never reported as drift - no other part of the reconciler needs to
change.
"""
return bool(getattr(monitor, 'isactive', True))
def _monitor_identity(monitor):
"""The (kind, osname) a monitor watches, or None when it has no identity.
Mirrors how the services panel reads a monitor's OS identity: a process
monitor is keyed by its processname, a service monitor by its servicename.
"""
kind = (getattr(monitor, 'kind', '') or '').lower() or None
if kind == 'process':
osname = getattr(monitor, 'processname', None)
elif kind == 'service':
osname = getattr(monitor, 'servicename', None)
else:
osname = None
osname = (osname or '').strip()
if not kind or not osname:
return None
return kind, osname
def _reported_running_keys(reported_services):
"""Set of (kind, lowercased-name) the snapshot reports as RUNNING.
Tolerant of the two shapes the agent snapshot uses: dict entries with
name/service and state/status (and an optional kind), or bare strings.
Only running entries land in the set; a stopped or unknown entry is absent,
which is what makes an expected-running monitor read as drift.
"""
running = set()
for entry in reported_services or ():
if isinstance(entry, dict):
name = entry.get('name') or entry.get('service') or ''
raw = (entry.get('state') or entry.get('status') or '').strip().lower()
kind = (entry.get('kind') or '').lower() or None
else:
name, raw, kind = str(entry), '', None
name = name.strip().lower()
if not name or raw not in RUNNING_STATES:
continue
running.add((kind, name))
return running
def _is_running(kind, osname, running_keys):
"""Whether a monitor's identity appears running in the snapshot.
A snapshot entry may omit kind (older agents, or a plain string list), so a
match on name alone under an unknown kind still counts as running - the
reconciler must not invent drift just because the agent did not label a row.
"""
needle = osname.strip().lower()
if (kind, needle) in running_keys:
return True
if (None, needle) in running_keys:
return True
return False
def compute_drift(asset_id, reported_services):
"""Compute the current drift list for an endpoint. Pure, no writes.
Resolves the monitors attached to the asset's PC type (shipped defaults plus
site policy), keeps only those that are expected-running, and returns one
entry per monitor whose identity is not reported as running. Returns an empty
list when nothing is attached or nothing has drifted, so the caller can store
it as "computed, no drift" distinct from a null "never computed".
Isolated from Flask request state: it reads the monitor catalog and the
passed-in snapshot only, so the ingest path calls it inside its own
transaction and tests call it directly.
"""
try:
from plugins.endpoints.api.routes import _resolved_monitors_for_asset
except Exception:
return []
monitors = _resolved_monitors_for_asset(asset_id) or []
if not monitors:
return []
running_keys = _reported_running_keys(reported_services)
drift = []
seen = set()
for monitor in monitors:
if not monitor_expects_running(monitor):
continue
identity = _monitor_identity(monitor)
if identity is None:
continue
kind, osname = identity
key = (kind, osname.lower())
if key in seen:
continue
seen.add(key)
if _is_running(kind, osname, running_keys):
continue
drift.append({
'monitorid': getattr(monitor, 'endpointmonitorid', None),
'kind': kind,
'name': osname,
'displayname': getattr(monitor, 'displayname', None) or osname,
'expected': 'running',
'observed': 'notrunning',
})
return drift

7
requirements-dev.in
View File

@@ -10,3 +10,10 @@
pytest>=7.0
pytest-flask>=1.2
pytest-cov>=4.0
# Parallel test execution. The suite is ~2300 tests whose cost is per-test
# fixture teardown (create_all/drop_all is ~0.06s each, ~40% of the wall clock)
# rather than any single slow test, so it parallelises almost linearly: 331s ->
# 103s on 8 cores, same 2309 passing. NOT in pytest.ini addopts - worker startup
# is dead weight on the targeted runs that make up most of a working day. Opt in
# with -n auto when running the whole suite.
pytest-xdist>=3.5

36
requirements-dev.txt
View File

@@ -1,9 +1,5 @@
# This file was autogenerated by uv via the following command:
# uv pip compile requirements-dev.in -c requirements.txt --universal --generate-hashes -o requirements-dev.txt
#
# The -c constraint pins every shared dependency to the SAME version production
# runs. Without it the two files were compiled at different times and drifted:
# CI tested against alembic 1.18.5 while sites installed 1.18.4.
alembic==1.18.4 \
--hash=sha256:a5ed4adcf6d8a4cb575f3d759f071b03cd6e5c7618eb796cb52497be25bfe19a \
--hash=sha256:cb6e1fd84b6174ab8dbb2329f86d631ba9559dd78df550b57804d607672cedbc
@@ -434,6 +430,10 @@ email-validator==2.3.0 \
# via
# -c requirements.txt
# -r requirements.in
execnet==2.1.2 \
--hash=sha256:63d83bfdd9a23e35b9c6a3261412324f964c2ec8dcd8d3c6916ee9373e0befcd \
--hash=sha256:67fba928dd5a544b783f6056f449e5e3931a5c378b128bc18501f7ea79e296ec
# via pytest-xdist
flask==3.1.3 \
--hash=sha256:0ef0e52b8a9cd932855379197dd8f94047b359ca0a78695144304cb45f87c9eb \
--hash=sha256:f4bcbefc124291925f1a26446da31a5178f9483862233b23c0c96a20701f670c
@@ -547,9 +547,9 @@ greenlet==3.5.0 ; platform_machine == 'AMD64' or platform_machine == 'WIN32' or
# via
# -c requirements.txt
# sqlalchemy
idna==3.13 \
--hash=sha256:585ea8fe5d69b9181ec1afba340451fba6ba764af97026f92a91d4eef164a242 \
--hash=sha256:892ea0cde124a99ce773decba204c5552b69c3c67ffd5f232eb7696135bc8bb3
idna==3.19 \
--hash=sha256:5e0811a4383b21dc5838069f801c4fb62113b7447663d2530d2bd6e77b49bf15 \
--hash=sha256:815e7be7a7806d54abb586dc943addc79e8b2ee16915059658cbeff4b1b43bf4
# via
# -c requirements.txt
# email-validator
@@ -715,10 +715,13 @@ mysql-connector-python==9.7.0 \
# via
# -c requirements.txt
# -r requirements.in
packaging==26.2 \
--hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \
--hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661
# via pytest
packaging==26.3 \
--hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 \
--hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c
# via
# -c requirements.txt
# -r requirements.in
# pytest
pluggy==1.6.0 \
--hash=sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3 \
--hash=sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746
@@ -735,9 +738,9 @@ pygments==2.20.0 \
--hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \
--hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176
# via pytest
pyjwt==2.12.1 \
--hash=sha256:28ca37c070cad8ba8cd9790cd940535d40274d22f80ab87f3ac6a713e6e8454c \
--hash=sha256:c74a7a2adf861c04d002db713dd85f84beb242228e671280bf709d765b03672b
pyjwt==2.13.0 \
--hash=sha256:41571c89ca91598c79e8ef18a2d07367d4810fbbd6f637794879baf1b7703423 \
--hash=sha256:66adcc2aff09b3f1bbd95fc1e1577df8ac8723c978552fd43304c8a290ac5728
# via
# -c requirements.txt
# flask-jwt-extended
@@ -754,6 +757,7 @@ pytest==9.1.1 \
# -r requirements-dev.in
# pytest-cov
# pytest-flask
# pytest-xdist
pytest-cov==7.1.0 \
--hash=sha256:30674f2b5f6351aa09702a9c8c364f6a01c27aae0c1366ae8016160d1efc56b2 \
--hash=sha256:a0461110b7865f9a271aa1b51e516c9a95de9d696734a2f71e3e78f46e1d4678
@@ -762,6 +766,10 @@ pytest-flask==1.3.0 \
--hash=sha256:58be1c97b21ba3c4d47e0a7691eb41007748506c36bf51004f78df10691fa95e \
--hash=sha256:c0e36e6b0fddc3b91c4362661db83fa694d1feb91fa505475be6732b5bc8c253
# via -r requirements-dev.in
pytest-xdist==3.8.0 \
--hash=sha256:202ca578cfeb7370784a8c33d6d05bc6e13b4f25b5053c30a152269fd10f0b88 \
--hash=sha256:7e578125ec9bc6050861aa93f2d59f1d8d085595d6551c2c90b6f4fad8d3a9f1
# via -r requirements-dev.in
python-dotenv==1.2.2 \
--hash=sha256:1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a \
--hash=sha256:2c371a91fbd7ba082c2c1dc1f8bf89ca22564a087c2c287cd9b662adde799cf3

162
scripts/Start-ShopDBDev.ps1 Normal file
View File

@@ -0,0 +1,162 @@
<#
.SYNOPSIS
Save-and-run launcher for a ShopDB dev workstation (stable or edge).
.DESCRIPTION
Download this one file and run it. It clones (or updates) the repo at the
channel branch into a local path, then hands off to scripts/dev-bootstrap.ps1
for the real dependency-check / build / up work.
ge-aero/shopdb-flask is a PRIVATE repo, but this needs no token pasting: the
clone uses a plain https URL and Git Credential Manager supplies your stored
GitHub credential automatically - the same way `git clone` works in the wiki
DEVELOPMENT-SETUP procedure. If you have ever cloned this repo or signed in to
GitHub in Visual Studio / VS Code on this box, GCM already has what it needs.
This is a downloaded FILE, not a web `iex` one-liner, precisely because a
private repo cannot be fetched anonymously from raw.githubusercontent.com. A
saved script sidesteps that entirely - only the git operations need auth, and
GCM handles those.
.PARAMETER Channel
'stable' (the newest release tag) or 'edge' (main, the trunk). Defaults to 'stable'.
.PARAMETER TargetDir
Where to clone. Defaults to %USERPROFILE%\shopdb-dev, a separate working copy
from any repo you already have open.
.EXAMPLE
# Bleeding edge:
powershell -ExecutionPolicy Bypass -File .\Start-ShopDBDev.ps1 -Channel edge
.EXAMPLE
# Stable:
powershell -ExecutionPolicy Bypass -File .\Start-ShopDBDev.ps1 -Channel stable
.NOTES
ASCII only, no emoji or smart punctuation, per repo style. PowerShell 5.1+.
Needs Git (with Credential Manager, the Git-for-Windows default);
dev-bootstrap.ps1 offers to winget-install anything else that is missing.
#>
[CmdletBinding()]
param(
[ValidateSet('stable', 'edge')]
[string]$Channel = 'stable',
[string]$TargetDir
)
$ErrorActionPreference = 'Stop'
# Reset leftovers from an interrupted or timed-out prior run so the retry starts
# clean. Proxy bottlenecks can kill a build mid-flight, leaving orphaned
# containers holding the channel ports, dangling build cache from the partial
# build, and sometimes a half-written clone. This is scoped to the channel and
# is non-destructive to data: the compose down keeps named volumes (the DB), and
# only dangling build cache is pruned - never tagged images or volumes.
function Clear-InterruptedRun {
param(
[Parameter(Mandatory)][string]$Project,
[Parameter(Mandatory)][string]$Target
)
if (Get-Command docker -ErrorAction SilentlyContinue) {
# docker/compose write warnings to stderr (e.g. "No resource found to
# remove"), which would trip $ErrorActionPreference='Stop' and abort the
# whole run. These cleanup steps are best-effort, so soften the preference
# and swallow all streams for their duration.
$previous = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
try {
# Only try compose down when the engine is actually responding, otherwise
# skip quietly - a fresh box may not have Docker started yet.
docker info *> $null
if ($LASTEXITCODE -eq 0) {
Write-Host "==> Cleaning up any interrupted '$Project' stack" -ForegroundColor Cyan
# --remove-orphans clears containers left by a killed build; volumes are
# intentionally kept so the DB survives a retry.
docker compose -p $Project down --remove-orphans *> $null
# Reclaim only dangling build cache from failed/partial builds.
docker builder prune -f *> $null
}
}
finally {
$ErrorActionPreference = $previous
}
}
# A directory that exists but has no valid .git is a broken/partial clone;
# remove it so the clone below can start fresh instead of failing.
if ((Test-Path $Target) -and -not (Test-Path (Join-Path $Target '.git'))) {
Write-Host "==> Removing incomplete clone at $Target" -ForegroundColor Cyan
Remove-Item -Path $Target -Recurse -Force
}
# A clone created by an elevated (Administrator) run is owned by
# BUILTIN\Administrators; a later non-elevated run then trips git's
# "detected dubious ownership" guard and every git op fails with exit 128.
# That stale clone is effectively unusable for this user, so detect the
# condition and remove it here for a clean re-clone rather than surfacing a
# misleading auth error later.
if (Test-Path (Join-Path $Target '.git')) {
$previous = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
try { $probe = git -C $Target rev-parse --is-inside-work-tree 2>&1 } catch { $probe = "$_" }
finally { $ErrorActionPreference = $previous }
if ($probe -match 'dubious ownership') {
Write-Host "==> Removing clone with mismatched ownership at $Target (was it created from an elevated prompt?)" -ForegroundColor Cyan
Remove-Item -Path $Target -Recurse -Force
}
}
}
# Plain https URL. No token in the URL: Git Credential Manager injects the
# stored GitHub credential for github.com, so clone/fetch authenticate against
# the private repo without anything sensitive touching this script or disk.
$repoUrl = 'https://github.com/ge-aero/shopdb-flask.git'
# Channels follow how this repo ships: a release is a TAG, the trunk is main.
# Upstream this read 'dev-be' for edge, which is that repo's trunk branch and
# does not exist here - the launcher would have cloned a missing branch.
$branch = 'main'
if ($Channel -ne 'edge') {
# Newest release tag, so 'stable' means a thing that was released rather
# than whatever the trunk happens to be today.
$tag = (git ls-remote --tags --refs --sort='-v:refname' $repoUrl 'v*' |
Select-Object -First 1) -replace '.*refs/tags/', ''
if ($tag) { $branch = $tag }
else { Write-Warning "No v* tag found; falling back to main for 'stable'." }
}
$target = if ($TargetDir) { $TargetDir } else { Join-Path $env:USERPROFILE 'shopdb-dev' }
Write-Host "ShopDB dev launcher" -ForegroundColor White
Write-Host " Channel : $Channel ($branch)" -ForegroundColor White
Write-Host " Target : $target" -ForegroundColor White
if (-not (Get-Command git -ErrorAction SilentlyContinue)) {
throw 'git is required. Install Git for Windows (winget install Git.Git) and re-run.'
}
$project = if ($Channel -eq 'edge') { 'shopdb-edge' } else { 'shopdb-stable' }
Clear-InterruptedRun -Project $project -Target $target
if (Test-Path (Join-Path $target '.git')) {
Write-Host "==> Updating existing clone" -ForegroundColor Cyan
git -C $target fetch origin $branch
if ($LASTEXITCODE -ne 0) { throw "git fetch failed ($LASTEXITCODE). If this is an auth error, clone the repo once in Visual Studio / VS Code so Credential Manager stores your GitHub login." }
git -C $target checkout $branch
if ($LASTEXITCODE -ne 0) { throw "git checkout failed ($LASTEXITCODE)." }
git -C $target pull --ff-only origin $branch
if ($LASTEXITCODE -ne 0) { throw "git pull failed ($LASTEXITCODE)." }
} else {
Write-Host "==> Cloning $branch" -ForegroundColor Cyan
git clone --branch $branch $repoUrl $target
if ($LASTEXITCODE -ne 0) { throw "git clone failed ($LASTEXITCODE). If this is an auth error, sign in to GitHub once in Visual Studio / VS Code (or run 'git clone $repoUrl' manually) so Credential Manager stores your login, then re-run." }
}
$bootstrap = Join-Path $target 'scripts\dev-bootstrap.ps1'
if (-not (Test-Path $bootstrap)) {
throw "dev-bootstrap.ps1 not found at $bootstrap. Is the '$branch' branch correct?"
}
Write-Host "==> Handing off to dev-bootstrap.ps1" -ForegroundColor Cyan
& $bootstrap -Channel $Channel

346
scripts/build-channel-installer.ps1 Normal file
View File

@@ -0,0 +1,346 @@
<#
.SYNOPSIS
Build a channel-aware ShopDB distributable installer (stable or edge).
.DESCRIPTION
Produces ONE self-contained installer package for the requested channel:
stable -> built from 'main', no DEV badge (SHOPDB_ENV_LABEL empty).
edge -> built from 'main' (the trunk), red DEV badge (SHOPDB_ENV_LABEL=DEV).
This is the distributable counterpart to scripts/dev-bootstrap.ps1 (which
stands a dev box up FROM the repo). Here the operator has no repo: the script
builds the app image from the channel branch on a connected build box, saves
it plus MySQL into a loadable tarball (reusing the exact bundle mechanics of
build-offline-bundle.ps1), and drops a runnable package:
dist/shopdb-<channel>-<version>/
shopdb-stack-<version>.tar.gz # docker load input
shopdb-stack-<version>.tar.gz.sha256
docker-compose.airgap.yml # image-only run template
docker-compose.channel.yml # SHOPDB_ENV_LABEL overlay
.env # channel-filled, secrets as CHANGE_ME_* sentinels
Install-ShopDB.ps1 # site-side one-command installer
README.txt
Keeping stable and edge as ONE parameterized builder (not two scripts) is the
same anti-drift decision as the config-driven badge: the channels differ only
by branch, image tag, and label - all data, not code.
.PARAMETER Channel
'stable' (the newest release tag) or 'edge' (main, the trunk). Required.
.PARAMETER Version
Image tag / bundle version. Defaults to a channel-prefixed date stamp so two
channels built the same day never collide (e.g. 'stable-2024.06.01').
.PARAMETER Ref
Override the git ref to build. Defaults to the newest v* tag for stable, 'main' for
edge. Lets a release engineer cut an installer from a tag or fix branch.
.PARAMETER MysqlImage
MySQL image the stack runs (default mysql:8.4). Must match db.image in
docker-compose.airgap.yml.
.PARAMETER OutDir
Root for the generated package (default ./dist).
.PARAMETER SkipBuild
Reuse an already-built shopdb-flask:<Version> image; only save + package.
.EXAMPLE
pwsh scripts/build-channel-installer.ps1 -Channel stable
.EXAMPLE
pwsh scripts/build-channel-installer.ps1 -Channel edge -Version edge-preview3
.NOTES
ASCII only, no emoji or smart punctuation, per repo style. PowerShell 5.1+.
Requires Docker Desktop and (unless -SkipBuild) clean access to build the
image. Secrets ship as CHANGE_ME_* placeholders and the generated installer
refuses to start until they are rotated, so a package can never go live on
shipped credentials.
#>
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[ValidateSet('stable', 'edge')]
[string]$Channel,
[string]$Version,
[string]$Ref,
[string]$MysqlImage = 'mysql:8.4',
[string]$OutDir = 'dist',
[switch]$SkipBuild
)
$ErrorActionPreference = 'Stop'
# Channel -> (branch, badge label). The single source of channel truth, mirrored
# from dev-bootstrap.ps1 so the two entry points never disagree about what a
# channel means.
$channels = @{
stable = @{ Ref = 'main'; EnvLabel = '' }
edge = @{ Ref = 'main'; EnvLabel = 'DEV' }
}
function Assert-LastExit([string]$What) {
if ($LASTEXITCODE -ne 0) { throw "$What failed (exit $LASTEXITCODE)" }
}
if (-not (Get-Command docker -ErrorAction SilentlyContinue)) {
throw 'docker not found on PATH. Install Docker Desktop and retry.'
}
if (-not (Get-Command git -ErrorAction SilentlyContinue)) {
throw 'git not found on PATH. Install Git and retry.'
}
$config = $channels[$Channel]
$branch = if ($Ref) { $Ref } else { $config.Ref }
if (-not $Version) { $Version = "$Channel-$(Get-Date -Format 'yyyy.MM.dd')" }
$appImage = "shopdb-flask:$Version"
$repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..')).Path
Write-Host "ShopDB channel installer builder" -ForegroundColor White
Write-Host " Channel : $Channel" -ForegroundColor White
Write-Host " Branch : $branch" -ForegroundColor White
Write-Host " Version : $Version" -ForegroundColor White
Write-Host " Label : $(if ($config.EnvLabel) { $config.EnvLabel } else { '(none)' })" -ForegroundColor White
Push-Location $repoRoot
try {
# --- 1. Build the image from the channel branch --------------------------
if (-not $SkipBuild) {
# Build from the channel branch WITHOUT switching the operator's working
# tree: create a throwaway worktree at the ref, build there, remove it.
# This keeps a release build reproducible and side-effect-free even when
# the operator has local edits checked out.
$worktree = Join-Path ([System.IO.Path]::GetTempPath()) "shopdb-build-$Version"
Write-Host "==> Fetching origin/$branch" -ForegroundColor Cyan
git fetch origin $branch
Assert-LastExit 'git fetch'
if (Test-Path $worktree) {
git worktree remove --force $worktree 2>$null
Remove-Item $worktree -Recurse -Force -ErrorAction SilentlyContinue
}
Write-Host "==> Adding build worktree at $worktree" -ForegroundColor Cyan
git worktree add --detach $worktree "origin/$branch"
Assert-LastExit 'git worktree add'
try {
Write-Host "==> Building $appImage from $branch" -ForegroundColor Cyan
docker build -t $appImage $worktree
Assert-LastExit 'docker build'
} finally {
Write-Host "==> Removing build worktree" -ForegroundColor Cyan
git worktree remove --force $worktree 2>$null
Remove-Item $worktree -Recurse -Force -ErrorAction SilentlyContinue
}
} else {
Write-Host "==> Skipping build; reusing $appImage" -ForegroundColor Yellow
}
Write-Host "==> Pulling $MysqlImage" -ForegroundColor Cyan
docker pull $MysqlImage
Assert-LastExit 'docker pull'
# --- 2. Save image bundle into the package dir ---------------------------
$pkgName = "shopdb-$Channel-$Version"
$pkgDir = Join-Path $OutDir $pkgName
if (Test-Path $pkgDir) { Remove-Item $pkgDir -Recurse -Force }
New-Item -ItemType Directory -Path $pkgDir -Force | Out-Null
$stem = "shopdb-stack-$Version"
$tar = Join-Path $pkgDir "$stem.tar"
$gz = Join-Path $pkgDir "$stem.tar.gz"
Write-Host "==> Saving $appImage + $MysqlImage" -ForegroundColor Cyan
docker save -o $tar $appImage $MysqlImage
Assert-LastExit 'docker save'
Write-Host "==> Compressing to $gz" -ForegroundColor Cyan
$inStream = [System.IO.File]::OpenRead($tar)
$outStream = [System.IO.File]::Create($gz)
$gzip = New-Object System.IO.Compression.GzipStream($outStream, [System.IO.Compression.CompressionMode]::Compress)
try {
$inStream.CopyTo($gzip)
} finally {
$gzip.Dispose(); $outStream.Dispose(); $inStream.Dispose()
}
Remove-Item $tar
$hash = (Get-FileHash $gz -Algorithm SHA256).Hash.ToLower()
"$hash $stem.tar.gz" | Set-Content -Path "$gz.sha256" -NoNewline
# --- 3. Copy the run templates into the package -------------------------
Copy-Item (Join-Path $repoRoot 'docker-compose.airgap.yml') $pkgDir
Copy-Item (Join-Path $repoRoot 'docker-compose.channel.yml') $pkgDir
# --- 4. Channel-filled .env ---------------------------------------------
# Secrets ship as explicit CHANGE_ME_* sentinels, NOT generated values. The
# site-side installer refuses to start until every sentinel is replaced, so a
# distributable can never silently go live on shipped credentials. Ports,
# image tag, and the channel label are real because they are not secrets.
$apiPort = if ($Channel -eq 'edge') { 5011 } else { 5001 }
$mysqlPort = if ($Channel -eq 'edge') { 3316 } else { 3306 }
$envLines = @(
"# ShopDB $Channel installer .env (built $(Get-Date -Format s)).",
"# REQUIRED: replace every CHANGE_ME_* value below before installing.",
"# Generate strong secrets, e.g.:",
"# python -c ""import secrets; print(secrets.token_urlsafe(64))""",
"FLASK_APP=wsgi.py",
"FLASK_ENV=production",
"SECRET_KEY=CHANGE_ME_SECRET_KEY",
"JWT_SECRET_KEY=CHANGE_ME_JWT_SECRET_KEY",
"DATABASE_URL=mysql+pymysql://shopdb:CHANGE_ME_DB_PASSWORD@db:3306/shopdb_flask?charset=utf8mb4",
"CORS_ORIGINS=https://localhost:$apiPort",
"LOG_LEVEL=INFO",
"MYSQL_ROOT_PASSWORD=CHANGE_ME_ROOT_PASSWORD",
"MYSQL_PASSWORD=CHANGE_ME_DB_PASSWORD",
"MYSQL_PORT=$mysqlPort",
"API_PORT=$apiPort",
"IMAGE_TAG=$Version",
"SHOPDB_ENV_LABEL=$($config.EnvLabel)"
)
Set-Content -Path (Join-Path $pkgDir '.env') -Value $envLines -Encoding ASCII
# --- 5. Site-side installer script --------------------------------------
# One command at the site: verify checksum, load the image, bring the stack
# up with the airgap template + channel overlay. Mirrors the run steps in
# docs/DEPLOY-AIRGAP.md but wraps them so an operator runs a single file.
$installer = @'
<#
.SYNOPSIS
Install / start this ShopDB stack on the target host (one command).
.DESCRIPTION
Verifies the image-bundle checksum, docker-loads it, and brings the stack up
from the bundled compose templates and .env. Re-runnable: on later runs it
skips the load if the image is already present and just applies the compose.
#>
[CmdletBinding()]
param([switch]$Recreate)
$ErrorActionPreference = 'Stop'
Set-Location $PSScriptRoot
if (-not (Get-Command docker -ErrorAction SilentlyContinue)) {
throw 'docker not found. Install Docker Desktop / Engine and retry.'
}
$bundle = Get-ChildItem -Filter 'shopdb-stack-*.tar.gz' | Select-Object -First 1
if (-not $bundle) { throw 'No shopdb-stack-*.tar.gz found next to this script.' }
# Refuse to start on shipped placeholder secrets. A distributable must never go
# live on credentials that ship in the box; the operator has to rotate every
# CHANGE_ME_* in .env first. This is a hard gate, not a warning.
$sentinels = Select-String -Path '.env' -Pattern 'CHANGE_ME_' -SimpleMatch
if ($sentinels) {
Write-Host 'Refusing to start: .env still has placeholder secrets.' -ForegroundColor Red
Write-Host 'Replace every CHANGE_ME_* value with a real secret, then re-run:'
foreach ($line in $sentinels) { Write-Host " $($line.Line)" -ForegroundColor Yellow }
Write-Host ''
Write-Host 'Generate strong values, e.g.:'
Write-Host ' python -c "import secrets; print(secrets.token_urlsafe(64))"'
throw 'Placeholder secrets present in .env.'
}
$sumFile = "$($bundle.Name).sha256"
if (Test-Path $sumFile) {
Write-Host "==> Verifying $($bundle.Name) checksum" -ForegroundColor Cyan
$expected = ((Get-Content $sumFile -Raw).Trim() -split '\s+')[0].ToLower()
$actual = (Get-FileHash $bundle.FullName -Algorithm SHA256).Hash.ToLower()
if ($expected -ne $actual) { throw "Checksum mismatch. Expected $expected, got $actual." }
Write-Host " OK" -ForegroundColor Green
} else {
Write-Host "==> No checksum file; skipping verification" -ForegroundColor Yellow
}
Write-Host "==> Loading image bundle" -ForegroundColor Cyan
docker load -i $bundle.FullName
if ($LASTEXITCODE -ne 0) { throw "docker load failed ($LASTEXITCODE)." }
$composeArgs = @(
'compose', '--env-file', '.env',
'-f', 'docker-compose.airgap.yml',
'-f', 'docker-compose.channel.yml'
)
if ($Recreate) { $composeArgs += @('up', '-d', '--force-recreate') }
else { $composeArgs += @('up', '-d') }
Write-Host "==> Starting the stack" -ForegroundColor Cyan
& docker @composeArgs
if ($LASTEXITCODE -ne 0) { throw "docker compose up failed ($LASTEXITCODE)." }
$apiPort = (Select-String -Path '.env' -Pattern '^API_PORT=(.+)$').Matches.Groups[1].Value
Write-Host ''
Write-Host "ShopDB is up: https://localhost:$apiPort" -ForegroundColor Green
Write-Host "Seed the first admin (once):"
Write-Host " docker compose --env-file .env -f docker-compose.airgap.yml exec api flask seed admin <user> <email> <password>"
'@
Set-Content -Path (Join-Path $pkgDir 'Install-ShopDB.ps1') -Value $installer -Encoding ASCII
# --- 6. README ----------------------------------------------------------
$badgeLine = if ($config.EnvLabel) {
"This is the EDGE (bleeding-edge) build from '$branch'. The sidebar shows a red '$($config.EnvLabel)' badge."
} else {
"This is the STABLE build from '$branch'. No DEV badge."
}
$readme = @(
"ShopDB $Channel installer ($Version)",
"==================================================",
"",
$badgeLine,
"",
"Contents:",
" shopdb-stack-$Version.tar.gz Docker image bundle (app + MySQL)",
" shopdb-stack-$Version.tar.gz.sha256 Checksum for the bundle",
" docker-compose.airgap.yml Image-only run template",
" docker-compose.channel.yml Channel label overlay",
" .env Pre-filled config (ROTATE SECRETS)",
" Install-ShopDB.ps1 One-command installer",
"",
"Install (Windows, from this folder):",
" 1. Edit .env: replace every CHANGE_ME_* value with a real secret.",
" 2. powershell -ExecutionPolicy Bypass -File .\Install-ShopDB.ps1",
"",
"The installer REFUSES to start while any CHANGE_ME_* placeholder remains,",
"so the stack can never come up on shipped credentials.",
"",
"The API listens on https://localhost:$apiPort (self-signed cert on first",
"run). Adjust the port or CORS origins in .env if needed. Both channels can",
"run on one host at once because stable and edge use different ports and",
"compose project state.",
"",
"Generate strong secrets with:",
" python -c ""import secrets; print(secrets.token_urlsafe(64))"""
)
Set-Content -Path (Join-Path $pkgDir 'README.txt') -Value $readme -Encoding ASCII
# --- 7. Summary ---------------------------------------------------------
$sizeMb = [math]::Round((Get-Item $gz).Length / 1MB, 1)
Write-Host ''
Write-Host "Installer package ready:" -ForegroundColor Green
Write-Host " $pkgDir"
Write-Host " bundle : $stem.tar.gz ($sizeMb MB)"
Write-Host " sha256 : $hash"
Write-Host " label : $(if ($config.EnvLabel) { $config.EnvLabel } else { '(none, stable)' })"
Write-Host ''
Write-Host "Ship the folder; the operator runs Install-ShopDB.ps1."
}
finally {
Pop-Location
}

2
scripts/check-naming-and-style.sh
View File

@@ -21,6 +21,8 @@ VIOLATIONS=0
EXCLUDES=(
--exclude-dir=venv
--exclude-dir=env
--exclude-dir=.venv
--exclude-dir=node_modules
--exclude-dir=__pycache__
--exclude-dir=dist

344
scripts/dev-bootstrap.ps1 Normal file
View File

@@ -0,0 +1,344 @@
<#
.SYNOPSIS
Turnkey ShopDB developer-workstation bootstrap for Windows.
.DESCRIPTION
Stands up a ShopDB stack from a git branch on a fresh dev box with as few
manual steps as possible:
1. Checks the tools a dev needs (git, Docker) and offers to install any
that are missing via winget (with a prompt - nothing is installed
silently). Python is optional and only checked when -CheckPython is
passed, because the app itself runs entirely in the container.
2. Picks a channel:
stable -> pulls the 'stable' image (built from main), no DEV badge.
edge -> pulls the 'edge' image (built from main), red DEV badge.
The two channels use different compose project names, host ports, and
named volumes, so they run side by side without colliding.
3. Generates a .env for the channel (dev-grade secrets, NOT for production)
if one does not already exist, then PULLS the prebuilt channel image
from GHCR and brings up the stack. Nothing is built on the host: the
image is produced by CI, so a pilot needs only Docker (no Node/npm and
no source compile behind the corporate proxy).
Windows-only by design: pre-release audience is Windows. Not a production
installer - see deploy/windows/installer/shopdb-install.ps1 for that.
.PARAMETER Channel
'stable' (the newest release tag) or 'edge' (main, the trunk). Defaults to 'stable'.
.PARAMETER Ref
Override the git ref to build. Defaults to the newest v* tag for stable, 'main' for
edge. Retained for compatibility; the pull flow selects the image by channel
tag, so this now only affects which branch the working tree is prepared on.
.PARAMETER CheckPython
Also verify a local Python 3.11+ (only needed if you run tests/tooling on
the host rather than in the container).
.PARAMETER SkipDependencyInstall
Report missing dependencies and stop instead of offering winget installs.
Handy for CI-style dry runs.
.EXAMPLE
./scripts/dev-bootstrap.ps1 -Channel edge
.NOTES
ASCII only, no emoji or smart punctuation, per repo style. PowerShell 5.1+.
#>
[CmdletBinding()]
param(
[ValidateSet('stable', 'edge')]
[string]$Channel = 'stable',
[string]$Ref,
[switch]$CheckPython,
[switch]$SkipDependencyInstall
)
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
# Channel -> (branch, compose project, host ports, badge label). Kept in one
# table so adding a channel later is a single edit. Ports are offset for edge so
# both stacks bind distinct host ports; the container-internal ports never move.
$channels = @{
stable = @{
Ref = 'main'
Project = 'shopdb-stable'
ApiPort = 5001
MysqlPort = 3306
EnvLabel = ''
ImageTag = 'stable'
}
edge = @{
Ref = 'main'
Project = 'shopdb-edge'
ApiPort = 5011
MysqlPort = 3316
EnvLabel = 'DEV'
ImageTag = 'edge'
}
}
# GHCR repository that CI (.github/workflows/publish-images.yml) publishes the
# per-channel images to. Pilots PULL this instead of building from source,
# because building behind an inspecting proxy is unreliable.
$imageRepo = 'ghcr.io/ge-aero/shopdb-flask'
function Write-Step {
param([string]$Message)
Write-Host ''
Write-Host "==> $Message" -ForegroundColor Cyan
}
function Write-Ok {
param([string]$Message)
Write-Host " OK: $Message" -ForegroundColor Green
}
function Write-Warn {
param([string]$Message)
Write-Host " WARN: $Message" -ForegroundColor Yellow
}
function Test-Command {
param([string]$Name)
return [bool](Get-Command $Name -ErrorAction SilentlyContinue)
}
# Detect a corporate TLS-inspection root CA (e.g. Zscaler) by inspecting the
# certificate chain the host uses to reach the npm registry, then stage it as
# corp-root-ca.pem in the build context so the Docker frontend build can trust
# it. On a clean network this simply stages a public root, which is harmless.
# Writes nothing and returns quietly when detection is not possible.
function Set-CorpRootCa {
param(
[Parameter(Mandatory)][string]$RepoRoot
)
$certPath = Join-Path $RepoRoot 'corp-root-ca.pem'
try {
$request = [System.Net.HttpWebRequest]::Create('https://registry.npmjs.org/')
$request.Method = 'HEAD'
$request.Timeout = 10000
try { $response = $request.GetResponse(); $response.Close() } catch { }
$cert = $request.ServicePoint.Certificate
if (-not $cert) {
Write-Warn 'Could not read the TLS certificate chain; skipping corp CA staging.'
return
}
$chain = New-Object System.Security.Cryptography.X509Certificates.X509Chain
$chain.ChainPolicy.RevocationMode = 'NoCheck'
$cert2 = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($cert)
[void]$chain.Build($cert2)
$root = $chain.ChainElements[$chain.ChainElements.Count - 1].Certificate
$issuer = $root.Subject
$builder = New-Object System.Text.StringBuilder
[void]$builder.AppendLine('-----BEGIN CERTIFICATE-----')
[void]$builder.AppendLine([System.Convert]::ToBase64String($root.RawData, 'InsertLineBreaks'))
[void]$builder.AppendLine('-----END CERTIFICATE-----')
Set-Content -Path $certPath -Value $builder.ToString() -Encoding ASCII
Write-Ok "Staged corp-root-ca.pem from root CA: $issuer"
}
catch {
Write-Warn "Corp CA detection failed ($($_.Exception.Message)); continuing without it."
}
}
# Confirm a dependency, offering a winget install when it is missing. Returns
# $true when the dependency is present (or was just installed), $false when the
# dev declined or install failed.
function Confirm-Dependency {
param(
[Parameter(Mandatory)][string]$Command,
[Parameter(Mandatory)][string]$DisplayName,
[Parameter(Mandatory)][string]$WingetId,
[string]$PostInstallNote
)
if (Test-Command $Command) {
Write-Ok "$DisplayName found."
return $true
}
Write-Warn "$DisplayName not found (command '$Command')."
if ($SkipDependencyInstall) {
Write-Warn "Skipping install (-SkipDependencyInstall). Install '$WingetId' manually."
return $false
}
if (-not (Test-Command 'winget')) {
Write-Warn "winget is unavailable, so $DisplayName cannot be auto-installed."
Write-Warn "Install it from the Microsoft Store / App Installer, then re-run."
return $false
}
$answer = Read-Host " Install $DisplayName now via winget ($WingetId)? [y/N]"
if ($answer -notmatch '^(y|yes)$') {
Write-Warn "Skipped $DisplayName. It is required; re-run once it is installed."
return $false
}
Write-Host " Running: winget install --id $WingetId" -ForegroundColor DarkGray
winget install --id $WingetId --exact --accept-source-agreements --accept-package-agreements
if ($LASTEXITCODE -ne 0) {
Write-Warn "winget exited $LASTEXITCODE installing $DisplayName."
return $false
}
if ($PostInstallNote) { Write-Warn $PostInstallNote }
# winget updates the machine PATH, but this session will not see it until
# relaunch, so re-probe rather than claim success outright.
return (Test-Command $Command)
}
# Docker being installed is not the same as the engine running. A stopped
# Docker Desktop passes the command check but fails every compose call.
function Test-DockerEngine {
try {
docker info --format '{{.ServerVersion}}' 2>$null | Out-Null
return ($LASTEXITCODE -eq 0)
} catch {
return $false
}
}
# --- Resolve paths and channel -------------------------------------------------
$repoRoot = Split-Path -Parent $PSScriptRoot
$config = $channels[$Channel]
$branch = if ($Ref) { $Ref } else { $config.Ref }
Write-Host "ShopDB dev bootstrap" -ForegroundColor White
Write-Host " Channel : $Channel" -ForegroundColor White
Write-Host " Branch : $branch" -ForegroundColor White
Write-Host " Project : $($config.Project)" -ForegroundColor White
Write-Host " Repo : $repoRoot" -ForegroundColor White
# --- 1. Dependencies -----------------------------------------------------------
Write-Step 'Checking developer dependencies'
$allPresent = $true
$allPresent = (Confirm-Dependency -Command 'git' -DisplayName 'Git' -WingetId 'Git.Git') -and $allPresent
$allPresent = (Confirm-Dependency -Command 'docker' -DisplayName 'Docker Desktop' -WingetId 'Docker.DockerDesktop' `
-PostInstallNote 'Docker Desktop needs a sign-out/restart and must be started once before use.') -and $allPresent
if ($CheckPython) {
$allPresent = (Confirm-Dependency -Command 'python' -DisplayName 'Python 3.11+' -WingetId 'Python.Python.3.11') -and $allPresent
}
if (-not $allPresent) {
throw 'One or more required dependencies are missing. Resolve the warnings above and re-run.'
}
Write-Step 'Checking the Docker engine is running'
if (-not (Test-DockerEngine)) {
throw 'Docker is installed but the engine is not responding. Start Docker Desktop and re-run.'
}
Write-Ok 'Docker engine is responding.'
# --- 2. Check out the channel branch ------------------------------------------
Write-Step "Preparing the '$branch' branch"
Push-Location $repoRoot
try {
$dirty = git status --porcelain
if ($dirty) {
Write-Warn 'Working tree has uncommitted changes; leaving the current branch as-is.'
Write-Warn "Building from the checked-out tree instead of switching to '$branch'."
} else {
git fetch origin $branch
git checkout $branch
git pull --ff-only origin $branch
Write-Ok "On '$branch', up to date with origin."
}
# --- 3. Channel .env ------------------------------------------------------
Write-Step 'Ensuring a channel .env exists'
$envPath = Join-Path $repoRoot ".env.$($config.Project)"
if (Test-Path $envPath) {
Write-Ok "Reusing existing $([System.IO.Path]::GetFileName($envPath))."
# A .env written before the pull-based flow existed will not carry a
# SHOPDB_IMAGE line, and the pull compose file requires it. Top it up in
# place rather than force the pilot to delete and re-key the file.
$image = "$imageRepo`:$($config.ImageTag)"
$envText = Get-Content -Path $envPath
if (-not ($envText | Where-Object { $_ -match '^SHOPDB_IMAGE=' })) {
Add-Content -Path $envPath -Value "SHOPDB_IMAGE=$image" -Encoding ASCII
Write-Ok "Added SHOPDB_IMAGE=$image to the existing .env."
}
} else {
# Dev-grade secrets: generated locally, never for production. A real
# deploy fills .env from the wiki runbook, not from this script.
$secret = [Convert]::ToBase64String((1..48 | ForEach-Object { Get-Random -Maximum 256 }))
$jwt = [Convert]::ToBase64String((1..48 | ForEach-Object { Get-Random -Maximum 256 }))
$dbPass = [Convert]::ToBase64String((1..18 | ForEach-Object { Get-Random -Maximum 256 })) -replace '[^A-Za-z0-9]', 'x'
$rootPass = [Convert]::ToBase64String((1..18 | ForEach-Object { Get-Random -Maximum 256 })) -replace '[^A-Za-z0-9]', 'x'
$apiPort = $config.ApiPort
$mysqlPort = $config.MysqlPort
$image = "$imageRepo`:$($config.ImageTag)"
$lines = @(
"# Auto-generated by scripts/dev-bootstrap.ps1 for the $Channel channel.",
"# DEV-GRADE ONLY. Do not reuse these secrets for a production site.",
"FLASK_APP=wsgi.py",
"FLASK_ENV=production",
"SECRET_KEY=$secret",
"JWT_SECRET_KEY=$jwt",
"DATABASE_URL=mysql+pymysql://shopdb:$dbPass@db:3306/shopdb_flask?charset=utf8mb4",
"CORS_ORIGINS=https://localhost:$apiPort",
"LOG_LEVEL=INFO",
"MYSQL_ROOT_PASSWORD=$rootPass",
"MYSQL_PASSWORD=$dbPass",
"MYSQL_PORT=$mysqlPort",
"API_PORT=$apiPort",
"SHOPDB_IMAGE=$image",
"SHOPDB_ENV_LABEL=$($config.EnvLabel)"
)
Set-Content -Path $envPath -Value $lines -Encoding ASCII
Write-Ok "Wrote $([System.IO.Path]::GetFileName($envPath)) (API on $apiPort, MySQL on $mysqlPort)."
}
# --- 4. Pull and start ----------------------------------------------------
# No local build: the image is produced by CI and published to GHCR (see
# .github/workflows/publish-images.yml). Pilots may have only Docker
# installed, and building the frontend from source behind an inspecting
# proxy is unreliable, so we pull the finished channel image instead.
Write-Step "Pulling and starting the '$($config.Project)' stack"
$composeArgs = @(
'compose',
'-p', $config.Project,
'--env-file', $envPath,
'-f', 'docker-compose.pull.yml'
)
& docker @composeArgs pull
if ($LASTEXITCODE -ne 0) { throw "docker compose pull failed ($LASTEXITCODE). If this is an auth error, run: docker login ghcr.io" }
& docker @composeArgs up -d
if ($LASTEXITCODE -ne 0) { throw "docker compose up failed ($LASTEXITCODE)." }
Write-Step 'Done'
Write-Ok "The $Channel stack is up."
Write-Host " UI/API : https://localhost:$($config.ApiPort)" -ForegroundColor White
if ($config.EnvLabel) {
Write-Host " Badge : sidebar shows '$($config.EnvLabel)' (edge channel)." -ForegroundColor White
}
Write-Host " Logs : docker compose -p $($config.Project) logs -f api" -ForegroundColor White
Write-Host " Stop : docker compose -p $($config.Project) down" -ForegroundColor White
} finally {
Pop-Location
}

335
scripts/sim_policy_monitor_e2e.py Normal file
View File

@@ -0,0 +1,335 @@
"""Live end-to-end simulation for the policy-driven monitor workflow.
Drives the REAL HTTP routes through the Flask app against the LIVE database
(the same one the container serves), exercising the exact flow a user performs
in the browser:
1. Log in as an admin (mint a real JWT).
2. Confirm the monitor catalog lists the four shipped monitors.
3. Create a local process monitor (msedge.exe) via POST /api/endpoints/monitors.
4. Create a Configuration Policy scoped to a PC type, carrying monitor:<id>.
5. Wrap it in an active Business Policy.
6. Create a Computer of that PC type.
7. GET the services panel and assert the monitor surfaces as an Unknown row.
8. Negative + validation probes.
9. Clean up everything created (leaves the DB as found).
Run inside the container:
docker compose exec -T api python scripts/sim_policy_monitor_e2e.py
Exit code 0 = all simulations passed. Non-zero = a failure (printed).
"""
import sys
import uuid
from pathlib import Path
from dotenv import load_dotenv
load_dotenv(Path(__file__).resolve().parent.parent / '.env')
from shopdb import create_app
from shopdb.extensions import db
from shopdb.core.models import User, Role, Asset, AssetType, Setting
from flask_jwt_extended import create_access_token
PASS = "PASS"
FAIL = "FAIL"
class Sim:
def __init__(self):
self.results = []
self.created = {
"monitor_ids": [],
"configpolicy_ids": [],
"businesspolicy_ids": [],
"asset_ids": [],
"computer_ids": [],
"computertype_ids": [],
"setting_created": False,
}
def check(self, name, condition, detail=""):
status = PASS if condition else FAIL
self.results.append((status, name, detail))
marker = "[ OK ]" if condition else "[FAIL]"
line = f"{marker} {name}"
if detail:
line += f" -- {detail}"
print(line, flush=True)
return condition
def summary(self):
passed = sum(1 for r in self.results if r[0] == PASS)
failed = sum(1 for r in self.results if r[0] == FAIL)
print("\n" + "=" * 60, flush=True)
print(f"SIMULATION SUMMARY: {passed} passed, {failed} failed", flush=True)
print("=" * 60, flush=True)
return failed == 0
def admin_headers(app):
"""Mint a real admin JWT bound to an existing admin user, or a temp one."""
with app.app_context():
user = (
User.query.join(User.roles)
.filter(Role.rolename == "admin")
.first()
)
if user is None:
role = Role.query.filter_by(rolename="admin").first()
if role is None:
role = Role(rolename="admin", description="Administrator")
db.session.add(role)
db.session.flush()
user = User(
username=f"sim_admin_{uuid.uuid4().hex[:8]}",
email=f"sim_{uuid.uuid4().hex[:8]}@test.local",
isactive=True,
)
if hasattr(user, "set_password"):
user.set_password("simpass123!")
user.roles.append(role)
db.session.add(user)
db.session.commit()
token = create_access_token(
identity=str(user.userid),
additional_claims={
"username": user.username,
"roles": [r.rolename for r in user.roles],
},
)
return {"Authorization": f"Bearer {token}"}
def jdata(resp):
body = resp.get_json()
return (body or {}).get("data")
def run():
app = create_app()
headers = admin_headers(app)
sim = Sim()
client = app.test_client()
tag = uuid.uuid4().hex[:8]
pctype_name = f"SimPCType_{tag}"
# --- 1. Monitor catalog lists the shipped four ---------------------------
resp = client.get("/api/endpoints/monitors", headers=headers)
sim.check("GET /monitors returns 200", resp.status_code == 200,
f"status={resp.status_code}")
catalog = jdata(resp) or []
shipped = [m for m in catalog if m.get("source") == "shipped"]
shipped_names = sorted({m["displayname"] for m in shipped})
sim.check("Shipped monitors seeded (eDNC, UDC, MTConnect x2)",
len(shipped) >= 4,
f"shipped={len(shipped)} names={shipped_names}")
# --- 2. Create a local process monitor (msedge.exe) ----------------------
display = f"sim_msedge_{tag}.exe"
resp = client.post("/api/endpoints/monitors", headers=headers, json={
"kind": "process",
"displayname": display,
"processname": display,
"isactive": True,
})
sim.check("POST /monitors (process) returns 201",
resp.status_code == 201, f"status={resp.status_code}")
monitor = jdata(resp) or {}
monitor_id = monitor.get("endpointmonitorid")
if monitor_id:
sim.created["monitor_ids"].append(monitor_id)
sim.check("Created monitor has an id", bool(monitor_id),
f"id={monitor_id}")
# --- 2b. Validation probe: missing name is rejected ----------------------
resp = client.post("/api/endpoints/monitors", headers=headers, json={
"kind": "process", "displayname": "", "processname": "",
})
sim.check("POST /monitors with blank name rejected (400)",
resp.status_code == 400, f"status={resp.status_code}")
# --- 3. Create a PC type to scope to -------------------------------------
resp = client.post("/api/computers/types", headers=headers,
json={"computertype": pctype_name})
sim.check("POST /computers/types returns 2xx",
resp.status_code in (200, 201), f"status={resp.status_code}")
pctype = jdata(resp) or {}
pctype_id = pctype.get("computertypeid")
if pctype_id:
sim.created["computertype_ids"].append(pctype_id)
sim.check("PC type created with id", bool(pctype_id), f"id={pctype_id}")
# --- 4. Enable the feature flag ------------------------------------------
with app.app_context():
flag = Setting.query.filter_by(
key="computers_configpolicies_enabled").first()
if flag is None:
db.session.add(Setting(
key="computers_configpolicies_enabled", value="true"))
sim.created["setting_created"] = True
else:
sim._prev_flag = flag.value
flag.value = "true"
db.session.commit()
sim.check("Feature flag computers_configpolicies_enabled=true", True)
# --- 5. Create a Configuration Policy carrying monitor:<id> ---------------
resp = client.post("/api/computers/policies/configuration", headers=headers,
json={
"name": f"Sim Edge Coverage {tag}",
"description": "sim",
"scopejson": {"type": "pctype", "id": pctype_id},
"actions": [
{"action": f"monitor:{monitor_id}",
"mode": "gate"},
],
})
sim.check("POST /policies/configuration returns 2xx",
resp.status_code in (200, 201), f"status={resp.status_code}")
cpolicy = jdata(resp) or {}
cpolicy_id = cpolicy.get("configurationpolicyid")
if cpolicy_id:
sim.created["configpolicy_ids"].append(cpolicy_id)
sim.check("Configuration Policy created with id", bool(cpolicy_id),
f"id={cpolicy_id}")
# --- 5b. Round-trip: the monitor action persisted ------------------------
resp = client.get(
f"/api/computers/policies/configuration/{cpolicy_id}", headers=headers)
body = jdata(resp) or {}
actions = body.get("actions", [])
sim.check("Config policy round-trips the monitor action",
any(a.get("action") == f"monitor:{monitor_id}" for a in actions),
f"actions={[a.get('action') for a in actions]}")
# --- 6. Wrap it in an active Business Policy ------------------------------
resp = client.post("/api/computers/policies/business", headers=headers,
json={
"name": f"Sim Process Monitor Test {tag}",
"priority": 100,
"isactive": True,
"items": [
{"configurationpolicyid": cpolicy_id,
"sortorder": 0},
],
})
sim.check("POST /policies/business returns 2xx",
resp.status_code in (200, 201), f"status={resp.status_code}")
bpolicy = jdata(resp) or {}
bpolicy_id = bpolicy.get("businesspolicyid")
if bpolicy_id:
sim.created["businesspolicy_ids"].append(bpolicy_id)
sim.check("Business Policy created with id", bool(bpolicy_id),
f"id={bpolicy_id}")
# --- 7. Create a Computer of that PC type --------------------------------
hostname = f"SIMPC{tag.upper()}"
with app.app_context():
from plugins.computers.models import Computer
atype = AssetType.query.filter_by(assettype="computer").first()
if atype is None:
atype = AssetType(assettype="computer")
db.session.add(atype)
db.session.flush()
asset = Asset(assetnumber=hostname, assettypeid=atype.assettypeid,
isactive=True)
db.session.add(asset)
db.session.flush()
comp = Computer(assetid=asset.assetid, hostname=hostname,
computertypeid=pctype_id)
db.session.add(comp)
db.session.commit()
asset_id = asset.assetid
computer_id = comp.computerid
sim.created["asset_ids"].append(asset_id)
sim.created["computer_ids"].append(computer_id)
sim.check("Computer created for the scoped PC type", bool(asset_id),
f"assetid={asset_id}")
# --- 8. The monitor surfaces in the services panel -----------------------
resp = client.get(f"/api/endpoints/asset/{asset_id}/services",
headers=headers)
sim.check("GET /asset/<id>/services returns 200",
resp.status_code == 200, f"status={resp.status_code}")
rows = jdata(resp) or []
edge = next((r for r in rows if r.get("name") == display), None)
sim.check("Policy-attached monitor surfaces in panel", edge is not None,
f"rows={[r.get('name') for r in rows]}")
if edge:
sim.check("Surfaced row is a process", edge.get("kind") == "process",
f"kind={edge.get('kind')}")
sim.check("Surfaced row is Unknown (not yet reported)",
edge.get("statelabel") == "Unknown",
f"statelabel={edge.get('statelabel')}")
action = edge.get("action") or {}
sim.check("Surfaced row offers a Start action targeting the process",
action.get("target") == display
and action.get("targetkind") == "process",
f"action={action}")
# --- 9. Negative: flag off hides the row ---------------------------------
with app.app_context():
flag = Setting.query.filter_by(
key="computers_configpolicies_enabled").first()
flag.value = "false"
db.session.commit()
resp = client.get(f"/api/endpoints/asset/{asset_id}/services",
headers=headers)
rows = jdata(resp) or []
sim.check("Flag off hides the policy monitor from the panel",
not any(r.get("name") == display for r in rows),
f"rows={[r.get('name') for r in rows]}")
with app.app_context():
flag = Setting.query.filter_by(
key="computers_configpolicies_enabled").first()
flag.value = "true"
db.session.commit()
# --- Cleanup -------------------------------------------------------------
cleanup(app, client, headers, sim)
return sim.summary()
def cleanup(app, client, headers, sim):
print("\n--- cleanup ---", flush=True)
for bid in sim.created["businesspolicy_ids"]:
client.delete(f"/api/computers/policies/business/{bid}", headers=headers)
for cid in sim.created["configpolicy_ids"]:
client.delete(f"/api/computers/policies/configuration/{cid}",
headers=headers)
for mid in sim.created["monitor_ids"]:
client.delete(f"/api/endpoints/monitors/{mid}", headers=headers)
with app.app_context():
from plugins.computers.models import Computer, ComputerType
for cid in sim.created["computer_ids"]:
row = db.session.get(Computer, cid)
if row:
db.session.delete(row)
for aid in sim.created["asset_ids"]:
row = db.session.get(Asset, aid)
if row:
db.session.delete(row)
for tid in sim.created["computertype_ids"]:
row = db.session.get(ComputerType, tid)
if row:
db.session.delete(row)
# Restore or remove the feature flag as found.
flag = Setting.query.filter_by(
key="computers_configpolicies_enabled").first()
if flag is not None:
if sim.created["setting_created"]:
db.session.delete(flag)
elif hasattr(sim, "_prev_flag"):
flag.value = sim._prev_flag
db.session.commit()
print("cleanup done", flush=True)
if __name__ == "__main__":
ok = run()
sys.exit(0 if ok else 1)

9
scripts/uninstall-shopdbagent.ps1
View File

@@ -114,6 +114,15 @@ Start-Sleep -Seconds 1
Write-Host "[3/6] Removing registry entries ..."
Remove-HklmBothViews -SubKey "SOFTWARE\Microsoft\Windows\CurrentVersion\Run" -Name "ShopDbEndpointTray"
Remove-HklmBothViews -SubKey "SOFTWARE\GE\ShopDB"
# Remove the All-Users Start Menu shortcut the site-agent installer created.
$commonPrograms = [Environment]::GetFolderPath([Environment+SpecialFolder]::CommonPrograms)
$shortcutPath = Join-Path $commonPrograms "ShopDB Agent.lnk"
if (Test-Path $shortcutPath) {
try { Remove-Item -Path $shortcutPath -Force -ErrorAction Stop; Write-Host " removed Start Menu shortcut $shortcutPath" }
catch { Write-Warning " could not remove Start Menu shortcut: $($_.Exception.Message)" }
}
# Drop the parent GE key only if we emptied it (some sites nest siblings under
# GE such as SFLD; never tree-delete a key we do not own).
foreach ($view in @([Microsoft.Win32.RegistryView]::Registry64,

48
shopdb/core/api/settings.py
View File

@@ -64,6 +64,40 @@ PUBLIC_SETTING_KEYS = {
'site_timezone',
}
# Environment channel label. Sourced from the SHOPDB_ENV_LABEL environment
# variable (set by the edge docker-compose override and the bleeding-edge
# installer, unset on stable/main), NOT from a settings row. It is injected as a
# synthetic public item into the settings list so the frontend picks it up
# through the same read-through it already uses for branding, with no extra
# request. Empty when unset, which is how a stable deployment renders no DEV
# badge. Public and read-only: it carries no secret and cannot be edited via the
# settings API, only by redeploying with a different environment value.
ENV_LABEL_KEY = 'env_label'
def _env_label() -> str:
"""The configured environment channel label (e.g. 'DEV'), or '' when unset."""
return (os.environ.get('SHOPDB_ENV_LABEL') or '').strip()
def _env_label_item() -> dict:
"""Synthetic, serialized settings item carrying the environment label.
Shaped like a real serialized Setting so the frontend read-through treats it
as just another public key. settingid is None because it is not a DB row.
"""
return {
'settingid': None,
'key': ENV_LABEL_KEY,
'value': _env_label(),
'valuetype': 'string',
'category': 'branding',
'description': 'Environment channel label (from SHOPDB_ENV_LABEL); '
'drives the DEV badge in the sidebar. Read-only.',
'createddate': None,
'modifieddate': None,
}
def _plugin_declared_settings() -> dict:
"""Declared settings ({key: entry}) of every loaded plugin, or {}."""
@@ -576,7 +610,19 @@ def list_settings():
# bootstrap keys); an authed principal sees everything (secrets masked).
if get_jwt_identity() is None:
settings = [s for s in settings if _is_public_setting(s)]
return success_response([_serialize_setting(s) for s in settings])
items = [_serialize_setting(s) for s in settings]
# Inject the environment channel label as a synthetic public item so the
# frontend read-through surfaces it alongside branding. It is env-sourced,
# not a DB row, and is always present (empty string when unset) so the
# public-settings contract can rely on the key existing. Drop any DB row that
# somehow shares the key so the env value is authoritative and appears once.
# Skip it when a category filter is active and does not select branding, so a
# scoped query is not polluted.
if not category or category == 'branding':
items = [i for i in items if i.get('key') != ENV_LABEL_KEY]
items.append(_env_label_item())
return success_response(items)
@settings_bp.route('/<key>', methods=['GET'])

2
shopdb/plugins/alembic_template.py
View File

@@ -53,7 +53,7 @@ PLUGIN_TABLE_OWNERS: dict[str, Iterable[str]] = {
'configurationpolicies', 'configurationpolicyactions',
'businesspolicies', 'businesspolicyitems'),
'employees': ('directoryemployees',),
'endpoints': ('endpointstates', 'endpointevents', 'endpointcommands', 'endpointapplications'),
'endpoints': ('endpointstates', 'endpointevents', 'endpointcommands', 'endpointapplications', 'endpointmonitors', 'operatorreports'),
'geenforce': ('manifestscopes', 'manifestentries', 'manifestentrypctypes',
'manifestentryhostnames', 'manifestentrymachinenumbers',
'manifestinusechecks', 'manifestinusecheckprocesses',

10
tests/test_core/test_authz.py
View File

@@ -98,7 +98,15 @@ EXEMPT_ENDPOINTS = {'knowledgebase.track_click', 'users.update_user',
# kill switch. Enqueue (WebUI -> operator) IS permission-gated
# on endpoints.command and IS swept. Contract in
# plugins/endpoints/AGENT-CONTRACT.md section 7.
'endpoints.report_command_result'}
'endpoints.report_command_result',
# Agent-facing operator issue report upload. Authenticated by
# the endpoints collector API key (same shape as the exempt
# collector blueprint), not JWT+role: the tray hands the
# screenshot and note to the service, which uploads on the
# collector key it already holds. The human-facing read and
# image-serve routes ARE permission-gated on endpoints.view
# and ARE swept.
'endpoints.upload_operator_report'}
@pytest.fixture(autouse=True)

8
tests/test_core/test_public_read_surface.py
View File

@@ -96,6 +96,14 @@ PUBLIC_ENDPOINTS = {
'endpoints.download_app_package',
'endpoints.download_app_package_by_app',
'endpoints.download_script_body',
# The endpoint agent fetches its server-resolved monitor watch-list (the
# processes/services site policy attaches to its PC type) on each telemetry
# cycle. Authenticated inside the handler by the endpoints collector API key
# - the same shape as claim_next_command - because a bay has no user session.
# It is a read of already-scoped policy output, not an authority: attaching a
# monitor is gated on the policy editor, audited with the requesting user.
'endpoints.resolved_monitors_for_agent',
}
# Reads that need a LOGIN but no named permission, because the handler decides

9
tests/test_plugin_migrations.py
View File

@@ -92,9 +92,12 @@ EXPECTED_HEAD_REVISION['warranty'] = 'warranty0002proof'
# endpoints is a NEW post-cutover plugin: its 0001 genuinely creates the
# endpointstates + endpointevents tables the core chain never owned, 0002
# adds the endpointcommands queue plus the MTConnect state columns, 0003
# adds the generic command target/targetkind columns, and 0004 adds the
# endpointapplications SBOM snapshot table.
EXPECTED_HEAD_REVISION['endpoints'] = 'endpoints0005cmdpin'
# adds the generic command target/targetkind columns, 0004 adds the
# endpointapplications SBOM snapshot table, 0005 pins a command sha256, 0006
# seeds the reusable monitor catalog, 0007 ships the shopfloor monitor policy
# bundle, 0008 adds the warn-only reconciler drift cache column, and 0009 adds
# the operatorreports table for tray issue reports.
EXPECTED_HEAD_REVISION['endpoints'] = 'endpoints0009opreports'
# Plugins built after the cutover: their 0001 baseline really creates tables the
# core chain never owned.

285
tests/test_plugins/test_endpoints_operator_reports.py Normal file
View File

@@ -0,0 +1,285 @@
"""Operator issue report upload, retention purge, and hardened serve.
The tray captures a screenshot and a free-text note; the agent posts them to
POST /api/endpoints/operatorreports on the collector key. The bytes land on
disk under the operator-reports uploads dir and the row holds metadata. The
human-facing panel and image-serve routes are JWT + endpoints.view gated, and
the serve route carries the inert-upload headers. A retention sweep purges rows
and files older than the configured window.
"""
import io
import os
from datetime import timedelta
from shopdb.core.models import Asset, AssetType, Setting
UPLOAD_URL = '/api/endpoints/operatorreports'
PANEL_URL = '/api/endpoints/panel/{}/operatorreports'
IMAGE_URL = '/api/endpoints/operatorreports/{}/image'
COLLECTOR_KEY = 'testcollectorkey'
def _collector_headers(app):
app.config['COLLECTOR_API_KEY'] = COLLECTOR_KEY
return {'X-API-Key': COLLECTOR_KEY}
def _computer(db, hostname):
"""Create a computer Asset + Computer; return the asset."""
from plugins.computers.models import Computer, ComputerType
atype = AssetType.query.filter_by(assettype='computer').first()
if not atype:
atype = AssetType(assettype='computer')
db.session.add(atype)
db.session.flush()
ctype = ComputerType.query.filter_by(computertype='CMM PC').first()
if not ctype:
ctype = ComputerType(computertype='CMM PC')
db.session.add(ctype)
db.session.flush()
asset = Asset(assetnumber=hostname, assettypeid=atype.assettypeid,
isactive=True)
db.session.add(asset)
db.session.flush()
comp = Computer(assetid=asset.assetid, hostname=hostname,
computertypeid=ctype.computertypeid)
db.session.add(comp)
db.session.commit()
return asset
def _png_bytes():
"""A tiny valid 1x1 PNG."""
import base64
return base64.b64decode(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk'
'YPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==')
def test_upload_creates_report_and_stores_screenshot(client, db, app):
"""A collector-key upload with a screenshot creates a row and a file."""
from plugins.endpoints.models import OperatorReport
asset = _computer(db, 'OPREPORT1')
data = {
'hostname': 'OPREPORT1',
'reason': 'Other (free text)',
'details': 'the CMM app will not start',
'screenshot': (io.BytesIO(_png_bytes()), 'shot.png'),
}
response = client.post(UPLOAD_URL, data=data,
content_type='multipart/form-data',
headers=_collector_headers(app))
assert response.status_code == 201
body = response.get_json()['data']
assert body['assetid'] == asset.assetid
assert body['details'] == 'the CMM app will not start'
assert body['hasimage'] is True
report = OperatorReport.query.filter_by(assetid=asset.assetid).first()
assert report is not None
assert report.imagefilename
stored = os.path.join(app.instance_path, 'operatorreports',
report.imagefilename)
assert os.path.isfile(stored)
def test_upload_without_screenshot_is_allowed(client, db, app):
"""A note-only report (no screenshot) is valid; hasimage is false."""
_computer(db, 'OPREPORT2')
response = client.post(
UPLOAD_URL,
data={'hostname': 'OPREPORT2', 'details': 'noisy fan'},
content_type='multipart/form-data',
headers=_collector_headers(app))
assert response.status_code == 201
assert response.get_json()['data']['hasimage'] is False
def test_upload_unknown_hostname_is_validation_error(client, db, app):
"""A hostname with no computer row is rejected, not silently stored."""
response = client.post(
UPLOAD_URL,
data={'hostname': 'NOSUCHBOX', 'details': 'x'},
content_type='multipart/form-data',
headers=_collector_headers(app))
assert response.status_code == 400
def test_upload_requires_collector_auth(client, db, app):
"""Without the collector key the upload refuses (no JWT fallback)."""
_computer(db, 'OPREPORT3')
app.config['COLLECTOR_API_KEY'] = COLLECTOR_KEY
response = client.post(
UPLOAD_URL,
data={'hostname': 'OPREPORT3', 'details': 'x'},
content_type='multipart/form-data')
assert response.status_code in (401, 403)
def test_panel_lists_reports_newest_first(client, db, app, auth_headers):
"""The panel route returns this asset's reports, newest first."""
asset = _computer(db, 'OPREPORT4')
for note in ('first', 'second'):
client.post(UPLOAD_URL,
data={'hostname': 'OPREPORT4', 'details': note},
content_type='multipart/form-data',
headers=_collector_headers(app))
response = client.get(PANEL_URL.format(asset.assetid), headers=auth_headers)
assert response.status_code == 200
reports = response.get_json()['data']['reports']
assert [r['details'] for r in reports] == ['second', 'first']
def test_served_image_is_hardened(client, db, app, auth_headers):
"""The screenshot serve route carries the inert-upload headers."""
from shopdb.utils.uploads import UPLOAD_CSP
asset = _computer(db, 'OPREPORT5')
client.post(UPLOAD_URL,
data={'hostname': 'OPREPORT5', 'details': 'x',
'screenshot': (io.BytesIO(_png_bytes()), 'shot.png')},
content_type='multipart/form-data',
headers=_collector_headers(app))
from plugins.endpoints.models import OperatorReport
report = OperatorReport.query.filter_by(assetid=asset.assetid).first()
response = client.get(IMAGE_URL.format(report.operatorreportid),
headers=auth_headers)
assert response.status_code == 200
assert response.headers.get('Content-Security-Policy') == UPLOAD_CSP
assert response.headers.get('X-Content-Type-Options') == 'nosniff'
def test_image_serve_requires_jwt(client, db, app):
"""The screenshot serve route is not reachable without a user JWT."""
asset = _computer(db, 'OPREPORT6')
client.post(UPLOAD_URL,
data={'hostname': 'OPREPORT6', 'details': 'x',
'screenshot': (io.BytesIO(_png_bytes()), 'shot.png')},
content_type='multipart/form-data',
headers=_collector_headers(app))
from plugins.endpoints.models import OperatorReport
report = OperatorReport.query.filter_by(assetid=asset.assetid).first()
response = client.get(IMAGE_URL.format(report.operatorreportid))
assert response.status_code in (401, 403)
def test_retention_purge_removes_expired_reports_and_files(client, db, app):
"""A report past the retention window is purged, file and all, on upload."""
from plugins.endpoints.models import OperatorReport
from plugins.endpoints.api.routes import (
OPERATOR_REPORT_RETENTION_SETTING,
)
db.session.add(Setting(key=OPERATOR_REPORT_RETENTION_SETTING, value='7'))
db.session.commit()
asset = _computer(db, 'OPREPORT7')
client.post(UPLOAD_URL,
data={'hostname': 'OPREPORT7', 'details': 'old',
'screenshot': (io.BytesIO(_png_bytes()), 'shot.png')},
content_type='multipart/form-data',
headers=_collector_headers(app))
old = OperatorReport.query.filter_by(assetid=asset.assetid).first()
old_file = os.path.join(app.instance_path, 'operatorreports',
old.imagefilename)
assert os.path.isfile(old_file)
# Backdate it past the window, then upload a fresh report to trigger the
# opportunistic sweep.
old.createdat = old.createdat - timedelta(days=8)
db.session.commit()
client.post(UPLOAD_URL,
data={'hostname': 'OPREPORT7', 'details': 'new'},
content_type='multipart/form-data',
headers=_collector_headers(app))
assert db.session.get(OperatorReport, old.operatorreportid) is None
assert not os.path.isfile(old_file)
remaining = OperatorReport.query.filter_by(assetid=asset.assetid).all()
assert [r.details for r in remaining] == ['new']
def test_retention_disabled_keeps_old_reports(client, db, app):
"""A retention of 0 disables the sweep; old rows survive."""
from plugins.endpoints.models import OperatorReport
from plugins.endpoints.api.routes import (
OPERATOR_REPORT_RETENTION_SETTING,
)
db.session.add(Setting(key=OPERATOR_REPORT_RETENTION_SETTING, value='0'))
db.session.commit()
asset = _computer(db, 'OPREPORT8')
client.post(UPLOAD_URL,
data={'hostname': 'OPREPORT8', 'details': 'old'},
content_type='multipart/form-data',
headers=_collector_headers(app))
old = OperatorReport.query.filter_by(assetid=asset.assetid).first()
old.createdat = old.createdat - timedelta(days=365)
db.session.commit()
client.post(UPLOAD_URL,
data={'hostname': 'OPREPORT8', 'details': 'new'},
content_type='multipart/form-data',
headers=_collector_headers(app))
assert db.session.get(OperatorReport, old.operatorreportid) is not None
# --- what the route refuses to store -----------------------------------------
#
# The stored file is served back as image/png, and the size cap is the only
# thing bounding what an agent key can push into the instance directory. Both
# guards used to be nominal: the cap was checked AFTER upload.read() had already
# pulled the whole body into memory, and the extension was forced to .png
# whatever the bytes were.
def test_upload_refuses_bytes_that_are_not_a_png(client, db, app):
from plugins.endpoints.models import OperatorReport
_computer(db, 'OPREPORT-NOTPNG')
data = {
'hostname': 'OPREPORT-NOTPNG',
'details': 'not really a screenshot',
'screenshot': (io.BytesIO(b'MZ\x90\x00 this is a PE, not an image'),
'shot.png'),
}
response = client.post(UPLOAD_URL, data=data,
content_type='multipart/form-data',
headers=_collector_headers(app))
assert response.status_code == 400
# Nothing was written: not the row, and not a file under the uploads dir.
assert OperatorReport.query.filter_by(
reportedhostname='OPREPORT-NOTPNG').first() is None
def test_upload_refuses_an_image_over_the_cap(client, db, app):
from plugins.endpoints.api.routes import OPERATOR_REPORT_MAX_IMAGE_BYTES
from plugins.endpoints.models import OperatorReport
_computer(db, 'OPREPORT-HUGE')
# A real PNG header so the size check is what rejects it, not the signature.
oversized = _png_bytes() + b'\x00' * (OPERATOR_REPORT_MAX_IMAGE_BYTES + 1)
data = {
'hostname': 'OPREPORT-HUGE',
'details': 'oversized',
'screenshot': (io.BytesIO(oversized), 'shot.png'),
}
response = client.post(UPLOAD_URL, data=data,
content_type='multipart/form-data',
headers=_collector_headers(app))
assert response.status_code == 400
assert 'limit' in response.get_json()['data']['error']['message']
assert OperatorReport.query.filter_by(
reportedhostname='OPREPORT-HUGE').first() is None

31
tests/test_plugins/test_endpoints_plugin.py
View File

@@ -69,11 +69,12 @@ def test_endpoints_asset_panels_are_computer_scoped():
"""Every asset panel targets computer assets only, with an endpoint."""
plugin = EndpointsPlugin()
panels = plugin.get_asset_panels()
assert len(panels) == 6
assert len(panels) == 7
for panel in panels:
assert panel['assettypes'] == ['computer']
assert panel['endpoint'].startswith('/api/endpoints/asset/{assetid}')
assert panel['render'] in ('keyvalue', 'list', 'table')
assert panel['endpoint'].startswith('/api/endpoints/')
assert '{assetid}' in panel['endpoint']
assert panel['render'] in ('keyvalue', 'list', 'table', 'operatorreports')
def test_services_and_processes_are_split_into_two_filtered_cards():
@@ -234,7 +235,6 @@ def test_new_hardware_telemetry_keys_are_logged_not_silently_known():
assert 'applications' in KNOWN_PAYLOAD_KEYS
def test_telemetrymeta_route_is_registered_for_freshness_gating():
"""The WebUI gates panel refreshes on lastreportedat advancing, so the
lightweight freshness probe must be exposed on the plugin blueprint."""
@@ -249,3 +249,26 @@ def test_telemetrymeta_route_is_registered_for_freshness_gating():
url_prefix='/api/endpoints')
rules = {rule.rule for rule in application.url_map.iter_rules()}
assert any(rule.endswith('/asset/<int:asset_id>/telemetrymeta') for rule in rules)
def test_fleet_list_route_resolves_without_trailing_slash():
"""The Fleet list route must match the slash-less URL the SPA calls.
The list is the only route on the bare blueprint prefix. If it demands a
trailing slash, Flask 308-redirects /api/endpoints to /api/endpoints/, and
behind the /shopdb reverse proxy that redirect drops the path prefix and
the list call fails while the detail routes (which carry a path suffix)
keep working. strict_slashes=False makes both forms resolve directly.
"""
from flask import Flask
application = Flask(__name__)
application.register_blueprint(
EndpointsPlugin().get_blueprint(), url_prefix='/api/endpoints')
adapter = application.url_map.bind('localhost')
endpoint, _ = adapter.match('/api/endpoints', method='GET')
assert endpoint.endswith('list_endpoints')
endpoint_slash, _ = adapter.match('/api/endpoints/', method='GET')
assert endpoint_slash.endswith('list_endpoints')

199
tests/test_plugins/test_endpoints_policy_monitors.py Normal file
View File

@@ -0,0 +1,199 @@
"""Policy-attached monitors surface in the Tracked Services/Processes panel.
A monitor attached to a PC type through an active Business Policy (over a scoped
Configuration Policy) must appear in GET /api/endpoints/asset/<id>/services even
before the agent has reported that process/service, so the PC Detail Monitors
tab shows the intended coverage as an Unknown row.
"""
from shopdb.core.models import Asset, AssetType, Setting
SERVICES_URL = '/api/endpoints/asset/{}/services'
def _computer(db, hostname, computertype):
"""Create a computer Asset + Computer of the given PC type; return the asset."""
from plugins.computers.models import Computer, ComputerType
atype = AssetType.query.filter_by(assettype='computer').first()
if not atype:
atype = AssetType(assettype='computer')
db.session.add(atype)
db.session.flush()
ctype = ComputerType.query.filter_by(computertype=computertype).first()
if not ctype:
ctype = ComputerType(computertype=computertype)
db.session.add(ctype)
db.session.flush()
asset = Asset(assetnumber=hostname, assettypeid=atype.assettypeid,
isactive=True)
db.session.add(asset)
db.session.flush()
comp = Computer(assetid=asset.assetid, hostname=hostname,
computertypeid=ctype.computertypeid)
db.session.add(comp)
db.session.flush()
return asset, ctype
def _attach_monitor_to_pctype(db, monitor, computertypeid):
"""Create a Configuration Policy scoped to the PC type carrying the monitor,
wrapped in an active Business Policy, with the feature flag enabled."""
from plugins.computers.models import (
ConfigurationPolicy, ConfigurationPolicyAction,
BusinessPolicy, BusinessPolicyItem,
)
db.session.add(Setting(key='computers_configpolicies_enabled', value='true'))
policy = ConfigurationPolicy(
name='Edge coverage',
scopejson={'type': 'pctype', 'id': computertypeid},
)
db.session.add(policy)
db.session.flush()
db.session.add(ConfigurationPolicyAction(
configurationpolicyid=policy.configurationpolicyid,
action=f'monitor:{monitor.endpointmonitorid}',
mode='gate',
))
business = BusinessPolicy(name='Test Policy', priority=100)
db.session.add(business)
db.session.flush()
db.session.add(BusinessPolicyItem(
businesspolicyid=business.businesspolicyid,
configurationpolicyid=policy.configurationpolicyid,
sortorder=0,
))
db.session.commit()
def test_policy_attached_process_monitor_surfaces_as_unknown_row(
client, db, auth_headers):
"""A msedge.exe process monitor attached to a PC type by policy appears in
the services panel as an Unknown process row, even with no agent report."""
from plugins.endpoints.models import EndpointMonitor
asset, ctype = _computer(db, 'F18BCDB4', 'CMM PC')
monitor = EndpointMonitor(
kind='process', displayname='msedge.exe', processname='msedge.exe',
source='local', isactive=True,
)
db.session.add(monitor)
db.session.flush()
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
response = client.get(SERVICES_URL.format(asset.assetid), headers=auth_headers)
assert response.status_code == 200
rows = response.get_json()['data']
edge = next((r for r in rows if r['name'] == 'msedge.exe'), None)
assert edge is not None, 'policy-attached monitor missing from panel'
assert edge['kind'] == 'process'
assert edge['statelabel'] == 'Unknown'
assert edge['statecolor'] == 'gray'
# Admin holds endpoints.command, so the row offers a Start control whose
# wire target is the OS process name.
assert edge['action']['target'] == 'msedge.exe'
assert edge['action']['targetkind'] == 'process'
assert edge['action']['running'] is False
def test_flag_off_hides_policy_monitor_from_panel(client, db, auth_headers):
"""With the feature flag off the panel returns only reported rows (none)."""
from plugins.endpoints.models import EndpointMonitor
asset, ctype = _computer(db, 'F18BCDB5', 'CMM PC')
monitor = EndpointMonitor(
kind='process', displayname='msedge.exe', processname='msedge.exe',
source='local', isactive=True,
)
db.session.add(monitor)
db.session.flush()
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
# Flip the flag off after the helper turned it on.
flag = Setting.query.filter_by(
key='computers_configpolicies_enabled').first()
flag.value = 'false'
db.session.commit()
response = client.get(SERVICES_URL.format(asset.assetid), headers=auth_headers)
assert response.status_code == 200
rows = response.get_json()['data']
assert not any(r['name'] == 'msedge.exe' for r in rows)
RESOLVED_URL = '/api/endpoints/monitors/resolved'
COLLECTOR_KEY = 'testcollectorkey'
def _collector_headers(app):
"""Authorize the agent-facing route on the shared collector key."""
app.config['COLLECTOR_API_KEY'] = COLLECTOR_KEY
return {'X-API-Key': COLLECTOR_KEY}
def test_resolved_monitors_returns_policy_process_for_agent(client, db, app):
"""The agent watch-list route returns a policy-attached process monitor,
shaped as processes[]/services[] of {name, app}, keyed by hostname."""
from plugins.endpoints.models import EndpointMonitor
asset, ctype = _computer(db, 'AGENTPC1', 'CMM PC')
monitor = EndpointMonitor(
kind='process', displayname='Edge', processname='msedge.exe',
source='local', isactive=True,
)
db.session.add(monitor)
db.session.flush()
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
response = client.get(RESOLVED_URL, query_string={'hostname': 'AGENTPC1'},
headers=_collector_headers(app))
assert response.status_code == 200
data = response.get_json()['data']
assert {'name': 'msedge.exe', 'app': 'Edge'} in data['processes']
assert data['services'] == []
def test_resolved_monitors_carries_launchpath_for_process(client, db, app):
"""A process monitor with a launch path set surfaces launchpath on the wire
so the agent can honor a site-provisioned Start path (after validation)."""
from plugins.endpoints.models import EndpointMonitor
asset, ctype = _computer(db, 'AGENTPC3', 'CMM PC')
monitor = EndpointMonitor(
kind='process', displayname='Edge', processname='msedge.exe',
launchpath=r'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe',
source='local', isactive=True,
)
db.session.add(monitor)
db.session.flush()
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
response = client.get(RESOLVED_URL, query_string={'hostname': 'AGENTPC3'},
headers=_collector_headers(app))
assert response.status_code == 200
data = response.get_json()['data']
edge = next((p for p in data['processes'] if p['name'] == 'msedge.exe'), None)
assert edge is not None
assert edge['launchpath'] == \
r'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe'
def test_resolved_monitors_requires_collector_auth(client, db, app):
"""Without the collector key the agent route refuses (no JWT fallback)."""
_computer(db, 'AGENTPC2', 'CMM PC')
app.config['COLLECTOR_API_KEY'] = COLLECTOR_KEY
response = client.get(RESOLVED_URL, query_string={'hostname': 'AGENTPC2'})
assert response.status_code in (401, 403)
def test_resolved_monitors_unknown_hostname_is_404(client, db, app):
"""A hostname with no computer row is a 404, not an empty 200."""
response = client.get(RESOLVED_URL, query_string={'hostname': 'NOSUCHBOX'},
headers=_collector_headers(app))
assert response.status_code == 404

238
tests/test_plugins/test_endpoints_reconciler.py Normal file
View File

@@ -0,0 +1,238 @@
"""Warn-only reconciler: drift computation and its exposure on the panels.
The reconciler runs server-side on agent check-in. For every monitor a site
policy attaches to an endpoint's PC type that is EXPECTED to be running, it flags
DRIFT when the latest snapshot does not report it running. It never starts or
stops anything - drift is surfaced for an operator to act on manually.
These tests cover the pure computer (compute_drift), the check-in write path
(apply_collector_payload persisting driftjson), the panel exposure, and the
observe-only seam (monitor_expects_running) that keeps a non-keepalive monitor
out of the drift list.
"""
from shopdb.core.models import Asset, AssetType, Setting
SERVICES_URL = '/api/endpoints/asset/{}/services'
STATUS_URL = '/api/endpoints/asset/{}/status'
def _computer(db, hostname, computertype):
"""Create a computer Asset + Computer of the given PC type; return the pair."""
from plugins.computers.models import Computer, ComputerType
atype = AssetType.query.filter_by(assettype='computer').first()
if not atype:
atype = AssetType(assettype='computer')
db.session.add(atype)
db.session.flush()
ctype = ComputerType.query.filter_by(computertype=computertype).first()
if not ctype:
ctype = ComputerType(computertype=computertype)
db.session.add(ctype)
db.session.flush()
asset = Asset(assetnumber=hostname, assettypeid=atype.assettypeid,
isactive=True)
db.session.add(asset)
db.session.flush()
comp = Computer(assetid=asset.assetid, hostname=hostname,
computertypeid=ctype.computertypeid)
db.session.add(comp)
db.session.flush()
return asset, ctype
def _attach_monitor_to_pctype(db, monitor, computertypeid):
"""Attach a monitor to a PC type via an active Business/Configuration Policy."""
from plugins.computers.models import (
ConfigurationPolicy, ConfigurationPolicyAction,
BusinessPolicy, BusinessPolicyItem,
)
if not Setting.query.filter_by(
key='computers_configpolicies_enabled').first():
db.session.add(
Setting(key='computers_configpolicies_enabled', value='true'))
policy = ConfigurationPolicy(
name='Coverage {}'.format(monitor.endpointmonitorid),
scopejson={'type': 'pctype', 'id': computertypeid},
)
db.session.add(policy)
db.session.flush()
db.session.add(ConfigurationPolicyAction(
configurationpolicyid=policy.configurationpolicyid,
action='monitor:{}'.format(monitor.endpointmonitorid),
mode='gate',
))
business = BusinessPolicy(name='BP {}'.format(monitor.endpointmonitorid),
priority=100)
db.session.add(business)
db.session.flush()
db.session.add(BusinessPolicyItem(
businesspolicyid=business.businesspolicyid,
configurationpolicyid=policy.configurationpolicyid,
sortorder=0,
))
db.session.commit()
def _process_monitor(db, displayname, processname):
from plugins.endpoints.models import EndpointMonitor
monitor = EndpointMonitor(
kind='process', displayname=displayname, processname=processname,
source='local', isactive=True,
)
db.session.add(monitor)
db.session.flush()
return monitor
# ---------------------------------------------------------------------------
# Pure computer
# ---------------------------------------------------------------------------
def test_compute_drift_flags_attached_monitor_not_reported_running(
client, db, app):
"""An expected-running monitor absent from the snapshot is drift."""
from plugins.endpoints.services.reconciler import compute_drift
asset, ctype = _computer(db, 'DRIFT01', 'CMM PC')
monitor = _process_monitor(db, 'eDNC', 'dncMain.exe')
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
drift = compute_drift(asset.assetid, [])
assert len(drift) == 1
assert drift[0]['name'] == 'dncMain.exe'
assert drift[0]['kind'] == 'process'
assert drift[0]['expected'] == 'running'
assert drift[0]['observed'] == 'notrunning'
def test_compute_drift_empty_when_monitor_reported_running(client, db, app):
"""A monitor the snapshot reports running is not drift."""
from plugins.endpoints.services.reconciler import compute_drift
asset, ctype = _computer(db, 'DRIFT02', 'CMM PC')
monitor = _process_monitor(db, 'eDNC', 'dncMain.exe')
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
reported = [{'name': 'dncMain.exe', 'kind': 'process', 'state': 'running'}]
assert compute_drift(asset.assetid, reported) == []
def test_compute_drift_empty_when_no_monitor_attached(client, db, app):
"""No attached monitors means nothing to reconcile."""
from plugins.endpoints.services.reconciler import compute_drift
asset, _ = _computer(db, 'DRIFT03', 'CMM PC')
assert compute_drift(asset.assetid, []) == []
def test_monitor_expects_running_is_the_observe_only_seam():
"""The 'should be up' decision lives in one function; inactive is excluded."""
from plugins.endpoints.services.reconciler import monitor_expects_running
class _M:
isactive = True
active = _M()
inactive = _M()
inactive.isactive = False
assert monitor_expects_running(active) is True
assert monitor_expects_running(inactive) is False
# ---------------------------------------------------------------------------
# Check-in write path
# ---------------------------------------------------------------------------
def test_checkin_persists_drift_on_endpoint_state(client, db, app):
"""apply_collector_payload caches drift on the state in the same commit."""
from plugins.endpoints.plugin import EndpointsPlugin
from plugins.endpoints.models import EndpointState
asset, ctype = _computer(db, 'CHKIN01', 'CMM PC')
monitor = _process_monitor(db, 'eDNC', 'dncMain.exe')
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
plugin = EndpointsPlugin()
plugin.apply_collector_payload({'hostname': 'CHKIN01', 'services': []})
db.session.commit()
state = EndpointState.query.filter_by(assetid=asset.assetid).first()
assert state.driftjson is not None
import json
drift = json.loads(state.driftjson)
assert [d['name'] for d in drift] == ['dncMain.exe']
def test_checkin_clears_drift_when_process_returns(client, db, app):
"""A later report that shows the process running clears the cached drift."""
from plugins.endpoints.plugin import EndpointsPlugin
from plugins.endpoints.models import EndpointState
import json
asset, ctype = _computer(db, 'CHKIN02', 'CMM PC')
monitor = _process_monitor(db, 'eDNC', 'dncMain.exe')
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
plugin = EndpointsPlugin()
plugin.apply_collector_payload({'hostname': 'CHKIN02', 'services': []})
db.session.commit()
plugin.apply_collector_payload({
'hostname': 'CHKIN02',
'services': [{'name': 'dncMain.exe', 'kind': 'process',
'state': 'running'}],
})
db.session.commit()
state = EndpointState.query.filter_by(assetid=asset.assetid).first()
assert json.loads(state.driftjson) == []
# ---------------------------------------------------------------------------
# Panel exposure
# ---------------------------------------------------------------------------
def test_status_panel_surfaces_drift_warning(client, db, app, auth_headers):
"""The status panel adds a Needs attention field when drift is cached."""
from plugins.endpoints.plugin import EndpointsPlugin
asset, ctype = _computer(db, 'PANEL01', 'CMM PC')
monitor = _process_monitor(db, 'eDNC', 'dncMain.exe')
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
plugin = EndpointsPlugin()
plugin.apply_collector_payload({'hostname': 'PANEL01', 'services': []})
db.session.commit()
response = client.get(STATUS_URL.format(asset.assetid), headers=auth_headers)
assert response.status_code == 200
labels = [f['label'] for f in response.get_json()['data']['fields']]
assert 'Needs attention' in labels
def test_services_panel_marks_drifted_row(client, db, app, auth_headers):
"""The synthetic Unknown row for an expected-running monitor is drift-flagged."""
from plugins.endpoints.plugin import EndpointsPlugin
asset, ctype = _computer(db, 'PANEL02', 'CMM PC')
monitor = _process_monitor(db, 'eDNC', 'dncMain.exe')
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
plugin = EndpointsPlugin()
plugin.apply_collector_payload({'hostname': 'PANEL02', 'services': []})
db.session.commit()
response = client.get(SERVICES_URL.format(asset.assetid), headers=auth_headers)
assert response.status_code == 200
rows = response.get_json()['data']
edenc = next((r for r in rows if r['name'] == 'eDNC'), None)
assert edenc is not None
assert edenc.get('drift') is True

141
tests/test_plugins/test_endpoints_shipped_monitor_policy.py Normal file
View File

@@ -0,0 +1,141 @@
"""The shipped shopfloor app monitors ride a read-only Site Policy.
Migration endpoints0007shippedpolicy offloads the eDNC/UDC/MTConnect monitors
from the agent's hardcoded appsettings.json into a product-owned policy bundle:
a shipped ConfigurationPolicy "Shopfloor App Monitors" (global scope, one
desired-state monitor action per shipped monitor) wrapped in a shipped
BusinessPolicy "RPCA Shopfloor App Monitors".
Run the endpoints chain against a fresh engine, alongside the computers policy
tables the bundle writes into, and assert the bundle exists, is source='shipped'
(read-only), and attaches every shipped monitor as a desired/running:true action.
"""
import importlib.util
import json
from pathlib import Path
from sqlalchemy import create_engine, inspect, text
from alembic.migration import MigrationContext
from alembic.operations import Operations
_ROOT = Path(__file__).resolve().parent.parent.parent
_ENDPOINTS_VERSIONS = _ROOT / 'plugins' / 'endpoints' / 'migrations' / 'versions'
_COMPUTERS_VERSIONS = _ROOT / 'plugins' / 'computers' / 'migrations' / 'versions'
CONFIG_POLICY_NAME = 'Shopfloor App Monitors'
BUSINESS_POLICY_NAME = 'RPCA Shopfloor App Monitors'
def _load(path, prefix):
spec = importlib.util.spec_from_file_location(prefix + path.stem, str(path))
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def _run_bundle_migrations(connection):
"""Build the policy tables (computers 0004), the monitor catalog (endpoints
0006) and the shipped bundle (endpoints 0007) on one engine, in order.
The core baseline creates every plugin's tables before per-plugin chains run,
so on a real deploy the computers policy tables already exist when endpoints
0007 runs. Here we stand up exactly the three migrations the bundle depends
on - the earlier endpoints revisions carry FKs to core tables that a bare
engine does not have, and are irrelevant to what 0007 seeds.
"""
operations = Operations(MigrationContext.configure(connection))
operations._install_proxy()
try:
_load(_COMPUTERS_VERSIONS / '0004_policies.py',
'computers_mig_').upgrade()
_load(_ENDPOINTS_VERSIONS / '0006_endpoint_monitors.py',
'endpoints_mig_0006_').upgrade()
_load(_ENDPOINTS_VERSIONS / '0007_shipped_monitor_policy.py',
'endpoints_mig_0007_').upgrade()
finally:
operations._remove_proxy()
connection.commit()
def test_shipped_bundle_seeds_read_only_site_policy():
engine = create_engine('sqlite://')
with engine.connect() as connection:
_run_bundle_migrations(connection)
config = connection.execute(text(
'SELECT configurationpolicyid, source, scopejson '
'FROM configurationpolicies WHERE name = :name'),
{'name': CONFIG_POLICY_NAME}).first()
assert config is not None, 'shipped Configuration Policy was not seeded'
config_id, config_source, scopejson = config
assert config_source == 'shipped'
assert json.loads(scopejson) == {'type': 'true'}
business = connection.execute(text(
'SELECT businesspolicyid, source FROM businesspolicies '
'WHERE name = :name'), {'name': BUSINESS_POLICY_NAME}).first()
assert business is not None, 'shipped Business Policy was not seeded'
business_id, business_source = business
assert business_source == 'shipped'
item = connection.execute(text(
'SELECT configurationpolicyid FROM businesspolicyitems '
'WHERE businesspolicyid = :id'), {'id': business_id}).first()
assert item is not None and item[0] == config_id
def test_shipped_bundle_attaches_every_shipped_monitor():
engine = create_engine('sqlite://')
with engine.connect() as connection:
_run_bundle_migrations(connection)
monitor_ids = {row[0] for row in connection.execute(text(
'SELECT endpointmonitorid FROM endpointmonitors '
"WHERE source = 'shipped'"))}
assert monitor_ids, 'no shipped monitors to attach'
config_id = connection.execute(text(
'SELECT configurationpolicyid FROM configurationpolicies '
'WHERE name = :name'), {'name': CONFIG_POLICY_NAME}).scalar()
actions = connection.execute(text(
'SELECT action, mode, desiredstate FROM configurationpolicyactions '
'WHERE configurationpolicyid = :id'), {'id': config_id}).all()
attached_ids = set()
for action, mode, desiredstate in actions:
assert action.startswith('monitor:')
# Gate, not desired: the shipped bundle reports state and exposes
# Start/Stop in the UI, but never auto-starts a process. A desired
# running state is a deliberate per-monitor opt-in, not shipped.
assert mode == 'gate'
assert desiredstate is None
attached_ids.add(int(action[len('monitor:'):]))
assert attached_ids == monitor_ids
def test_shipped_bundle_is_idempotent():
engine = create_engine('sqlite://')
with engine.connect() as connection:
_run_bundle_migrations(connection)
# Re-run only the shipped-bundle migration; it must not duplicate rows.
operations = Operations(MigrationContext.configure(connection))
operations._install_proxy()
try:
_load(_ENDPOINTS_VERSIONS / '0007_shipped_monitor_policy.py',
'endpoints_mig_rerun_').upgrade()
finally:
operations._remove_proxy()
connection.commit()
config_count = connection.execute(text(
'SELECT COUNT(*) FROM configurationpolicies WHERE name = :name'),
{'name': CONFIG_POLICY_NAME}).scalar()
business_count = connection.execute(text(
'SELECT COUNT(*) FROM businesspolicies WHERE name = :name'),
{'name': BUSINESS_POLICY_NAME}).scalar()
assert config_count == 1
assert business_count == 1