Compare commits

..

9 Commits
gh-wj ... wilm

Author SHA1 Message Date
benjamin.baker
77eb594268 Make the server authoritative for what endpoints watch and launch
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / gates (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / agent (push) Has been cancelled
CI / lean-build (push) Has been cancelled
Fleet health showed dncMain.exe red on endpoints that never ran eDNC. Three
layers each re-added the monitor, so fixing any one of them in isolation left
the others to put it back.

The installer baked a watch list into appsettings.json on every package, and
appsettings.json is never rewritten on the box, so an entry outlived whatever
policy justified it. The server auto-attached shipped monitors by PC type name.
The agent then UNIONED the server's resolved list with its local one, so an
entry could never be removed from the server side at all.

Now: the installer seeds nothing, auto-attach by PC type is gone, and
MonitorConfig.Merge replaces the local list with the server's. Null and empty
stay distinct - null means the fetch failed so keep what we have, empty means
watch nothing. Collapsing those blanks monitoring fleet-wide on one failed HTTP
call, or makes detaching the last monitor silently do nothing forever.

Ingest filters reported services against the resolved monitors as well. The
agent config sits on the endpoint, so it is not a trustworthy account of what
that endpoint should be reporting.

Launch paths move the same way, which exposed an older bug. MTConnect installs
to ProgramData and starts through run.bat, but LaunchPathPolicy approved only
the Program Files roots, so the launchpath on its shipped monitor row - seeded
in 0006, present from the start - was rejected every time as outside every
approved root. That was invisible because a local launchpaths entry wins in
ResolveLaunchPath and never reaches validation; remote start worked only
through the bypass. The same precedence meant editing an eDNC, UDC or MTConnect
path in the WebUI could never reach an existing endpoint.

So ProgramData is approved and the local launch map ships empty. ProgramData is
world-writable and the agent launches as SYSTEM, which is why it was excluded
originally; it is deliberately too broad for now and carries a REVISIT comment.
The remaining gates still hold - admin-configured monitor row, exists,
launchable extension, re-checked after symlink resolution. Narrowing it to
ProgramData\udc needs the fleet's non-Program-Files install locations
confirmed first.

Also seeds the Office ProPlus LTSC monitor templates, x64 and x86 as separate
rows since they differ only by path, and adds an agent test project. It links
the platform-pure sources rather than referencing the net8.0-windows assembly
so the semantics run on the Linux CI runners. Writing those tests caught a
merge bug where only the dedupe key was trimmed and the stored name was not.

Note for the wj merge: plugins/usb/frontend/views/usbSettings.spec.js fails on
this branch and is untouched here. Commit da91a29 added four kiosk settings the
save assertion does not expect. The spec looks like the stale side, but that is
the usb author's call - flagging rather than guessing.
2026-09-20 15:04:29 -04:00
benjamin.baker
790ed4ec66 Separate shipped policies from site policies in the list
The list mixed both kinds of configuration policy into one table, so the only
way to tell a shipped default from something the site had written was to read
the source column. Split them into two tables under their own headings.

The split is presentational only. Both tables render the same Edit and Delete
buttons gated on the same iseditable(p) rule, so a shipped policy is not
stranded behind a dead button the way it was when the buttons gated on
source === 'local'.

The spec counted exactly two disabled bindings, which the second table would
have broken while the behaviour it was guarding stayed correct. It now counts
Edit and Delete in pairs and asserts the two always agree, so adding a group
does not fail a test about button gating.
2026-09-20 13:23:23 -04:00
benjamin.baker
3c91e7c314 Resolve Git Bash explicitly, and skip on a missing privilege or tool
Fifteen tests failed on this workstation and had been written off as
environmental. Thirteen of them were not.

Those thirteen ran subprocess.run(['bash', ...]). shutil.which('bash') finds
System32\bash.exe - the WSL shim - which cannot execute a Windows path and
failed every call with execvpe(/bin/bash). They now locate Git Bash from
git.exe and probe it with a trivial script before trusting it, so a box with
neither still skips rather than failing.

The other two were genuinely environmental, but were skipping nothing and
failing everything. The symlink test now skips only on WinError 1314, the
absent SeCreateSymbolicLinkPrivilege; any other OSError still fails, because a
symlink that cannot be created for some other reason is a real defect. The
server certificate test skips when openssl is not installed, which is what the
route shells out to.

Being labelled environmental is why these sat for weeks. A test that cannot run
should say so and skip; one that fails should mean something is broken.
2026-09-20 13:23:11 -04:00
benjamin.baker
0288587522 Check the installed packages against the lockfiles
A clean lockfile does not mean a clean workstation. pip-audit reported
vulnerabilities here while requirements.txt was already pinned to fixed
versions - the installed environment had drifted from the pins, and nothing
compared the two. The same drift had left requirements-dev.txt uninstalled, so
pytest.ini's -n auto failed for want of pytest-xdist.

Add scripts/check-dependency-drift.py, comparing installed distributions
against both lockfiles. MISSING is reported separately from MISMATCHED: a
package that was never installed and a package at the wrong version need
different fixes, and folding them together hid the missing dev requirements.

Wire it into the style gate advisorially, with DEPENDENCY_DRIFT_ENFORCE=1 for
CI. A developer mid-upgrade should not be blocked by a gate that is telling
them something they already know.

The gate now probes python, python3 and py by executing them rather than by
looking them up on PATH. On Windows, python3 resolves to the Microsoft Store
alias stub, which is present on PATH and does nothing when run.

CONTRIBUTING.md gains the Windows-specific traps this exposed: npm.cmd rather
than npm under a Restricted execution policy, the explicit Git Bash path
because bare bash finds the WSL shim, and npm test rather than npx vitest run
because only the pretest hook stages plugin frontends.
2026-09-20 13:22:55 -04:00
benjamin.baker
8d6e5db347 Merge remote-tracking branch 'origin/wj' into wilm 2026-09-19 14:40:46 -04:00
benjamin.baker
32f4ccdd48 Bound the Dell warranty sync per run
The sync route asked Dell about every pending service tag in one request.
bulk_lookup sends 100 tags per HTTP call at a 15 second timeout, so a
freshly-onboarded site was dozens of sequential calls inside a single web
request. A gunicorn worker timeout or a proxy would plausibly kill it first,
leaving the user with a failure after minutes and no way to tell what landed.
F3 released the pooled connection before the vendor call, which stopped one
sync from taking the app down, but did not make the request itself survivable.

Extract the sync into a service taking a per-run tag limit. The route now caps
at 200 tags - two Dell calls - and returns how many are still pending so the
caller can run it again. A flask warranty sync-dell command does the same work
from the command line for a large first-run backfill.

The chunk is sampled at random rather than taken off a stable order. A serial
Dell does not recognize never gains a warranty and so stays pending forever;
under a stable order those dead tags would occupy every chunk and the sync
would never reach the rest of the fleet.

Nothing runs this on a schedule. The operation is used a few times a year, so
a cron would idle-poll the warranty tables tens of thousands of times a year
for nothing and log a configuration failure on every run at sites without Dell
credentials.

The two existing sync tests patched get_provider on the route module; the
provider is now resolved in the service, so they patch it there.
2026-09-19 14:21:41 -04:00
benjamin.baker
530f6a60f8 Release the database connection across Dell warranty lookups
Both Dell routes held a pooled connection open for the duration of the
vendor HTTP call. sync_dell was the severe case: bulk_lookup asks Dell
about 100 service tags per request and calls them sequentially, so a
fleet-sized sync pinned one connection for minutes. On gevent workers
that exhausts the pool and fails every request in the application, not
just warranty.

Copy the values the lookup needs into locals, commit to return the
connection, then reacquire for the writes. The locals are load-bearing:
a commit expires loaded attributes, so reading them afterwards would
quietly check a connection back out and restore the defect.

Preload the existing warranty links in one query. The write loop ran a
query per asset plus one per link, which would have traded the long
network hold for a long write transaction.

Transaction boundaries do not show up in behavioural tests, so the new
tests assert session state at the moment the provider is called. Both
were confirmed to fail against the unfixed code, including the
post-commit attribute-access variant that leaves all other tests green.

Single commit and the existing error handling are unchanged, so the
sync stays all-or-nothing.
2026-09-19 08:54:59 -04:00
benjamin.baker
e00d86827b Merge remote-tracking branch 'origin/wj' into wilm 2026-09-18 17:25:24 -04:00
benjamin.baker
2ff7b7e906 Let Corporate Tools push the Report IP tool without granting script run
Report IP is an icon on every Public Desktop in the company, so a help desk
caller asking for it to be re-run is asking for something they could already
do themselves at the keyboard. Getting there through ShopDB meant holding
endpoints.runscript, which is the permission that lets someone push ANY script
in the library - the agent self-upgrade included. Handing that over to run a
desktop shortcut is the bundling the permission split was created to take
apart, so the work was simply not being done through the tool.

The allowance is on one named script, never on the run_script action. The
script id rides in the caller-controlled request body, so relaxing the action
would grant the whole library and quietly rebuild the endpoints.deploy bundle.
The enqueue route resolves the target to a real Script row first and matches
the STORED name, because matching what the requester typed would let anyone
name their way past the grant.

Granting on a name is only defensible while the name cannot be made to mean
something else. The body is source-controlled at
plugins/endpoints/api/templates/report-ip-shortcut.ps1, seeded from there,
pinned by SHA-256 on delivery, and the managed fields are re-asserted on every
seed - so an operator cannot flip the row to runcontext=system, deactivate it,
or edit the body and leave the name still pushable.

The template lives under the plugin rather than agent/ because the image copies
plugins/ but not agent/: reading it from agent/ works on a developer checkout
and fails in the deployed container.

The script runs in the signed-in user's session. This is not a preference.
Launched as SYSTEM the tool re-enters the interactive session and the launcher
exits 0 immediately, reporting success having observed nothing - the same
session-0 boundary the collector already documents for mapped drives. User
context fails closed when nobody is logged on, which is the honest answer.

The seed runs in the migrate service alongside the other seeders, so an
existing site picks the script up on the upgrade it already performs.
2026-09-18 17:25:20 -04:00
89 changed files with 19896 additions and 19491 deletions

View File

@@ -6,12 +6,11 @@
# gates - proves those checks actually RAN rather than skipping
# frontend - vitest + Vue build
# lean-build - ADR-013: a per-site build carries only its plugins
# api-contract - the published spec matches the code that serves it
#
# The MySQL-8 migration job is NOT here; it runs on the internal CI server. See
# the note at the bottom of this file. It was listed here for a while after it
# moved, which reads as coverage this workflow does not have - do not re-add it
# to this list without re-adding the job to a runner that can host it.
# migrations-mysql - the REAL multi-site deploy path: fresh flask db upgrade
# + every plugin's chain on utf8mb4 MySQL 8, idempotent on
# a second run. The pytest suite only exercises SQLite
# create_all(), so this is what catches an Alembic
# regression on MySQL before it ships.
name: CI
@@ -54,60 +53,6 @@ jobs:
- uses: actions/checkout@v5
- run: bash scripts/check-naming-and-style.sh
api-contract:
# The published API contract has to match the code that serves it. These
# gates catch a route that exists and is undocumented, a query parameter the
# handler reads that the spec omits, and a public route nobody declared
# public. They already work; until now nothing ran them except by hand, and
# the spec had silently drifted by three operations.
runs-on: arc-runner-set
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: '3.14'
cache: pip
- run: pip install -r requirements-dev.txt
- run: |
python -m pytest -q \
tests/test_api_inventory_parity.py \
tests/test_openapi_spec.py \
tests/test_openapi_examples.py \
tests/test_core/test_public_read_surface.py
- name: Seed the plugin registry the generator needs
# gen_openapi.py refuses to run while a bundled plugin is disabled, and
# instance/plugins.json is gitignored, so a clean checkout has none.
# The step used to pass only because the pytest step above writes that
# file at import time and never removes it - green by accident of step
# order, and red the moment the steps are reordered, with an error
# about disabled plugins that has nothing to do with the spec.
run: |
python - <<'PY'
import datetime, json, pathlib
names = sorted(d.name for d in pathlib.Path('plugins').iterdir()
if (d / 'manifest.json').is_file())
now = datetime.datetime.now().isoformat()
out = pathlib.Path('instance'); out.mkdir(exist_ok=True)
(out / 'plugins.json').write_text(json.dumps({'plugins': {
n: {'name': n, 'version': '1.0.0', 'installed_at': now,
'enabled': True, 'migrations_applied': [], 'config': {}}
for n in names}}, indent=2))
print('registry seeded with', len(names), 'plugins')
PY
- name: The spec must be regenerated, not just passing
# --refresh is derived from the code, so if it changes anything the
# committed spec was stale. Fail here rather than let a generated client
# be built from a file that no longer describes the API.
run: |
python scripts/gen_openapi.py --refresh
if ! git diff --quiet docs/openapi.json docs/api-inventory.json; then
echo "docs/openapi.json is stale. Run:"
echo " python scripts/gen_openapi.py --refresh"
git --no-pager diff --stat docs/openapi.json docs/api-inventory.json
exit 1
fi
gates:
# Prove the gates RAN. Several skip themselves when the file they read is
# absent, which is correct in this checkout (docs/ becomes the wiki) and was
@@ -142,6 +87,24 @@ jobs:
- run: npx vitest run
- run: npm run build
agent:
# Semantic checks on the endpoint agent's pure logic. Until this existed the
# agent had no tests at all, so a change to what an endpoint watches - or to
# the guard that decides which paths it will execute - was reviewed by
# reading alone.
#
# The project targets net8.0 and links the platform-pure sources rather than
# referencing ShopDbEndpointAgent.Core, which is net8.0-windows and cannot
# run here. See the csproj header for why, and for what that trades away.
# Runs in well under a second; it does not meaningfully add to gate time.
runs-on: arc-runner-set
steps:
- uses: actions/checkout@v5
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
- run: dotnet test agent/ShopDbEndpointAgent.Core.Tests/ShopDbEndpointAgent.Core.Tests.csproj --nologo
lean-build:
# ADR-013 Phase 5: prove a per-site build carries only its chosen plugins.
# The profile is the WJ plugin set, so this job proves the build this branch

View File

@@ -206,31 +206,73 @@ Tests are required for:
- New plugin hooks (contract test)
- Schema migrations (before/after data integrity)
## Adding or changing an API route
### Running the gates on Windows
The published contract is generated from the code, so it goes stale the moment
a route changes and nobody regenerates it. A stale spec is worse than no spec:
it is what a client generator, an agent, or the MCP server in `tools/mcp/` will
believe.
Three commands in this repository resolve to the wrong program on a Windows dev
box. Each one fails in a way that looks like a code fault, which is why all
three are written out here rather than left to be rediscovered.
After adding or changing a route:
```bash
& "C:\Program Files\Git\bin\bash.exe" scripts/check-naming-and-style.sh
cd frontend; npm.cmd test; npm.cmd run build; cd ..
python -m pytest tests/ -q
```
python scripts/gen_openapi.py --refresh
- **`bash` resolves to WSL.** `C:\Windows\System32\bash.exe` is on PATH by
default. With no distribution installed it fails with
`execvpe(/bin/bash) failed: No such file or directory`, and with one installed
it cannot read the Windows path you handed it. Always give the Git Bash path
in full. The same trap bit the test suite: `tests/test_naming_gate_migrations.py`
used bare `bash`, and its 13 failures were miscategorised as "environmental"
for weeks. It now resolves Git Bash explicitly and probes it before use.
- **`npm` resolves to `npm.ps1`.** PowerShell prefers the `.ps1` over the
`.cmd`, and the default execution policy is `Restricted`, so bare `npm` dies
with "npm.ps1 cannot be loaded because running scripts is disabled on this
system" before any test runs. Use `npm.cmd`. Nothing in the repo causes this
and nothing in the repo can fix it; it is machine policy. If you would rather
fix it once, `Set-ExecutionPolicy -Scope CurrentUser RemoteSigned` also works,
but that is a machine-level change and `npm.cmd` is free.
- **`npm test`, not `npx vitest run`.** The `pretest` hook runs
`stage-frontend.mjs`; bare vitest never stages plugin frontends and silently
skips their specs.
That rewrites `docs/api-inventory.json` and `docs/openapi.json` from the live
app. It fills in what is derived and leaves prose you have written alone, so run
it freely and then replace any seeded summary with the reason the route exists.
### Keeping the environment matched to the lockfiles
If it prints `review` against your route, it could not find a gate on the view.
Do not hand-edit the inventory - the next refresh overwrites it. Either gate the
view, or, if the route is genuinely public, name it in `PUBLIC_ENDPOINTS` in
`tests/test_core/test_public_read_surface.py` with the reason. The generator
reads that list, so the published contract and the enforced gate cannot
disagree.
```bash
python scripts/check-dependency-drift.py
```
The `api-contract` CI job runs the parity gates and then regenerates and fails
if anything changed, so a stale spec fails the pull request rather than being
discovered months later.
Editing a lockfile changes what a FUTURE install produces. It does not touch an
already-populated `site-packages`. Commit `a8f8580` raised eight pins carrying
advisories; no install was re-run afterwards, so that workstation kept the old
libraries for three weeks while `requirements.txt` said otherwise. Every local
test run in that window exercised different libraries than a container build
would have.
Run this after any pull that touches `requirements.txt` or
`requirements-dev.txt`, and fix drift with:
```bash
python -m pip install --require-hashes -r requirements.txt
python -m pip install --require-hashes -r requirements-dev.txt
```
The check reports MISSING separately from MISMATCHED, because absent packages
are the class that gets missed: `pytest-xdist` was simply not installed, which
surfaced only as `pytest: error: unrecognized arguments: -n` - a message that
reads as a broken `pytest.ini` and invites deleting `-n auto` instead of
installing the plugin.
`scripts/check-naming-and-style.sh` runs this check too, advisory by default.
CI sets `DEPENDENCY_DRIFT_ENFORCE=1` to make it fail the build.
If you use the repo-local `env/` virtualenv, check it specifically - it drifts
independently of the global interpreter, and both were stale in the incident
above:
```bash
env/Scripts/python.exe scripts/check-dependency-drift.py
```
## Branches: what belongs on main, and what does not

View File

@@ -133,20 +133,6 @@ COPY wsgi.py ./
# Make entrypoint and bootstrap scripts executable
RUN chmod +x /app/scripts/docker-entrypoint.sh /app/scripts/bootstrap-plugins.sh
# The two files under docs/ that the APP SERVES at runtime, not documentation.
# shopdb/core/api/docs.py resolves both relative to the repo root, so without
# them /api/docs renders the Redoc shell and then 404s fetching its own spec -
# which is what every Docker deployment did. Copied by name rather than copying
# docs/ wholesale: the rest of that tree is prose and has no business in a lean
# runtime image.
COPY docs/openapi.json docs/llms.txt ./docs/
# Fail the BUILD on a syntax error rather than the pod at runtime. Docker never
# imports the application, so a broken .py sails through every layer and only
# surfaces as a crash-looping container: observed, four restarts before anyone
# noticed. compileall touches nothing at runtime and costs a couple of seconds.
RUN python -m compileall -q shopdb plugins migrations wsgi.py
# Built SPA from stage 1. Flask serves it via register_frontend_routes.
COPY --from=frontendbuild /build/frontend/dist ./frontend/dist

View File

@@ -0,0 +1,312 @@
using ShopDbEndpointAgent.Core;
using Xunit;
namespace ShopDbEndpointAgent.Core.Tests;
// Pins LaunchPathPolicy, the safeguard that lets the agent honor a launch path
// chosen in the WebUI without turning "write access to a file" into "remote code
// execution as LocalSystem".
//
// These tests build a real directory tree under the temp dir and point the
// policy at it via the approved-roots override, rather than asserting against
// hardcoded C:\Program Files strings. That keeps them honest about the checks
// (which touch the disk: File.Exists, link resolution) while staying runnable on
// the Linux CI runners. Cases that genuinely require Windows semantics, notably
// NTFS alternate data streams and junctions, are called out where they appear.
public sealed class LaunchPathPolicyTests : IDisposable
{
private readonly string root;
public LaunchPathPolicyTests()
{
root = Path.Combine(Path.GetTempPath(),
"shopdb-launchpolicy-" + Guid.NewGuid().ToString("n"));
Directory.CreateDirectory(root);
}
public void Dispose()
{
try { Directory.Delete(root, recursive: true); }
catch (IOException) { }
catch (UnauthorizedAccessException) { }
}
private LaunchPathPolicy PolicyForRoot()
{
return new LaunchPathPolicy(new[] { root });
}
private string CreateFile(string relativePath)
{
var full = Path.Combine(root, relativePath);
Directory.CreateDirectory(Path.GetDirectoryName(full)!);
File.WriteAllText(full, "");
return full;
}
[Fact]
public void An_approved_executable_under_an_approved_root_is_accepted()
{
var target = CreateFile(Path.Combine("UDC", "udc.exe"));
var accepted = PolicyForRoot().TryValidate(target, out var fullPath, out var reason);
Assert.True(accepted, reason);
Assert.Equal(target, fullPath);
}
[Theory]
[InlineData("udc.exe")]
[InlineData("run.bat")]
[InlineData("run.cmd")]
public void Every_launchable_extension_is_accepted(string fileName)
{
// .bat and .cmd are deliberate: some shop-floor apps ship a launcher
// script rather than an exe.
var target = CreateFile(fileName);
var accepted = PolicyForRoot().TryValidate(target, out _, out var reason);
Assert.True(accepted, reason);
}
[Theory]
[InlineData("payload.dll")]
[InlineData("payload.ps1")]
[InlineData("readme.txt")]
[InlineData("payload.vbs")]
public void A_non_launchable_extension_is_rejected(string fileName)
{
var target = CreateFile(fileName);
var accepted = PolicyForRoot().TryValidate(target, out _, out var reason);
Assert.False(accepted);
Assert.Contains("not launchable", reason);
}
[Fact]
public void A_path_outside_every_approved_root_is_rejected()
{
var outside = Path.Combine(Path.GetTempPath(),
"shopdb-outside-" + Guid.NewGuid().ToString("n"));
Directory.CreateDirectory(outside);
try
{
var target = Path.Combine(outside, "evil.exe");
File.WriteAllText(target, "");
var accepted = PolicyForRoot().TryValidate(target, out _, out var reason);
Assert.False(accepted);
Assert.Contains("outside every approved launch root", reason);
}
finally
{
Directory.Delete(outside, recursive: true);
}
}
[Fact]
public void A_sibling_root_with_a_shared_prefix_is_rejected()
{
// Guards the root comparison: "<root>" must not also match
// "<root>Evil", which a naive StartsWith without a separator allows.
var sibling = root + "Evil";
Directory.CreateDirectory(sibling);
try
{
var target = Path.Combine(sibling, "evil.exe");
File.WriteAllText(target, "");
var accepted = PolicyForRoot().TryValidate(target, out _, out var reason);
Assert.False(accepted);
Assert.Contains("outside every approved launch root", reason);
}
finally
{
Directory.Delete(sibling, recursive: true);
}
}
[Fact]
public void A_traversal_escaping_an_approved_root_is_rejected()
{
// GetFullPath normalizes the "..", so the check sees where the path
// actually lands rather than the approved-looking prefix it started
// with.
var target = Path.Combine(root, "..", "escaped.exe");
var accepted = PolicyForRoot().TryValidate(target, out _, out var reason);
Assert.False(accepted);
Assert.Contains("outside every approved launch root", reason);
}
[Fact]
public void A_path_that_does_not_exist_is_rejected()
{
var target = Path.Combine(root, "missing.exe");
var accepted = PolicyForRoot().TryValidate(target, out _, out var reason);
Assert.False(accepted);
Assert.Contains("does not exist", reason);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData(" ")]
public void An_empty_candidate_is_rejected(string? candidate)
{
// This is the common case, not an edge case: most monitors carry no
// launch path at all, and the caller must fail closed rather than
// treating "no path" as "any path".
var accepted = PolicyForRoot().TryValidate(candidate, out _, out var reason);
Assert.False(accepted);
Assert.Equal("empty path", reason);
}
[Fact]
public void A_rejected_path_never_reports_a_usable_full_path()
{
// The caller keys off the bool but passes fullPath onward, so a
// rejection must not leave anything launchable in it.
var accepted = PolicyForRoot().TryValidate(
Path.Combine(root, "missing.exe"), out var fullPath, out _);
Assert.False(accepted);
Assert.Equal("", fullPath);
}
[Fact]
public void An_alternate_data_stream_is_rejected()
{
// Windows-only semantics. On net8.0 GetFullPath no longer throws on an
// interior colon, so "readme.txt:payload.exe" would otherwise report an
// .exe extension, pass the root check, and exist as a stream.
if (!OperatingSystem.IsWindows())
return;
CreateFile("readme.txt");
var target = Path.Combine(root, "readme.txt:payload.exe");
var accepted = PolicyForRoot().TryValidate(target, out _, out var reason);
Assert.False(accepted);
Assert.Contains("alternate data stream", reason);
}
[Fact]
public void A_symlink_pointing_outside_an_approved_root_is_rejected()
{
// The lexical checks constrain the NAME; this pins that the policy also
// resolves what the name actually refers to. Creating a link needs no
// privilege beyond write access to the parent, and vendor install trees
// under Program Files are not reliably locked down.
var outside = Path.Combine(Path.GetTempPath(),
"shopdb-linktarget-" + Guid.NewGuid().ToString("n"));
Directory.CreateDirectory(outside);
try
{
var realTarget = Path.Combine(outside, "payload.exe");
File.WriteAllText(realTarget, "");
var link = Path.Combine(root, "innocent.exe");
try
{
File.CreateSymbolicLink(link, realTarget);
}
catch (UnauthorizedAccessException)
{
return;
}
catch (IOException)
{
return;
}
var accepted = PolicyForRoot().TryValidate(link, out _, out var reason);
Assert.False(accepted);
Assert.Contains("resolves outside every approved launch root", reason);
}
finally
{
Directory.Delete(outside, recursive: true);
}
}
[Fact]
public void An_empty_roots_override_falls_back_to_the_shipped_defaults()
{
// A site that configures no roots must not end up with an empty
// allowlist, which would fail every launch closed.
var policy = new LaunchPathPolicy(Array.Empty<string>());
var target = CreateFile("udc.exe");
var accepted = policy.TryValidate(target, out _, out var reason);
Assert.False(accepted);
Assert.Contains("outside every approved launch root", reason);
}
[Fact]
public void A_whitespace_only_root_override_is_ignored()
{
var policy = new LaunchPathPolicy(new[] { " ", "" });
var target = CreateFile("udc.exe");
var accepted = policy.TryValidate(target, out _, out _);
Assert.False(accepted);
}
[Fact]
public void ProgramData_is_an_approved_root_so_MTConnect_can_start()
{
// MTConnect installs to ProgramData\udc\mtconnect_udc\agent and starts
// through run.bat. With only the Program Files roots approved, its
// server-provisioned launch path was rejected outright, which is why
// remote start only ever worked through a local launchpaths pin that
// skipped validation. Deliberately broad for now - see BuildDefaultRoots.
var programData = Environment.GetFolderPath(
Environment.SpecialFolder.CommonApplicationData);
Assert.False(string.IsNullOrWhiteSpace(programData));
Assert.Contains(
LaunchPathPolicy.DefaultApprovedRoots,
root => root.StartsWith(programData, StringComparison.OrdinalIgnoreCase));
}
[Fact]
public void The_Program_Files_roots_are_still_approved()
{
// Widening to ProgramData must not have replaced the original roots:
// eDNC and UDC both install under Program Files.
var programFiles = Environment.GetFolderPath(
Environment.SpecialFolder.ProgramFiles);
Assert.False(string.IsNullOrWhiteSpace(programFiles));
Assert.Contains(
LaunchPathPolicy.DefaultApprovedRoots,
root => root.StartsWith(programFiles, StringComparison.OrdinalIgnoreCase));
}
[Fact]
public void A_launcher_script_under_an_approved_root_is_accepted()
{
// The run.bat pattern: no directly launchable .exe, so the approved
// extension set has to keep covering .bat.
var policy = PolicyForRoot();
var target = CreateFile("run.bat");
var accepted = policy.TryValidate(target, out var validated, out var reason);
Assert.True(accepted, reason);
Assert.EndsWith("run.bat", validated, StringComparison.OrdinalIgnoreCase);
}
}

View File

@@ -0,0 +1,71 @@
using ShopDbEndpointAgent.Core;
using Xunit;
namespace ShopDbEndpointAgent.Core.Tests;
// Pins the local launch map, which decides where process_start looks for an
// executable before it consults the server.
//
// The rule these tests protect: launch paths are SERVER authoritative. A local
// entry wins over the server-provisioned path and bypasses LaunchPathPolicy
// validation entirely, so anything shipped here is both unvalidated and immune
// to WebUI edits. That combination is what left stale pins on endpoints, so the
// compiled defaults are empty on purpose and a test guards that.
public class LaunchPathResolutionTests
{
[Fact]
public void No_launch_paths_are_compiled_into_the_agent()
{
// A compiled pin cannot be changed from the WebUI and skips validation.
// eDNC, UDC and MTConnect all carry their launchpath on the monitor row.
Assert.Empty(AgentConfig.DefaultLaunchPaths);
}
[Fact]
public void A_config_without_launch_paths_resolves_to_an_empty_map()
{
// Empty here is what makes ResolveLaunchPath fall through to the
// server-provisioned path on the monitor row.
var config = new AgentConfig();
Assert.Empty(config.ResolveLaunchPaths());
}
[Fact]
public void A_local_override_still_wins_when_a_site_sets_one()
{
// The escape hatch stays available for an endpoint that cannot use the
// server value; it just is not shipped by default any more.
var config = new AgentConfig();
config.LaunchPaths["UDC.exe"] = @"D:\Custom\UDC.exe";
var resolved = config.ResolveLaunchPaths();
Assert.Equal(@"D:\Custom\UDC.exe", resolved["UDC"]);
}
[Fact]
public void A_local_override_key_is_normalized_to_a_bare_image_name()
{
// Keys are stored bare and the map is case-insensitive; CommandExecutor
// normalizes the lookup side the same way, so '.exe' or not both hit.
var config = new AgentConfig();
config.LaunchPaths["DncMain.exe"] = @"D:\Dnc\DncMain.exe";
var resolved = config.ResolveLaunchPaths();
Assert.True(resolved.ContainsKey("DncMain"));
Assert.True(resolved.ContainsKey("dncmain"));
Assert.False(resolved.ContainsKey("DncMain.exe"));
}
[Fact]
public void Blank_launch_path_entries_are_ignored()
{
var config = new AgentConfig();
config.LaunchPaths["UDC.exe"] = " ";
config.LaunchPaths[" "] = @"D:\Stray.exe";
Assert.Empty(config.ResolveLaunchPaths());
}
}

View File

@@ -0,0 +1,203 @@
using ShopDbEndpointAgent.Core;
using Xunit;
namespace ShopDbEndpointAgent.Core.Tests;
// Pins MonitorConfig.Merge, which decides what an endpoint watches.
//
// The rule these tests exist to protect: the SERVER is authoritative when it
// answers, and a non-answer is not an answer. Concretely, null means "the fetch
// failed, keep what we have" while an empty-but-present list means "watch
// nothing". Collapsing those two cases is the bug this suite is here to catch,
// and it is a bug in both directions. Treat null as empty and one failed HTTP
// call blanks monitoring on every endpoint at once. Treat empty as null and
// detaching the last monitor in the WebUI silently does nothing forever.
public class MonitorConfigMergeTests
{
private static MonitorConfig Local(params string[] processNames)
{
return new MonitorConfig
{
Processes = processNames
.Select(name => new MonitorTarget { Name = name })
.ToArray(),
};
}
private static string[] Names(MonitorTarget[] targets)
{
return targets.Select(target => target.Name).OrderBy(name => name).ToArray();
}
[Fact]
public void A_server_list_replaces_the_local_one_rather_than_unioning()
{
// The regression that motivated the change: dncMain.exe was baked into
// appsettings.json by the installer and kept being reported on every
// endpoint even though no policy attached it. Under the old union it
// survived forever because the server had no way to retract it.
var local = Local("dncMain.exe", "UDC.exe");
var server = new ResolvedMonitors
{
Processes = new[] { new MonitorTarget { Name = "notepad.exe" } },
};
var merged = MonitorConfig.Merge(local, server);
Assert.Equal(new[] { "notepad.exe" }, Names(merged.Processes));
}
[Fact]
public void A_null_server_response_keeps_the_current_watch_list()
{
// FetchResolvedMonitorsAsync returns null on every failure path. A
// transient blip must not blank monitoring fleet-wide.
var local = Local("dncMain.exe");
var merged = MonitorConfig.Merge(local, null);
Assert.Equal(new[] { "dncMain.exe" }, Names(merged.Processes));
}
[Fact]
public void An_empty_server_response_clears_the_watch_list()
{
// The counterpart to the test above, and the reason null and empty must
// stay distinguishable: detaching the last monitor has to take effect.
var local = Local("dncMain.exe");
var server = new ResolvedMonitors();
var merged = MonitorConfig.Merge(local, server);
Assert.Empty(merged.Processes);
Assert.Empty(merged.Services);
}
[Fact]
public void Services_are_replaced_independently_of_processes()
{
var local = new MonitorConfig
{
Processes = new[] { new MonitorTarget { Name = "dncMain.exe" } },
Services = new[] { new MonitorTarget { Name = "MTConnectAgent" } },
};
var server = new ResolvedMonitors
{
Processes = new[] { new MonitorTarget { Name = "udc.exe" } },
};
var merged = MonitorConfig.Merge(local, server);
Assert.Equal(new[] { "udc.exe" }, Names(merged.Processes));
Assert.Empty(merged.Services);
}
[Fact]
public void A_null_local_config_with_a_null_server_response_is_not_a_crash()
{
// Defensive: the worker holds a singleton AgentConfig whose Monitors
// property is assignable, so null is reachable.
var merged = MonitorConfig.Merge(null, null);
Assert.Empty(merged.Processes);
Assert.Empty(merged.Services);
}
[Fact]
public void Server_entries_are_deduplicated_case_insensitively_by_name()
{
// Names are matched by string equality downstream, in the collector and
// in the launch-path lookup, so a duplicate would be reported twice.
var server = new ResolvedMonitors
{
Processes = new[]
{
new MonitorTarget { Name = "UDC.exe", App = "first" },
new MonitorTarget { Name = "udc.exe", App = "second" },
},
};
var merged = MonitorConfig.Merge(Local(), server);
var single = Assert.Single(merged.Processes);
Assert.Equal("second", single.App);
}
[Fact]
public void Blank_server_entries_are_dropped()
{
var server = new ResolvedMonitors
{
Processes = new[]
{
new MonitorTarget { Name = " " },
new MonitorTarget { Name = "" },
new MonitorTarget { Name = "udc.exe" },
},
};
var merged = MonitorConfig.Merge(Local(), server);
Assert.Equal(new[] { "udc.exe" }, Names(merged.Processes));
}
[Fact]
public void Server_names_are_trimmed_so_downstream_matching_works()
{
// The collector compares the reported name against enumerated process
// images by exact string equality; an untrimmed name never matches and
// the monitor silently reports stopped forever.
var server = new ResolvedMonitors
{
Processes = new[] { new MonitorTarget { Name = " udc.exe " } },
};
var merged = MonitorConfig.Merge(Local(), server);
Assert.Equal("udc.exe", Assert.Single(merged.Processes).Name);
}
[Fact]
public void A_server_provisioned_launch_path_survives_the_replace()
{
// CommandExecutor.ServerLaunchPathFor reads the merged watch list to
// find launch paths, so dropping LaunchPath here would break
// process_start for policy-provisioned apps.
var server = new ResolvedMonitors
{
Processes = new[]
{
new MonitorTarget
{
Name = "udc.exe",
App = "UDC",
LaunchPath = @"C:\Program Files\UDC\udc.exe",
},
},
};
var merged = MonitorConfig.Merge(Local(), server);
var target = Assert.Single(merged.Processes);
Assert.Equal(@"C:\Program Files\UDC\udc.exe", target.LaunchPath);
Assert.Equal("UDC", target.App);
}
[Fact]
public void A_local_only_monitor_does_not_survive_a_successful_fetch()
{
// The security half of the change. A tampered appsettings.json could
// otherwise add a never-running entry and park the asset at a permanent
// condition, training operators to ignore a genuinely amber endpoint.
var local = Local("fabricated-never-running.exe");
var server = new ResolvedMonitors
{
Processes = new[] { new MonitorTarget { Name = "udc.exe" } },
};
var merged = MonitorConfig.Merge(local, server);
Assert.DoesNotContain(merged.Processes,
target => target.Name == "fabricated-never-running.exe");
}
}

View File

@@ -0,0 +1,60 @@
<Project Sdk="Microsoft.NET.Sdk">
<!-- Targets plain net8.0, NOT net8.0-windows, and deliberately does not
ProjectReference ShopDbEndpointAgent.Core.
Core targets net8.0-windows because it needs WMI, the registry and the
service control manager. Every CI runner in this repo is Linux
(arc-runner-set), and a net8.0 project cannot even reference a
net8.0-windows assembly. A test project that mirrored Core's target
would therefore never run in CI, and a suite that does not run is not a
gate - it is a file nobody reads.
So the platform-PURE sources are linked in and compiled again here as
ordinary .NET 8 code. The three files below have exactly one using
between them (System.Text.Json.Serialization); they are data shapes and
string logic with no Windows surface, so they behave identically on
Linux. This keeps the suite in the existing gates at no infrastructure
cost.
The tradeoff, stated plainly: these tests exercise a recompilation of
the source, not the shipped Windows assembly. That is sound only while
the linked files stay platform-pure. Adding a Windows-only using to one
of them breaks this build loudly, which is fine. Adding an #if WINDOWS
branch would NOT break it and would quietly make the tested path differ
from the shipped one - so do not do that to a linked file. The exit from
this arrangement is multi-targeting Core as net8.0-windows;net8.0, at
which point these tests reference the real assembly and carry over
unchanged.
AgentConfigProvider (Microsoft.Win32) and CommandExecutor
(System.ServiceProcess, System.Security.*) are NOT linked here: they
cannot compile on Linux, and testing them honestly needs a seam so the
registry and SCM can be faked. That refactor is tracked separately. -->
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<RootNamespace>ShopDbEndpointAgent.Core.Tests</RootNamespace>
<AssemblyName>ShopDbEndpointAgent.Core.Tests</AssemblyName>
<InvariantGlobalization>true</InvariantGlobalization>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<Compile Include="..\ShopDbEndpointAgent.Core\AgentConfig.cs"
Link="Linked\AgentConfig.cs" />
<Compile Include="..\ShopDbEndpointAgent.Core\Contracts.cs"
Link="Linked\Contracts.cs" />
<Compile Include="..\ShopDbEndpointAgent.Core\LaunchPathPolicy.cs"
Link="Linked\LaunchPathPolicy.cs" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.11.1" />
<PackageReference Include="xunit" Version="2.9.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.8.2" />
</ItemGroup>
</Project>

View File

@@ -81,22 +81,27 @@ public sealed class AgentConfig
[JsonPropertyName("approvedlaunchroots")]
public List<string> ApprovedLaunchRoots { get; set; } = new();
// Shipped defaults for the org's common shop-floor apps. A site's
// appsettings.json LaunchPaths entries win over these on a per-image basis.
// MTConnect is intentionally absent here: it is a Windows service controlled
// by the fixed mtconnect_* handler (install.bat + service control), not the
// generic process_start path. eDNC and UDC launch their runtime executables
// directly.
// Intentionally EMPTY: launch paths are server authoritative.
//
// Launch paths now live on the monitor rows the WebUI owns, the same place
// the watch list comes from, so editing a path in the WebUI reaches every
// endpoint with no rebuild and no redeploy. A compiled default here (or an
// appsettings.json launchpaths entry) still WINS over the server path and
// skips LaunchPathPolicy validation entirely, which is exactly how a stale
// pinned path would outlive the configuration that justified it.
//
// This stays empty on purpose. Populate it only as a deliberate local
// override for one endpoint that cannot use the server value, and expect
// that endpoint to stop tracking WebUI changes for that image.
public static readonly IReadOnlyDictionary<string, string> DefaultLaunchPaths =
new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
{
["DncMain.exe"] = @"C:\Program Files (x86)\Dnc\bin\DncMain.exe",
["UDC.exe"] = @"C:\Program Files\UDC\UDC.exe",
};
new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
// The effective launch map: shipped defaults overlaid with appsettings.json
// entries (site wins). Keys are normalized to bare image names so a lookup
// tolerates 'DncMain' or 'DncMain.exe'.
// The effective launch map: shipped defaults (now empty) overlaid with
// appsettings.json entries. Normally empty, in which case process_start
// resolves through the server-provisioned path on the monitor row. Keys are
// normalized to bare image names so a lookup tolerates 'DncMain' or
// 'DncMain.exe'. Anything here BYPASSES LaunchPathPolicy - see
// DefaultLaunchPaths.
public IReadOnlyDictionary<string, string> ResolveLaunchPaths()
{
var resolved = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
@@ -212,38 +217,58 @@ public sealed class MonitorConfig
[JsonPropertyName("services")] public MonitorTarget[] Services { get; set; }
= Array.Empty<MonitorTarget>();
// Union the local (appsettings.json) watch list with the server-resolved one,
// keyed case-insensitively by wire name so a target watched both locally and
// by policy is not enumerated twice. Server entries win on the label (App) so
// a policy display name overrides a stale local one, but a local-only target
// is preserved - the server list augments, it does not replace. Nulls on
// either side degrade to the other, which is what keeps monitoring alive when
// a fetch fails.
// The SERVER decides what this endpoint watches. A server list replaces the
// local (appsettings.json) one outright rather than merging into it.
//
// This used to union the two, which made the effective watch list "whatever
// the server says PLUS whatever the box says", and the box half could not be
// retracted from the server at all. Two consequences, both bad. Operationally,
// monitors the installer had baked into appsettings.json kept being reported
// long after policy stopped attaching them, so detaching a monitor in the
// WebUI silently did nothing on already-deployed endpoints. Security-wise, it
// made a local file an input to server-side health: anyone able to edit it
// could add a never-running entry and park the asset at a permanent
// condition, training operators to ignore an endpoint that is amber for a
// fabricated reason.
//
// The caller MUST only call this when the fetch succeeded. A failed fetch
// returns null from FetchResolvedMonitorsAsync and the caller keeps the
// current list - that is what stops a transient network blip from blanking
// monitoring fleet-wide. A null server argument here is therefore treated as
// "no answer, keep local" rather than "watch nothing"; an EMPTY but present
// server list is a real answer and does clear the watch list, which is what
// makes detaching the last monitor take effect.
public static MonitorConfig Merge(MonitorConfig? local, ResolvedMonitors? server)
{
if (server is null)
return local ?? new MonitorConfig();
return new MonitorConfig
{
Processes = MergeTargets(local?.Processes, server?.Processes),
Services = MergeTargets(local?.Services, server?.Services),
Processes = Authoritative(server.Processes),
Services = Authoritative(server.Services),
};
}
private static MonitorTarget[] MergeTargets(
MonitorTarget[]? local, MonitorTarget[]? server)
// Copies the server list, dropping unusable entries. Names are trimmed on
// the TARGET, not just on the dictionary key: downstream matching reads
// target.Name (the collector compares it against enumerated process images,
// and CommandExecutor.ServerLaunchPathFor matches it against the requested
// image), so a padded name that was only trimmed for de-duplication would
// never match anything and the monitor would report stopped forever.
private static MonitorTarget[] Authoritative(MonitorTarget[]? server)
{
if (server is null || server.Length == 0)
return Array.Empty<MonitorTarget>();
var byName = new Dictionary<string, MonitorTarget>(
StringComparer.OrdinalIgnoreCase);
foreach (var target in local ?? Array.Empty<MonitorTarget>())
foreach (var target in server)
{
if (string.IsNullOrWhiteSpace(target?.Name))
continue;
byName[target.Name.Trim()] = target;
}
foreach (var target in server ?? Array.Empty<MonitorTarget>())
{
if (string.IsNullOrWhiteSpace(target?.Name))
continue;
byName[target.Name.Trim()] = target;
target.Name = target.Name.Trim();
byName[target.Name] = target;
}
return byName.Values.ToArray();
}
@@ -258,12 +283,13 @@ public sealed class MonitorTarget
// Human-facing application label, optional. Falls back to Name when empty.
[JsonPropertyName("app")] public string App { get; set; } = "";
// Optional server-provisioned launch path for a process monitor. Only set on
// process targets that a site leader configured a path for. The agent NEVER
// launches this blindly: LaunchPathPolicy validates it against an approved-
// root allowlist first, and a local appsettings.json launchpaths entry for
// the same image still wins. Null/empty means "no server path" - the agent
// falls back to its local launch map, and process_start fails closed when
// neither yields a validated path.
// Optional server-provisioned launch path for a process monitor. Set on
// process targets a site leader configured a path for, and now the PRIMARY
// source for process_start since the local launch map ships empty. The
// agent never launches this blindly: LaunchPathPolicy validates it against
// the approved-root allowlist, checks the extension, confirms it exists and
// re-checks it after resolving symlinks/junctions. A local appsettings.json
// launchpaths entry for the same image still wins if one is present.
// Null/empty means "no server path" and process_start fails closed.
[JsonPropertyName("launchpath")] public string? LaunchPath { get; set; }
}

View File

@@ -237,8 +237,10 @@ public sealed class CommandResult
// The server-resolved watch list for this endpoint (processes and services the
// site's policy attaches to the box's PC type), fetched each telemetry cycle so
// a policy-attached monitor is enumerated without editing appsettings.json on
// the box. Same {name, app} shape as the local MonitorConfig, so the merge is a
// direct list union. Empty lists mean "watch nothing extra beyond local".
// the box. Same {name, app} shape as the local MonitorConfig, and it REPLACES
// that local list rather than unioning with it - the server is authoritative on
// what is watched. Empty lists mean "watch nothing", which is how detaching the
// last monitor takes effect; "no answer" is a null response, not an empty one.
public sealed class ResolvedMonitors
{
[JsonPropertyName("processes")] public MonitorTarget[] Processes { get; set; }

View File

@@ -23,8 +23,10 @@ public sealed class LaunchPathPolicy
private static readonly string[] ApprovedExtensions =
{ ".exe", ".bat", ".cmd" };
// Shipped org defaults: the standard 64- and 32-bit install roots. Resolved
// from the environment so a non-C: system drive still matches.
// Shipped org defaults: the standard 64- and 32-bit install roots plus
// ProgramData. See BuildDefaultRoots for why ProgramData is included and
// why it should be narrowed later. Resolved from the environment so a
// non-C: system drive still matches.
public static IReadOnlyList<string> DefaultApprovedRoots { get; } =
BuildDefaultRoots();
@@ -222,12 +224,37 @@ public sealed class LaunchPathPolicy
return full.TrimEnd(Path.DirectorySeparatorChar);
}
// REVISIT: ProgramData is deliberately broad here and should be narrowed.
//
// Shop-floor apps do not all install to Program Files. MTConnect lives at
// ProgramData\udc\mtconnect_udc\agent and starts through run.bat, so with
// only the Program Files roots approved, every server-provisioned launch
// path for it was rejected as "outside every approved launch root". That
// was previously masked because a local appsettings.json launchpaths entry
// bypasses validation entirely; now that launch paths are server
// authoritative, the root list is the only gate and it has to cover the
// real install locations.
//
// WARNING: ProgramData is world-writable by default on Windows. A
// non-administrator who can write under an approved ProgramData subtree can
// plant a .bat/.cmd/.exe there, and the agent runs launch paths as SYSTEM.
// The other gates still apply (the path must come from a monitor row an
// admin configured, must exist, must be a launchable extension, and is
// re-checked after symlink/junction resolution), so this is not an open
// door - but it is a wider blast radius than Program Files, whose ACLs
// require elevation to write.
//
// Narrow this to the specific subtrees in use (ProgramData\udc rather than
// all of ProgramData) once the full set of non-Program-Files install
// locations across the fleet is confirmed. A site can already do that today
// via AgentConfig.ApprovedLaunchRoots, which REPLACES this list.
private static IReadOnlyList<string> BuildDefaultRoots()
{
var candidates = new[]
{
Environment.GetFolderPath(Environment.SpecialFolder.ProgramFiles),
Environment.GetFolderPath(Environment.SpecialFolder.ProgramFilesX86),
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData),
};
return candidates
.Where(path => !string.IsNullOrWhiteSpace(path))

View File

@@ -231,9 +231,11 @@ public sealed class TelemetryWorker : BackgroundService
{
// Refresh the watch list from server policy BEFORE collecting so a
// policy-attached monitor (added in the WebUI, never in appsettings.json)
// is enumerated and reported this cycle. Server entries merge over the
// local list; a failed fetch returns null and Merge keeps the local list,
// so monitoring never blanks on a transient blip.
// is enumerated and reported this cycle. The server list REPLACES the
// local one, so detaching a monitor in the WebUI actually stops it being
// watched even on endpoints whose installer baked it into
// appsettings.json. A failed fetch returns null and Merge keeps the
// current list, so monitoring never blanks on a transient blip.
var resolved = await client.FetchResolvedMonitorsAsync(
Environment.MachineName, token);
if (resolved is not null)

View File

@@ -90,7 +90,8 @@ services:
flask plugin upgrade-all &&
flask seed permissions &&
flask seed settings &&
flask seed reference-data
flask seed reference-data &&
flask seed shipped-scripts
volumes:
- ./plugins:/app/plugins:ro
- ./scripts:/app/scripts:ro

View File

@@ -171,11 +171,6 @@ before signing (section 4).
Flask cannot register blueprints after the first request, any adopt or
enable takes full effect only on restart, and the UI says so. There is no
"install button" that pretends otherwise.
AMENDED by ADR-020: the restart requirement is unchanged, but on gunicorn the
worker now initiates it rather than the operator. The honesty requirement is
kept and made explicit - the UI holds a plugin at "starting" until several
distinct workers confirm it is live, and on runtimes where no self-recycle
happens (Windows/IIS) it still tells the operator to restart.
### 4. Trust model: verify at adopt AND at every load and migrate
@@ -339,9 +334,6 @@ against the real code, not the idealized layout:
until the deferred baseline re-org ADR.
- Restarts remain required after adopt/enable (Flask blueprint constraint);
the marketplace UX is honest about it rather than working around it.
See ADR-020: the restart is now self-initiated per worker on gunicorn. The
constraint and the honesty both stand; only the manual step is removed, and
only where the mechanism is proven.
- Boot adds a signature + tree-hash check per enabled plugin (milliseconds,
but nonzero).

View File

@@ -1,100 +0,0 @@
<!-- docmeta
purpose: explanation
audience: developer
status: current
summary: Why enabling a plugin no longer needs a manual restart, and where it still does.
loadwith: adr/ADR-013-plugin-catalog-and-lean-builds.md
-->
# ADR-020: Worker self-recycle when the enabled-plugin set changes
Status: Accepted
Date: 2026-09-20
Amends: ADR-013 (see "Relationship to ADR-013")
## Context
Enabling a plugin could not make its routes live in a running process. Flask
refuses `register_blueprint` once the app has served its first request, and even
without that rule each gunicorn worker holds its own app object, so registering
in the worker that served the toggle would leave the other workers answering 404
for the same URL. The operator therefore had to restart the app by hand.
That restart was not merely inconvenient. Because every worker also holds its own
`PluginRegistry`, loaded once at boot, the window between a toggle and a restart
was a window in which workers disagreed about reality:
- a dependency check refused a plugin whose dependency another worker had just
enabled
- a whole-file registry write from a stale snapshot erased other workers'
changes, and the API answered 200 while doing it
- a health check that compared the registry against loaded plugins reported "no
restart needed" while the routes 404'd
## Decision
Flask's constraint stands. A plugin toggle still requires each serving process to
restart. What changes is who initiates it.
1. Each worker fingerprints `instance/plugins.json` (content hash) at boot.
2. After serving a response, if the fingerprint has changed, the worker exits
gracefully and its supervisor starts a replacement.
3. `instance/plugins.json` is the single cross-worker source of truth for the
enabled set. It is written only through the locked merge-write in
`PluginRegistry._save`. Per-worker registries are caches and must be
refreshed from disk before any decision is made from them.
4. Recycles are staggered per worker, so the pool rolls rather than dropping at
once, and are bounded by the supervisor's graceful timeout.
5. This applies only where a supervisor will return the process AND the exit
path has been exercised. Today that is gunicorn. It never applies under the
development server.
6. `/setup/health` describes the process that answered it. Callers must not
infer fleet state from a single reply; confirm across several distinct
workers.
## This is a convergence mechanism, not a security control
Disabling a plugin has never stopped its code being imported. `GET /api/plugins`
instantiates every plugin class on disk, core request handlers import plugin
models gated on `ImportError` rather than on enabled state, and ADR-013:23 says
plainly that "Disabled is not absent". Containment comes from the lean build
(ADR-013) and from signature enforcement (PLUGIN-SIGNING.md).
A stale worker briefly serving a disabled plugin's routes is therefore a user
experience artifact, not a breach, and nothing here should be cited as though it
were a boundary.
## Relationship to ADR-013
ADR-013 states that an enable "takes full effect only on restart, and the UI says
so. There is no 'install button' that pretends otherwise", and that "the
marketplace UX is honest about it rather than working around it".
That principle is kept, not reversed. The restart still happens; the process
still reloads through the ordinary boot path, so verification and the import
guard apply exactly as on a manual restart. What is removed is the manual step,
on the one runtime where the mechanism is proven.
The honesty requirement is now explicit rather than implicit:
- Where a recycle will happen, the UI holds the plugin at "starting" until
several distinct workers confirm it is live, and only then says "ready".
- Where a recycle will NOT happen, the UI keeps the old message and the operator
restarts. Saying "restarting" on a runtime that does not is worse than the
manual step it replaced.
## Consequences
- A toggle takes effect in seconds without operator action, on gunicorn.
- Worker disagreement about the enabled set becomes transient rather than
standing, which removes a class of lost-update and stale-read bugs.
- Windows/IIS (waitress behind HttpPlatformHandler) is single-process and
unproven here, so it keeps the manual restart. Adding it requires testing on
the Windows VM and handling HttpPlatformHandler's `rapidFailsPerMinute`, which
stops relaunching after repeated exits.
- A worker recycling is indistinguishable from a worker dying, so the supervisor
must be configured to replace workers. All shipped compose files and the
Kubernetes manifest run gunicorn and satisfy this.
- Migrations run on their own connection with a bounded `lock_wait_timeout`, so
a DDL that cannot take its metadata lock fails that one enable instead of
queueing every later reader behind it.

View File

@@ -31,15 +31,14 @@ anything else.
| [009](ADR-009-frontend-plugin-gating.md) | What happens when someone opens a URL belonging to a disabled plugin? | In force |
| [010](ADR-010-frontend-plugin-hooks.md) | How does a plugin add UI without editing core Vue files? | In force |
| [011](ADR-011-machines-rename.md) | Why is the code called `machines` and what happened to `machinetypes`? | In force |
| [012](ADR-012-geenforce-manifest-ownership.md) | Does ShopDB own GE-Enforce manifests, and does it own the engine? | **Record only on this line.** The plugin it decided is not bundled here; ADR-013 and the API-token resource scopes trace back to it |
| [012](ADR-012-geenforce-manifest-ownership.md) | Does ShopDB own GE-Enforce manifests, and does it own the engine? | In force |
| [013](ADR-013-plugin-catalog-and-lean-builds.md) | How does a site ship only the plugins it chose, and how is plugin code trusted? | In force |
| [014](ADR-014-schema-lean-per-site.md) | Does a lean site still create tables for plugins it never installed? | In force |
| [015](ADR-015-site-specific-configuration.md) | Where may one site's hostnames, shares and defaults live? | In force |
| [016](ADR-016-credential-delivery.md) | How does a bay get a controller credential without a cleartext file on a share? | **Superseded by [019](ADR-019-per-pc-endpoint-credentials.md), and never built.** Do not build against it |
| [017](ADR-017-buildings-and-levels.md) | How does the floor map hold more than one floor and more than one building? | In force |
| [018](ADR-018-metrology-instruments-are-machines.md) | Where do CMMs and vision systems live now that `measuringtools` is retired? | In force, and this line is the one it describes: there is no measuring-tool asset class |
| [019](ADR-019-per-pc-endpoint-credentials.md) | How does the server know WHICH PC is calling, and what replaced the fleet-wide key? | In force |
| [020](ADR-020-worker-self-recycle-on-plugin-change.md) | How does enabling a plugin take effect without the operator restarting the app? | A worker fingerprints plugins.json at boot and exits when it changes; the supervisor replaces it. Amends [013](ADR-013-plugin-catalog-and-lean-builds.md) |
| [018](ADR-018-metrology-instruments-are-machines.md) | Where do CMMs and vision systems live now that `measuringtools` is retired? | In force |
| [019](ADR-019-per-pc-endpoint-credentials.md) | How does the server know WHICH PC is calling, and what replaced the fleet-wide key? | In force. Built, not yet merged into this tree |
Related decisions that are not ADRs, kept as records of why something was built:
[docs/proposals](../proposals/README.md).

File diff suppressed because it is too large Load Diff

View File

@@ -6691,46 +6691,6 @@
"summary": "The generated OpenAPI 3.1 spec (machine + LLM readable).",
"description": "The generated OpenAPI 3.1 spec (machine + LLM readable).\n\n**Auth:** none (reviewed: named in the public-read or mutation sweep)\n\n**Params:** not documented\n\n**Example:**\n```\ncurl http://localhost:5001/api/docs/openapi.json\n```",
"security": [],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
}
},
"parameters": [
{
"name": "tag",
"in": "query",
"required": false,
"schema": {
"type": "string"
}
}
]
}
},
"/api/docs/operations.txt": {
"get": {
"tags": [
"core-docs"
],
"summary": "Every operation, one line each: method, path, tag, auth, summary.",
"description": "Every operation, one line each: method, path, tag, auth, summary.\n\n**Auth:** none (reviewed: named in the public-read or mutation sweep)\n\n**Params:** not documented\n\n**Example:**\n```\ncurl http://localhost:5001/api/docs/operations.txt\n```",
"security": [],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
@@ -12724,46 +12684,6 @@
}
}
},
"/api/setup/checklist": {
"get": {
"tags": [
"core-setup"
],
"summary": "What is configured and what is not, with a link to each fix.",
"description": "What is configured and what is not, with a link to each fix.\n\n**Auth:** jwt + role:admin\n\n**Params:** not documented\n\n**Example:**\n```\ncurl http://localhost:5001/api/setup/checklist -H \"Authorization: Bearer $TOK\"\n```",
"security": [
{
"bearerAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
}
}
}
},
"/api/setup/complete": {
"post": {
"tags": [
@@ -12858,46 +12778,6 @@
}
}
},
"/api/setup/health": {
"get": {
"tags": [
"core-setup"
],
"summary": "Post-setup reality check, including whether a restart is outstanding.",
"description": "Post-setup reality check, including whether a restart is outstanding.\n\n**Auth:** jwt + role:admin\n\n**Params:** not documented\n\n**Example:**\n```\ncurl http://localhost:5001/api/setup/health -H \"Authorization: Bearer $TOK\"\n```",
"security": [
{
"bearerAuth": []
}
],
"responses": {
"200": {
"description": "Success. Body is the success_response envelope: {status, data, meta}.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SuccessEnvelope"
}
}
}
},
"default": {
"description": "Error. Body is the error envelope; the code and message are nested under data.error.",
"content": {
"application/json": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
},
"401": {
"description": "Missing or invalid credentials."
},
"403": {
"description": "Authenticated, but not permitted."
}
}
}
},
"/api/setup/needs-admin": {
"get": {
"tags": [

View File

@@ -1246,16 +1246,6 @@ export const setupApi = {
seedStarter() {
return api.post('/setup/seed-starter')
},
// What is configured and what is not, with a route to each fix. Backs the
// standing Setup Checklist page as well as the wizard's review step.
checklist() {
return api.get('/setup/checklist')
},
// Account counts plus whether plugins.json and the running process disagree,
// which is the only way to tell that a plugin toggle needs a restart.
health() {
return api.get('/setup/health')
},
complete() {
return api.post('/setup/complete')
}

View File

@@ -14,21 +14,6 @@ export function isSetupLoaded() {
return loaded.value
}
// Force the next navigation to re-read setup state.
//
// WHY THIS IS NEEDED. The router guard only calls refreshSetupState() when
// isSetupLoaded() is false, so the answer is fetched once per PAGE LOAD. Signing
// out and back in is not a page load: the module keeps `loaded` true and the
// stale value with it. An admin already browsing when setup state changes - or
// anyone signing in on a tab that was open beforehand - therefore never gets
// steered into the wizard, and a hard refresh is the only way out. Call this on
// login and logout so the next navigation asks again.
export function resetSetupState() {
loaded.value = false
setupComplete.value = true // optimistic until re-read, so nothing flashes
setupSkipped.value = false // a new session should be offered the wizard
}
export async function refreshSetupState() {
try {
const response = await settingsApi.get('setup_complete')

View File

@@ -1,297 +1,288 @@
/**
* Core routes - Dashboard, Search, Map, Reports, Settings, Employees
*/
export default [
{
path: '',
name: 'dashboard',
component: () => import('../../views/Dashboard.vue')
},
{
path: 'search',
name: 'search',
component: () => import('../../views/SearchResults.vue')
},
{
path: 'map',
name: 'map',
component: () => import('../../views/MapView.vue')
},
{
path: 'map/editor',
name: 'map-editor',
component: () => import('../../views/MapEditor.vue'),
meta: { requiresAuth: true }
},
{
path: 'reports',
name: 'reports',
component: () => import('../../views/reports/ReportsIndex.vue')
},
// Settings
{
path: 'settings',
name: 'settings',
component: () => import('../../views/settings/SettingsIndex.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/catalog',
name: 'model-catalog',
component: () => import('../../views/settings/ModelCatalogHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: 'vendors',
name: 'model-catalog-vendors',
component: () => import('../../views/vendors/VendorsList.vue')
},
{
path: 'types',
name: 'model-catalog-types',
component: () => import('../../views/settings/ModelTypesList.vue')
},
{
path: 'models',
name: 'model-catalog-models',
component: () => import('../../views/settings/ModelsList.vue')
}
// Plugin settings routes (e.g., Operating Systems from computers plugin) are added
// dynamically by ModelCatalogHub via router.addRoute() from /api/plugins/settings-routes.
// This allows plugins to extend settings hubs without breaking lean builds.
// This hub is the ONLY one that does it: no other hub page below fetches
// or registers plugin routes, so a plugin declaring their hub id gets a
// tab nobody draws. The server knows the same list
// (SETTINGS_HUBS_WITH_PLUGIN_TABS in shopdb/plugins/base.py) and drops
// anything outside it; adding a hub means writing the code into that
// hub's .vue file AND adding it there. See ADR-019.
]
},
{
path: 'settings/assetconfig',
name: 'asset-config',
component: () => import('../../views/settings/AssetConfigHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: 'statuses',
name: 'asset-config-statuses',
component: () => import('../../views/settings/StatusesList.vue')
},
{
path: 'relationships',
name: 'asset-config-relationships',
component: () => import('../../views/settings/RelationshipTypesList.vue')
},
{
path: 'fields',
name: 'asset-config-fields',
component: () => import('../../views/settings/CustomFieldsList.vue')
},
{
path: 'identifiers',
name: 'asset-config-identifiers',
component: () => import('../../views/settings/AssetIdentifiersSettings.vue')
}
]
},
{
path: 'settings/security',
name: 'security',
component: () => import('../../views/settings/SecurityHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: 'authentication',
name: 'security-authentication',
component: () => import('../../views/settings/AuthenticationSettings.vue')
},
{
path: 'tokens',
name: 'security-tokens',
component: () => import('../../views/settings/ApiTokensList.vue')
},
{
path: 'users',
name: 'security-users',
component: () => import('../../views/settings/UsersList.vue')
},
{
path: 'audit',
name: 'security-audit',
component: () => import('../../views/settings/AuditSettings.vue')
},
{
path: 'auditlogs',
name: 'security-auditlogs',
component: () => import('../../views/settings/AuditLogs.vue')
}
]
},
{
path: 'settings/siteconfig',
name: 'site-config',
component: () => import('../../views/settings/SiteConfigHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: 'site',
name: 'site-config-site',
component: () => import('../../views/settings/SiteSettings.vue')
},
{
path: 'floormap',
name: 'site-config-floormap',
component: () => import('../../views/settings/FloorMapSettings.vue')
},
{
path: 'branding',
name: 'site-config-branding',
component: () => import('../../views/settings/BrandingSettings.vue')
},
{
path: 'imports',
name: 'site-config-imports',
component: () => import('../../views/settings/ImportSiteAssets.vue')
}
]
},
{
path: 'settings/integrations',
name: 'integrations',
component: () => import('../../views/settings/IntegrationsHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/servicenow',
name: 'servicenow-settings',
component: () => import('../../views/settings/ServiceNowSettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/locations',
name: 'locations-hub',
component: () => import('../../views/settings/LocationsHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: 'list',
name: 'locations-hub-locations',
component: () => import('../../views/settings/LocationsList.vue')
},
{
path: 'businessunits',
name: 'locations-hub-businessunits',
component: () => import('../../views/settings/BusinessUnitsList.vue')
}
]
},
{
path: 'settings/network',
name: 'network-hub',
component: () => import('../../views/settings/NetworkHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: []
},
{
path: 'settings/supportteams',
name: 'supportteams',
component: () => import('../../views/settings/SupportTeamsList.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/dashboarddefaults',
name: 'dashboarddefaults',
component: () => import('../../views/settings/DashboardDefaultsList.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/locationtypes',
name: 'location-types',
component: () => import('../../views/settings/LocationTypesList.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/assettypes',
component: () => import('../../views/settings/AssetTypeHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: '',
name: 'asset-types-hub',
component: () => import('../../views/settings/asset-types/TypesTabWrapper.vue'),
props: route => ({ assettypeid: route.params.assettypeid || route.params.assettypeid })
},
{
path: ':assettypeid/subtypes',
name: 'asset-types-hub-subtypes',
component: () => import('../../views/settings/asset-types/TypesTabWrapper.vue'),
props: route => ({ assettypeid: route.params.assettypeid })
},
]
},
// The old monolithic System Settings tab page is split into one routed page
// per section (below). This redirect keeps old bookmarks working: bare
// /settings/system and every /settings/system?tab=<key> land on the right page.
{
path: 'settings/system',
name: 'system-settings',
redirect: to => {
const tabRedirects = {
integrations: '/settings/servicenow',
branding: '/settings/branding',
printing: '/settings/printing',
email: '/settings/email',
audit: '/settings/audit',
auth: '/settings/authentication',
identifiers: '/settings/assetidentifiers',
search: '/settings/globalsearch',
map: '/settings/floormap',
pctype: '/settings/pctypemapping'
}
// vue-router carries the original ?tab= query onto the target; harmless
// since the new pages ignore it, and the path is what routes the view.
return tabRedirects[to.query.tab] || '/settings/servicenow'
},
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/printing',
name: 'printing-settings',
component: () => import('../../views/settings/PrintingSettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/email',
name: 'email-settings',
component: () => import('../../views/settings/EmailSettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/globalsearch',
name: 'global-search-settings',
component: () => import('../../views/settings/GlobalSearchSettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/plugins',
name: 'plugins',
component: () => import('../../views/settings/PluginsList.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/shopdbbackups',
name: 'shopdbbackups',
component: () => import('../../views/settings/ShopdbBackupsList.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
// The other half of the first-run wizard. The wizard asks only for what a
// site cannot run without; this is where the rest stays visible instead of
// being buried across a dozen settings pages.
path: 'settings/setupchecklist',
name: 'setup-checklist',
component: () => import('../../views/settings/SetupChecklist.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
}
]
/**
* Core routes - Dashboard, Search, Map, Reports, Settings, Employees
*/
export default [
{
path: '',
name: 'dashboard',
component: () => import('../../views/Dashboard.vue')
},
{
path: 'search',
name: 'search',
component: () => import('../../views/SearchResults.vue')
},
{
path: 'map',
name: 'map',
component: () => import('../../views/MapView.vue')
},
{
path: 'map/editor',
name: 'map-editor',
component: () => import('../../views/MapEditor.vue'),
meta: { requiresAuth: true }
},
{
path: 'reports',
name: 'reports',
component: () => import('../../views/reports/ReportsIndex.vue')
},
// Settings
{
path: 'settings',
name: 'settings',
component: () => import('../../views/settings/SettingsIndex.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/catalog',
name: 'model-catalog',
component: () => import('../../views/settings/ModelCatalogHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: 'vendors',
name: 'model-catalog-vendors',
component: () => import('../../views/vendors/VendorsList.vue')
},
{
path: 'types',
name: 'model-catalog-types',
component: () => import('../../views/settings/ModelTypesList.vue')
},
{
path: 'models',
name: 'model-catalog-models',
component: () => import('../../views/settings/ModelsList.vue')
}
// Plugin settings routes (e.g., Operating Systems from computers plugin) are added
// dynamically by ModelCatalogHub via router.addRoute() from /api/plugins/settings-routes.
// This allows plugins to extend settings hubs without breaking lean builds.
// This hub is the ONLY one that does it: no other hub page below fetches
// or registers plugin routes, so a plugin declaring their hub id gets a
// tab nobody draws. The server knows the same list
// (SETTINGS_HUBS_WITH_PLUGIN_TABS in shopdb/plugins/base.py) and drops
// anything outside it; adding a hub means writing the code into that
// hub's .vue file AND adding it there. See ADR-019.
]
},
{
path: 'settings/assetconfig',
name: 'asset-config',
component: () => import('../../views/settings/AssetConfigHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: 'statuses',
name: 'asset-config-statuses',
component: () => import('../../views/settings/StatusesList.vue')
},
{
path: 'relationships',
name: 'asset-config-relationships',
component: () => import('../../views/settings/RelationshipTypesList.vue')
},
{
path: 'fields',
name: 'asset-config-fields',
component: () => import('../../views/settings/CustomFieldsList.vue')
},
{
path: 'identifiers',
name: 'asset-config-identifiers',
component: () => import('../../views/settings/AssetIdentifiersSettings.vue')
}
]
},
{
path: 'settings/security',
name: 'security',
component: () => import('../../views/settings/SecurityHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: 'authentication',
name: 'security-authentication',
component: () => import('../../views/settings/AuthenticationSettings.vue')
},
{
path: 'tokens',
name: 'security-tokens',
component: () => import('../../views/settings/ApiTokensList.vue')
},
{
path: 'users',
name: 'security-users',
component: () => import('../../views/settings/UsersList.vue')
},
{
path: 'audit',
name: 'security-audit',
component: () => import('../../views/settings/AuditSettings.vue')
},
{
path: 'auditlogs',
name: 'security-auditlogs',
component: () => import('../../views/settings/AuditLogs.vue')
}
]
},
{
path: 'settings/siteconfig',
name: 'site-config',
component: () => import('../../views/settings/SiteConfigHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: 'site',
name: 'site-config-site',
component: () => import('../../views/settings/SiteSettings.vue')
},
{
path: 'floormap',
name: 'site-config-floormap',
component: () => import('../../views/settings/FloorMapSettings.vue')
},
{
path: 'branding',
name: 'site-config-branding',
component: () => import('../../views/settings/BrandingSettings.vue')
},
{
path: 'imports',
name: 'site-config-imports',
component: () => import('../../views/settings/ImportSiteAssets.vue')
}
]
},
{
path: 'settings/integrations',
name: 'integrations',
component: () => import('../../views/settings/IntegrationsHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/servicenow',
name: 'servicenow-settings',
component: () => import('../../views/settings/ServiceNowSettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/locations',
name: 'locations-hub',
component: () => import('../../views/settings/LocationsHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: 'list',
name: 'locations-hub-locations',
component: () => import('../../views/settings/LocationsList.vue')
},
{
path: 'businessunits',
name: 'locations-hub-businessunits',
component: () => import('../../views/settings/BusinessUnitsList.vue')
}
]
},
{
path: 'settings/network',
name: 'network-hub',
component: () => import('../../views/settings/NetworkHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: []
},
{
path: 'settings/supportteams',
name: 'supportteams',
component: () => import('../../views/settings/SupportTeamsList.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/dashboarddefaults',
name: 'dashboarddefaults',
component: () => import('../../views/settings/DashboardDefaultsList.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/locationtypes',
name: 'location-types',
component: () => import('../../views/settings/LocationTypesList.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/assettypes',
component: () => import('../../views/settings/AssetTypeHub.vue'),
meta: { requiresAuth: true, requiresAdmin: true },
children: [
{
path: '',
name: 'asset-types-hub',
component: () => import('../../views/settings/asset-types/TypesTabWrapper.vue'),
props: route => ({ assettypeid: route.params.assettypeid || route.params.assettypeid })
},
{
path: ':assettypeid/subtypes',
name: 'asset-types-hub-subtypes',
component: () => import('../../views/settings/asset-types/TypesTabWrapper.vue'),
props: route => ({ assettypeid: route.params.assettypeid })
},
]
},
// The old monolithic System Settings tab page is split into one routed page
// per section (below). This redirect keeps old bookmarks working: bare
// /settings/system and every /settings/system?tab=<key> land on the right page.
{
path: 'settings/system',
name: 'system-settings',
redirect: to => {
const tabRedirects = {
integrations: '/settings/servicenow',
branding: '/settings/branding',
printing: '/settings/printing',
email: '/settings/email',
audit: '/settings/audit',
auth: '/settings/authentication',
identifiers: '/settings/assetidentifiers',
search: '/settings/globalsearch',
map: '/settings/floormap',
pctype: '/settings/pctypemapping'
}
// vue-router carries the original ?tab= query onto the target; harmless
// since the new pages ignore it, and the path is what routes the view.
return tabRedirects[to.query.tab] || '/settings/servicenow'
},
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/printing',
name: 'printing-settings',
component: () => import('../../views/settings/PrintingSettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/email',
name: 'email-settings',
component: () => import('../../views/settings/EmailSettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/globalsearch',
name: 'global-search-settings',
component: () => import('../../views/settings/GlobalSearchSettings.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/plugins',
name: 'plugins',
component: () => import('../../views/settings/PluginsList.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
},
{
path: 'settings/shopdbbackups',
name: 'shopdbbackups',
component: () => import('../../views/settings/ShopdbBackupsList.vue'),
meta: { requiresAuth: true, requiresAdmin: true }
}
]

View File

@@ -1,6 +1,5 @@
import { defineStore } from 'pinia'
import { authApi, employeesApi } from '../api'
import { resetSetupState } from '../composables/setupState'
export const useAuthStore = defineStore('auth', {
state: () => ({
@@ -44,10 +43,6 @@ export const useAuthStore = defineStore('auth', {
localStorage.setItem('token', access_token)
localStorage.setItem('refreshToken', refresh_token)
await this.enrichFromDirectory()
// Signing in is not a page load, so the router guard would otherwise
// keep whatever setup state it cached earlier in this tab and skip
// steering a fresh admin into the wizard.
resetSetupState()
return { success: true }
} catch (error) {
@@ -69,9 +64,6 @@ export const useAuthStore = defineStore('auth', {
localStorage.removeItem('token')
localStorage.removeItem('refreshToken')
localStorage.removeItem('user')
// The next user to sign in on this tab gets a fresh answer, not this
// one's. Also clears a session-only "skip for now".
resetSetupState()
},
// Clear the forced-password-change flag after a successful change so the

View File

@@ -3,7 +3,7 @@
<div class="wizard-card">
<header class="wizard-head">
<h1>Set up ShopDB</h1>
<p class="wizard-sub">The few things a site cannot run without. Everything else is on the setup checklist afterwards.</p>
<p class="wizard-sub">A few quick steps to get this site ready.</p>
<ol class="wizard-steps">
<li v-for="(s, i) in steps" :key="s.key" :class="{ active: i === step, done: i < step }">
<span class="step-num">{{ i < step ? '&check;' : i + 1 }}</span>
@@ -13,25 +13,10 @@
</header>
<section class="wizard-body">
<!-- Administrator -->
<div v-if="current.key === 'admin'">
<h2>Administrator</h2>
<p class="hint">You are signed in as an administrator, so the account this site was claimed with already exists. This step just confirms what is there.</p>
<dl class="fact-list">
<div><dt>Accounts</dt><dd>{{ health.users ?? '-' }}</dd></div>
<div><dt>Administrators</dt><dd>{{ health.admins ?? '-' }}</dd></div>
</dl>
<p v-if="health.admins === 1" class="warn-note">
Only one administrator exists. If that account is lost, nobody can administer this site.
Consider adding a second before going live.
</p>
<p class="hint">Add or change accounts under <router-link to="/settings/security/users">Settings &gt; Users</router-link>.</p>
</div>
<!-- Site identity -->
<div v-else-if="current.key === 'site'">
<h2>Site identity</h2>
<p class="hint">Used for the dashboard header, QR codes and every absolute link the app generates.</p>
<!-- Site -->
<div v-if="current.key === 'site'">
<h2>Site details</h2>
<p class="hint">Used for the dashboard header and QR / absolute links.</p>
<div class="form-group">
<label>Facility name</label>
<input v-model="form.facility_name" type="text" class="form-control" placeholder="Main Plant" />
@@ -40,129 +25,24 @@
<label>Site base URL <span class="hint">(blank = use the browsing origin)</span></label>
<input v-model="form.site_base_url" type="url" class="form-control" placeholder="https://shopdb.example.net" />
</div>
<div class="form-group">
<!-- Every stored timestamp is rendered against this. Left unset it
falls back to the server's clock, which is rarely what a site
means once the server and the floor are in different places. -->
<label>Site timezone</label>
<select v-model="form.site_timezone" class="form-control">
<option value="">Use the server timezone</option>
<option v-for="tz in timezones" :key="tz" :value="tz">{{ tz }}</option>
</select>
</div>
<div class="form-group">
<label>Site logo <span class="hint">(optional)</span></label>
<input type="file" accept="image/*" @change="uploadSiteLogo($event)" :disabled="logoUploading" />
<input v-model="logoFilename" type="text" class="form-control" placeholder="No file selected" readonly />
<!-- Preview on the sidebar colour, which is where this logo
actually lives. --sidebar-bg is the same dark navy in every
theme, so there is no light-background case to show. The login
box is the one light surface it reaches, and .login-logo
already inverts it there. -->
<figure v-if="siteLogo" class="logo-chip">
<img :src="withBase(siteLogo)" alt="site logo on the sidebar background" />
<figcaption>As shown in the sidebar</figcaption>
</figure>
<small class="hint">Skip to keep the shipped branding. Changeable later under Settings &gt; Branding.</small>
</div>
</div>
<!-- Authentication -->
<div v-else-if="current.key === 'auth'">
<h2>Authentication</h2>
<p class="hint">How people sign in. Local accounts always work; turn on SAML to use your identity provider.</p>
<label class="check-row">
<input type="checkbox" v-model="form.saml_enabled" />
<span>Enable SAML single sign-on</span>
</label>
<div v-if="form.saml_enabled" class="config-block">
<div class="form-group">
<label>Entity ID</label>
<input v-model="form.saml_entity_id" type="text" class="form-control" placeholder="shopdb" />
</div>
<div class="form-group">
<label>ACS URL <span class="hint">(where the IdP posts back)</span></label>
<input v-model="form.saml_acs_url" type="url" class="form-control" :placeholder="suggestedAcsUrl" />
</div>
<div class="form-group">
<label>IdP metadata URL</label>
<input v-model="form.saml_idp_metadata_url" type="url" class="form-control" />
</div>
<div class="form-group">
<label>Admin group <span class="hint">(group whose members become admins)</span></label>
<input v-model="form.saml_admin_group" type="text" class="form-control" />
</div>
<label class="check-row">
<input type="checkbox" v-model="form.saml_auto_create_users" />
<span>Create accounts automatically on first SAML login</span>
</label>
<label class="check-row">
<input type="checkbox" v-model="form.saml_allow_local_login" />
<span>Keep local username and password login available</span>
</label>
<!-- Turning SAML on and local login off in the same step can strand
an operator whose IdP is not finished yet, with no way back in. -->
<p v-if="!form.saml_allow_local_login" class="warn-note">
With local login off, a misconfigured IdP locks everyone out, including you.
Leave it on until a SAML sign-in has been proven to work.
</p>
<img v-if="siteLogo" :src="withBase(siteLogo)" class="wizard-map-thumb" alt="site logo" />
<small class="hint">Skip to keep the shipped GE branding. You can change this later under Settings &gt; Branding.</small>
</div>
</div>
<!-- Plugins -->
<div v-else-if="current.key === 'plugins'">
<h2>Features</h2>
<p class="hint">Turn features on or off. Each one adds its own pages, API routes and tables.
Assets, applications, locations and users are core and always present, so they are not listed here.</p>
<div v-if="recommendedOff.length" class="rec-bar">
<span><strong>{{ recommendedOff.length }}</strong> recommended feature(s) are off.</span>
<button type="button" class="btn btn-secondary btn-sm" :disabled="bulkEnabling" @click="enableRecommended">
{{ bulkEnabling ? 'Enabling...' : 'Enable all recommended' }}
</button>
</div>
<!-- Plugins are read from plugins.json at startup, so a toggle here
does not reach the running process. Say so, and verify it. -->
<div v-if="health.restartrequired" class="restart-note">
<strong>Applying changes.</strong>
<span v-if="(health.plugins?.awaitingrestart || []).length">
Enabled but not yet loaded: <code>{{ (health.plugins.awaitingrestart || []).join(', ') }}</code>.
</span>
<span v-if="(health.plugins?.stillloaded || []).length">
Disabled but still running: <code>{{ (health.plugins.stillloaded || []).join(', ') }}</code>.
</span>
<p>Each worker picks the change up after finishing what it is serving,
so the site stays up while they roll over. This clears on its own,
usually within 20 seconds.</p>
<p class="progress-line">
<span class="spinner" aria-hidden="true"></span>
{{ readyWorkers.size }} of {{ REQUIRED_CLEAN_WORKERS }} workers confirmed
<span v-if="pollElapsed">({{ pollElapsed }}s)</span>
</p>
<button v-if="pollElapsed > 45" type="button" class="btn btn-secondary btn-sm"
:disabled="checkingHealth" @click="loadHealth">
{{ checkingHealth ? 'Checking...' : 'Check again' }}
</button>
</div>
<p v-else-if="health.plugins" class="ok-note">Enabled features all match what is running.</p>
<p class="hint">Turn features on or off. Changes to routes take effect after the app restarts.</p>
<div v-if="plugins.length" class="plugin-list">
<div v-for="p in sortedPlugins" :key="p.name" class="plugin-item" :class="{ recommended: p.recommended }">
<div v-for="p in plugins" :key="p.name" class="plugin-item">
<label class="plugin-row">
<input type="checkbox" :checked="p.enabled"
:disabled="isPending(p) || bulkEnabling"
@change="togglePlugin(p, $event.target.checked)" />
<span class="plugin-name">
{{ p.displayname || p.name }}
<span v-if="p.recommended" class="rec-tag">recommended</span>
<!-- The API answers instantly but the plugin is not usable
until every worker has picked it up, so say which of the
three states this row is actually in. -->
<span v-if="isPending(p)" class="state-tag starting">starting{{ workerDots }}</span>
<span v-else-if="justReady[p.name]" class="state-tag ready">ready</span>
</span>
<input type="checkbox" :checked="p.enabled" @change="togglePlugin(p, $event.target.checked)" />
<span class="plugin-name">{{ p.displayname || p.name }}</span>
<span class="plugin-desc">{{ p.description }}</span>
</label>
@@ -178,12 +58,14 @@
Connect to your own database
</label>
<!-- Self-hosted: transparency note -->
<div v-if="pluginModes[p.provisioning_note.mode_setting] !== 'external'" class="provision-note">
<p>{{ p.provisioning_note.note }}</p>
<p class="provision-tables">Creates in shopdb: <code v-for="t in p.provisioning_note.tables" :key="t">{{ t }}</code></p>
<p v-if="p.provisioning_note.docs" class="provision-docs">Schema: <code>{{ p.provisioning_note.docs }}</code></p>
</div>
<!-- External: connection config -->
<div v-else class="config-block">
<div v-for="field in p.config_schema" :key="field.key" class="form-group">
<label>{{ field.label }}<span v-if="field.secret" class="secret-tag">secret</span></label>
@@ -200,134 +82,74 @@
</div>
<p v-else class="hint">No optional plugins found.</p>
<!-- Plugin secrets carry an envvar and stay out of the database -->
<!-- Secrets are never stored in the DB; emit .env lines to paste -->
<div v-if="envLines" class="env-block">
<p class="hint">Paste these into <code>.env</code> and restart the app (these secrets are not stored in the database):</p>
<p class="hint">Paste these into <code>.env</code> and restart the app (secrets are not stored in the database):</p>
<pre class="env-pre">{{ envLines }}</pre>
<button type="button" class="btn btn-secondary btn-sm" @click="copyEnv">Copy</button>
</div>
</div>
<!-- Essentials: mail + backup destination -->
<div v-else-if="current.key === 'essentials'">
<h2>Mail and backups</h2>
<p class="hint">Without a mail server no alert or notification can leave the site, and without a destination no backup leaves this host.</p>
<h3 class="sub-head">Outgoing mail</h3>
<label class="check-row">
<input type="checkbox" v-model="form.smtp_enabled" />
<span>Send mail through an SMTP server</span>
</label>
<div v-if="form.smtp_enabled" class="config-block">
<div class="form-row">
<div class="form-group">
<label>SMTP host</label>
<input v-model="form.smtp_host" type="text" class="form-control" placeholder="smtp.example.net" />
</div>
<div class="form-group">
<label>Port</label>
<input v-model="form.smtp_port" type="number" class="form-control" placeholder="587" />
</div>
</div>
<div class="form-group">
<label>From address</label>
<input v-model="form.smtp_from_address" type="email" class="form-control" placeholder="shopdb@example.net" />
</div>
<div class="form-group">
<label>From name <span class="hint">(optional)</span></label>
<input v-model="form.smtp_from_name" type="text" class="form-control" placeholder="ShopDB" />
</div>
<div class="form-row">
<div class="form-group">
<label>Username <span class="hint">(optional)</span></label>
<input v-model="form.smtp_username" type="text" class="form-control" autocomplete="off" />
</div>
<div class="form-group">
<label>Password <span class="hint">(optional)</span></label>
<input v-model="form.smtp_password" type="password" class="form-control" autocomplete="new-password"
:placeholder="secretsAlreadySet.smtp_password ? 'unchanged' : ''" />
</div>
</div>
<label class="check-row">
<input type="checkbox" v-model="form.smtp_use_tls" />
<span>Use STARTTLS</span>
</label>
</div>
<h3 class="sub-head">Backup destination</h3>
<!-- Floor map -->
<div v-else-if="current.key === 'map'">
<h2>Floor map</h2>
<p class="hint">Upload your facility blueprint (light and dark) for the default level, and set its native pixel size. Additional levels can be added later in Settings.</p>
<div class="form-group">
<label>Where backups are copied</label>
<select v-model="form.backup_destination_kind" class="form-control">
<option value="local">This host only</option>
<option value="smb">Windows / SMB share</option>
<option value="aws">Amazon S3</option>
</select>
<small v-if="form.backup_destination_kind === 'local'" class="hint">
Nothing leaves this machine, so losing the host loses the backups with it.
</small>
<label>Blueprint image (light theme)</label>
<input type="file" accept="image/*" @change="uploadBlueprint('light', $event)" :disabled="mapUploading" />
<input v-model="blueprintLightFilename" type="text" class="form-control" placeholder="No file selected" readonly />
<img v-if="blueprintLight" :src="withBase(blueprintLight)" class="wizard-map-thumb" alt="light blueprint" />
</div>
<div v-if="form.backup_destination_kind === 'smb'" class="config-block">
<div class="form-row">
<div class="form-group"><label>Host</label><input v-model="form.backup_smb_host" type="text" class="form-control" /></div>
<div class="form-group"><label>Share</label><input v-model="form.backup_smb_share" type="text" class="form-control" /></div>
</div>
<div class="form-group"><label>Path within the share <span class="hint">(optional)</span></label><input v-model="form.backup_smb_path" type="text" class="form-control" /></div>
<div class="form-row">
<div class="form-group"><label>Username</label><input v-model="form.backup_smb_username" type="text" class="form-control" autocomplete="off" /></div>
<div class="form-group"><label>Password</label><input v-model="form.backup_smb_password" type="password" class="form-control" autocomplete="new-password"
:placeholder="secretsAlreadySet.backup_smb_password ? 'unchanged' : ''" /></div>
</div>
<div class="form-group">
<label>Blueprint image (dark theme)</label>
<input type="file" accept="image/*" @change="uploadBlueprint('dark', $event)" :disabled="mapUploading" />
<input v-model="blueprintDarkFilename" type="text" class="form-control" placeholder="No file selected" readonly />
<img v-if="blueprintDark" :src="withBase(blueprintDark)" class="wizard-map-thumb dark" alt="dark blueprint" />
</div>
<div v-else-if="form.backup_destination_kind === 'aws'" class="config-block">
<div class="form-row">
<div class="form-group"><label>Bucket</label><input v-model="form.backup_aws_bucket" type="text" class="form-control" /></div>
<div class="form-group"><label>Region</label><input v-model="form.backup_aws_region" type="text" class="form-control" placeholder="us-east-1" /></div>
<div class="form-row">
<div class="form-group">
<label>Map width (px)</label>
<input v-model.number="form.map_width" type="number" class="form-control" placeholder="2000" />
</div>
<div class="form-group"><label>Prefix <span class="hint">(optional)</span></label><input v-model="form.backup_aws_prefix" type="text" class="form-control" /></div>
<div class="form-row">
<div class="form-group"><label>Access key id</label><input v-model="form.backup_aws_accesskeyid" type="text" class="form-control" autocomplete="off" /></div>
<div class="form-group"><label>Secret access key</label><input v-model="form.backup_aws_secretaccesskey" type="password" class="form-control" autocomplete="new-password"
:placeholder="secretsAlreadySet.backup_aws_secretaccesskey ? 'unchanged' : ''" /></div>
<div class="form-group">
<label>Map height (px)</label>
<input v-model.number="form.map_height" type="number" class="form-control" placeholder="1200" />
</div>
</div>
</div>
<!-- Review -->
<div v-else-if="current.key === 'review'">
<h2>Review</h2>
<p class="hint">Seed the lookup data this site needs, then finish. Anything still outstanding stays on the setup checklist, reachable from Settings at any time.</p>
<div class="seed-row">
<button class="btn btn-secondary" :disabled="seeding" @click="seedReference">
{{ seeding ? 'Working...' : 'Seed reference data and permissions' }}
</button>
<button class="btn btn-secondary" :disabled="seeding" @click="seedStarter">
{{ seeding ? 'Working...' : 'Seed common vendors' }}
</button>
<!-- Starter data -->
<div v-else-if="current.key === 'data'">
<h2>Reference data</h2>
<p class="hint">Seed the lookup data a new site needs: statuses, types, permissions, and a short list of common vendors. Both are idempotent - safe to run more than once.</p>
<p class="hint">This does not create any assets. The site starts empty and assets are loaded afterwards via the import API, so an empty dashboard here is expected.</p>
<div class="starter-actions">
<div>
<button class="btn btn-secondary" :disabled="seeding" @click="seedReference">
{{ seeding ? 'Working...' : 'Seed core reference data' }}
</button>
<p class="hint">Statuses, machine/location/relationship types, permissions, default settings. Run this if you installed without the CLI seed.</p>
</div>
<div>
<button class="btn btn-secondary" :disabled="seeding" @click="seedStarter">
{{ seeding ? 'Working...' : 'Add common vendors' }}
</button>
<p class="hint">Dell, HP, Lenovo, and other common hardware vendors.</p>
</div>
</div>
<p v-if="seedResult" class="ok-note">{{ seedResult }}</p>
<p class="hint">Both are idempotent, so running them twice is harmless. Neither creates assets; the site starts empty and assets arrive through the import API.</p>
<p v-if="seedResult" class="seed-result">{{ seedResult }}</p>
</div>
<h3 class="sub-head">Still to do</h3>
<p v-if="checklistLoading" class="hint">Loading...</p>
<p v-else-if="checklistError" class="warn-note">{{ checklistError }}</p>
<p v-else-if="!checklist.length" class="ok-note">Nothing outstanding. This site is fully configured.</p>
<!-- GE-Enforce needs operational config the wizard does not collect,
so point at it rather than let a site assume it is ready. -->
<!-- Finish -->
<div v-else-if="current.key === 'finish'">
<h2>All set</h2>
<p class="hint">You can change any of this later under Settings. Finish to go to the dashboard.</p>
<div v-if="geenforceEnabled" class="finish-note">
<strong>GE-Enforce is on.</strong> Two more steps before the fleet
uses it: create a service token with the geenforce scopes under
Settings &gt; API Tokens, and set the on-share export root on the
GE-Enforce page.
</div>
<ul v-else class="check-list">
<li v-for="a in checklist" :key="a.key" :class="'st-' + a.status">
<span class="st-badge">{{ statusLabel(a.status) }}</span>
<span class="st-label">{{ a.label }}</span>
<span class="st-detail">{{ a.detail }}</span>
</li>
</ul>
</div>
</section>
@@ -349,7 +171,7 @@
<script setup>
import { withBase } from '../utils/basePath'
import { ref, computed, onMounted, onBeforeUnmount, watch } from 'vue'
import { ref, computed, onMounted } from 'vue'
import { useRouter } from 'vue-router'
import { settingsApi, pluginsApi, setupApi } from '../api'
import { useToast } from '../composables/toast'
@@ -359,192 +181,37 @@ import { refreshSetupState, setupSkipped } from '../composables/setupState'
const router = useRouter()
const toast = useToast()
// Six steps, and deliberately no more. The product has well over a hundred
// settings; asking for all of them up front would be worse than asking for
// none. These are the ones a site cannot sensibly run without. Everything else
// lives on the setup checklist, which stays reachable after setup finishes.
const steps = [
{ key: 'admin', label: 'Administrator' },
{ key: 'site', label: 'Site' },
{ key: 'auth', label: 'Authentication' },
{ key: 'plugins', label: 'Features' },
{ key: 'essentials', label: 'Mail & Backups' },
{ key: 'review', label: 'Review' },
{ key: 'map', label: 'Floor Map' },
{ key: 'data', label: 'Reference Data' },
{ key: 'finish', label: 'Finish' },
]
const step = ref(0)
const current = computed(() => steps[step.value])
// Booleans are kept as real booleans here and stringified on save, because the
// settings table stores everything as text.
const form = ref({
facility_name: '', site_base_url: '', site_timezone: '',
saml_enabled: false, saml_entity_id: '', saml_acs_url: '',
saml_idp_metadata_url: '', saml_admin_group: '',
saml_auto_create_users: false, saml_allow_local_login: true,
smtp_enabled: false, smtp_host: '', smtp_port: null,
smtp_from_address: '', smtp_from_name: '', smtp_username: '',
smtp_password: '', smtp_use_tls: true,
backup_destination_kind: 'local',
backup_smb_host: '', backup_smb_share: '', backup_smb_path: '',
backup_smb_username: '', backup_smb_password: '',
backup_aws_bucket: '', backup_aws_region: '', backup_aws_prefix: '',
backup_aws_accesskeyid: '', backup_aws_secretaccesskey: '',
facility_name: '', site_base_url: '',
map_width: null, map_height: null,
})
// Written but never read back into the form. The settings API masks these on
// read, so round-tripping the mask corrupts the stored value.
const SECRET_KEYS = new Set([
'smtp_password', 'backup_smb_password', 'backup_aws_secretaccesskey',
])
const secretsAlreadySet = ref({})
const BOOLEAN_KEYS = new Set([
'saml_enabled', 'saml_auto_create_users', 'saml_allow_local_login',
'smtp_enabled', 'smtp_use_tls',
])
const plugins = ref([])
const pluginConfig = ref({}) // flat map: setting/field key -> value
const pluginModes = ref({}) // mode_setting key -> 'selfhosted' | 'external'
const saving = ref(false)
const seeding = ref(false)
const seedResult = ref('')
const mapUploading = ref(false)
const blueprintLight = ref('')
const blueprintDark = ref('')
const logoUploading = ref(false)
const siteLogo = ref('')
// A file input shows the chosen name in the browser chrome, which the kiosk-
// style full-screen wizard hides, so the operator had no confirmation that the
// file they picked was the one being uploaded.
const logoFilename = ref('')
const health = ref({})
const checkingHealth = ref(false)
// Plugins toggled in this session that have not yet shown up as live. Driven by
// /setup/health rather than a timer, so the UI says "starting" for exactly as
// long as it really is starting.
const pendingPlugins = ref(new Set())
const justReady = ref({})
const readyWorkers = ref(new Set())
const pollElapsed = ref(0)
let pollTimer = null
let pollStartedAt = 0
const workerDots = ref('')
let dotTimer = null
function isPending(plugin) {
return pendingPlugins.value.has(plugin.name)
}
function startDots() {
if (dotTimer) return
dotTimer = setInterval(() => {
workerDots.value = workerDots.value.length >= 3 ? '' : workerDots.value + '.'
}, 500)
}
function stopDots() {
if (dotTimer) { clearInterval(dotTimer); dotTimer = null }
workerDots.value = ''
}
function stopPolling() {
if (pollTimer) { clearTimeout(pollTimer); pollTimer = null }
stopDots()
pollElapsed.value = 0
}
// Poll until the pool has rolled over.
//
// A single clean reply proves nothing about the fleet: it came from ONE worker,
// and the others may still hold the old set. Each reply names the worker that
// answered, so wait until several DISTINCT workers have each reported clean.
// Counting replies instead of workers let one recycled worker answer four times
// and declare victory while an idle worker still served the old routes.
const REQUIRED_CLEAN_WORKERS = 3
const MAX_WATCH_SECONDS = 120
const MAX_CONSECUTIVE_FAILURES = 10
function beginWatching(names) {
if (!names || !names.length) return // nothing succeeded: nothing to watch
names.forEach(n => pendingPlugins.value.add(n))
pendingPlugins.value = new Set(pendingPlugins.value)
readyWorkers.value = new Set()
startDots()
if (pollTimer) return
pollStartedAt = Date.now()
let failures = 0
const tick = async () => {
pollElapsed.value = Math.round((Date.now() - pollStartedAt) / 1000)
// Checked BEFORE the request, so a poll that keeps failing still stops.
// Returning early on error used to skip this and spin indefinitely.
if (pollElapsed.value > MAX_WATCH_SECONDS || failures >= MAX_CONSECUTIVE_FAILURES) {
stopPolling()
toast.error('Still applying. Restart the app if this does not clear.')
return
}
let data = null
try {
data = (await setupApi.health()).data?.data || {}
failures = 0
} catch (err) {
failures += 1 // a worker mid-recycle can refuse a poll
}
if (data) {
health.value = data
const awaiting = new Set(data?.plugins?.awaitingrestart || [])
const stale = new Set(data?.plugins?.stillloaded || [])
// stillloaded matters as much as awaitingrestart: DISABLING a plugin is
// only finished once no worker is still serving it.
const clean = awaiting.size === 0 && stale.size === 0
if (clean && data.worker) {
readyWorkers.value = new Set([...readyWorkers.value, data.worker])
} else if (!clean) {
readyWorkers.value = new Set() // someone regressed; start counting again
}
if (readyWorkers.value.size >= REQUIRED_CLEAN_WORKERS) {
for (const name of pendingPlugins.value) {
justReady.value[name] = true
setTimeout(() => { delete justReady.value[name] }, 6000)
}
pendingPlugins.value = new Set()
stopPolling()
return
}
}
pollTimer = setTimeout(tick, 2000)
}
pollTimer = setTimeout(tick, 1500)
}
onBeforeUnmount(stopPolling)
const checklist = ref([])
// Three states, not two: an empty list means "all green", which is the opposite
// of "still loading" and must not render as a permanent Loading hint.
const checklistLoading = ref(false)
const checklistError = ref('')
// Offered as a list rather than free text so a typo cannot silently leave every
// timestamp on the server clock. Intl gives the real zone list where available.
const timezones = computed(() => {
try {
if (typeof Intl.supportedValuesOf === 'function') return Intl.supportedValuesOf('timeZone')
} catch (err) { /* not supported in this browser */ }
return ['UTC', 'America/New_York', 'America/Chicago', 'America/Denver',
'America/Los_Angeles', 'Europe/London', 'Europe/Paris', 'Asia/Tokyo']
})
// GE-Enforce needs post-setup operational config (service token + share root)
// the wizard does not collect; the Review step points there when it is on.
const geenforceEnabled = computed(() =>
plugins.value.some(p => p.name === 'geenforce' && p.enabled))
const suggestedAcsUrl = computed(() => {
const base = (form.value.site_base_url || window.location.origin).replace(/\/+$/, '')
return `${base}/api/auth/saml/acs`
})
function statusLabel(status) {
return { ok: 'done', partial: 'partial', todo: 'to do', na: 'n/a' }[status] || status
}
const blueprintLightFilename = ref('')
const blueprintDarkFilename = ref('')
async function uploadSiteLogo(event) {
const file = event.target.files[0]
@@ -566,46 +233,24 @@ async function uploadSiteLogo(event) {
}
}
// Recommended first, then alphabetical inside each group, so the baseline a
// shop-floor site needs is what an operator sees before anything optional.
// The flag comes from each plugin's manifest, not a list held here.
const sortedPlugins = computed(() =>
[...plugins.value].sort((a, b) =>
(b.recommended === true) - (a.recommended === true) ||
(a.displayname || a.name).localeCompare(b.displayname || b.name)))
const recommendedOff = computed(() =>
plugins.value.filter(p => p.recommended && !p.enabled))
const bulkEnabling = ref(false)
async function enableRecommended() {
bulkEnabling.value = true
// Sequential, not Promise.all: each call rewrites plugins.json, and
// concurrent writers would race and lose toggles.
//
// Each is also tried independently. Aborting the loop on the first failure
// left the operator with one plugin enabled, no indication which, and a
// generic error - report what actually happened instead.
const failed = []
const enabledNames = []
for (const plugin of recommendedOff.value.slice()) {
try {
await pluginsApi.setEnabled(plugin.name, true)
plugin.enabled = true
enabledNames.push(plugin.name)
} catch (err) {
failed.push(plugin.displayname || plugin.name)
}
async function uploadBlueprint(theme, event) {
const file = event.target.files[0]
if (!file) return
if (theme === 'light') blueprintLightFilename.value = file.name
else blueprintDarkFilename.value = file.name
mapUploading.value = true
try {
const { data } = await settingsApi.uploadMapBlueprint(theme, file)
const url = data?.data?.value ? `${data.data.value}?t=${Date.now()}` : ''
if (theme === 'light') blueprintLight.value = url
else blueprintDark.value = url
toast.success('Blueprint uploaded')
} catch (err) {
toast.error(apiError(err, 'Upload failed'))
} finally {
mapUploading.value = false
event.target.value = ''
}
bulkEnabling.value = false
if (failed.length) {
toast.error(`Could not enable: ${failed.join(', ')}`)
}
// Only the ones that actually succeeded. Watching the failures too marked
// them "ready" the moment health came back clean, because a plugin that was
// never enabled never appears in awaitingrestart.
beginWatching(enabledNames)
}
function modeOf(plugin) {
@@ -613,15 +258,18 @@ function modeOf(plugin) {
return key ? (pluginModes.value[key] || 'selfhosted') : null
}
// GE-Enforce needs post-setup operational config (service token + share root)
// the wizard does not collect; the Finish step points there when it is on.
const geenforceEnabled = computed(() =>
plugins.value.some(p => p.name === 'geenforce' && p.enabled))
// Enabled plugins whose external connection config should be collected right
// now: those in external mode (or with config but no mode concept).
const configurablePlugins = computed(() =>
plugins.value.filter(p => p.enabled && (p.config_schema || []).length &&
(!p.provisioning_note?.mode_setting || modeOf(p) === 'external')))
// .env lines for plugin secret fields that declare an envvar. Plugin secrets
// keep the paste-into-.env model; the core settings on the Mail and Backups
// step are stored in the settings table, which the public settings read filters.
// .env lines for secret fields that have a value entered.
const envLines = computed(() => {
const lines = []
for (const plugin of configurablePlugins.value) {
@@ -637,58 +285,18 @@ const envLines = computed(() => {
// Which settings each step owns, so Next only saves what changed on that step.
const stepSettings = {
site: ['facility_name', 'site_base_url', 'site_timezone'],
auth: ['saml_enabled', 'saml_entity_id', 'saml_acs_url', 'saml_idp_metadata_url',
'saml_admin_group', 'saml_auto_create_users', 'saml_allow_local_login'],
essentials: ['smtp_enabled', 'smtp_host', 'smtp_port', 'smtp_from_address',
'smtp_from_name', 'smtp_username', 'smtp_password', 'smtp_use_tls',
'backup_destination_kind', 'backup_smb_host', 'backup_smb_share',
'backup_smb_path', 'backup_smb_username', 'backup_smb_password',
'backup_aws_bucket', 'backup_aws_region', 'backup_aws_prefix',
'backup_aws_accesskeyid', 'backup_aws_secretaccesskey'],
}
const ALL_FORM_KEYS = Object.values(stepSettings).flat()
async function loadHealth() {
checkingHealth.value = true
try {
const { data } = await setupApi.health()
health.value = data?.data || {}
} catch (err) { /* non-fatal: the wizard still works without it */ }
finally { checkingHealth.value = false }
}
async function loadChecklist() {
checklistLoading.value = true
checklistError.value = ''
try {
const { data } = await setupApi.checklist()
checklist.value = (data?.data?.areas || []).filter(a => a.status !== 'ok' && a.status !== 'na')
} catch (err) {
checklistError.value = apiError(err, 'Could not load the setup checklist')
} finally {
checklistLoading.value = false
}
site: ['facility_name', 'site_base_url'],
map: ['map_width', 'map_height'],
}
onMounted(async () => {
for (const key of ALL_FORM_KEYS) {
// Load current values for the settings the wizard edits.
const keys = ['facility_name', 'site_base_url', 'map_width', 'map_height']
for (const key of keys) {
try {
const response = await settingsApi.get(key)
const value = response.data?.data?.value
if (value === undefined || value === null || value === '') continue
// Never load a secret into its input. The API returns stored secrets as
// a mask, and pre-filling it meant typing appended to the mask and saved
// the mask plus the new text verbatim. Leave the field blank: blank is
// "unchanged" on save, and the placeholder says a value is already set.
if (SECRET_KEYS.has(key)) {
secretsAlreadySet.value[key] = true
continue
}
form.value[key] = BOOLEAN_KEYS.has(key)
? String(value).toLowerCase() === 'true'
: value
if (value !== undefined && value !== null && value !== '') form.value[key] = value
} catch (err) { /* setting may not exist yet */ }
}
try {
@@ -725,19 +333,11 @@ onMounted(async () => {
} catch (err) {
siteLogo.value = SHIPPED_LOGO
}
await loadHealth()
})
// The review step is only honest if it reflects what the earlier steps just
// saved, so re-read rather than showing a snapshot from page load.
watch(step, (value) => {
if (steps[value].key === 'review') loadChecklist()
if (steps[value].key === 'plugins') loadHealth()
})
async function saveStep() {
// Plugins step: save each plugin's directory mode, plus non-secret external
// connection config (plugin secrets stay in .env).
// connection config (secrets stay in .env).
if (current.value.key === 'plugins') {
for (const [modeKey, modeValue] of Object.entries(pluginModes.value)) {
await settingsApi.update(modeKey, modeValue)
@@ -750,7 +350,6 @@ async function saveStep() {
await settingsApi.update(field.key, String(value))
}
}
await loadHealth()
return
}
const keys = stepSettings[current.value.key]
@@ -758,11 +357,7 @@ async function saveStep() {
for (const key of keys) {
const value = form.value[key]
if (value === null || value === undefined) continue
// Blank means "leave the stored secret alone". There is deliberately no way
// to clear one from here: clearing a credential is a destructive action and
// belongs on the settings page that owns it, not in a first-run wizard.
if (value === '' && SECRET_KEYS.has(key)) continue
await settingsApi.update(key, BOOLEAN_KEYS.has(key) ? String(!!value) : String(value))
await settingsApi.update(key, String(value))
}
}
@@ -798,8 +393,7 @@ async function togglePlugin(plugin, enabled) {
try {
await pluginsApi.setEnabled(plugin.name, enabled)
plugin.enabled = enabled
beginWatching([plugin.name])
toast.success(`${plugin.displayname || plugin.name} ${enabled ? 'enabled' : 'disabled'}.`)
toast.success(`${plugin.name} ${enabled ? 'enabled' : 'disabled'} (restart to apply route changes).`)
} catch (err) {
toast.error(apiError(err, 'Could not change plugin'))
}
@@ -810,7 +404,6 @@ async function seedReference() {
try {
const response = await setupApi.seedReference()
seedResult.value = response.data?.message || 'Reference data seeded.'
await loadChecklist()
} catch (err) {
toast.error(apiError(err, 'Could not seed reference data'))
} finally {
@@ -824,7 +417,6 @@ async function seedStarter() {
const response = await setupApi.seedStarter()
const data = response.data?.data || {}
seedResult.value = data.addedcount ? `Added ${data.addedcount}: ${data.added.join(', ')}` : 'All common vendors already present.'
await loadChecklist()
} catch (err) {
toast.error(apiError(err, 'Could not seed starter data'))
} finally {
@@ -857,54 +449,37 @@ async function finish() {
.wizard-steps li { display: flex; align-items: center; gap: 0.4rem; font-size: 0.82rem; color: var(--text-light); }
.wizard-steps li.active { color: var(--primary); font-weight: 600; }
.wizard-steps li.done { color: var(--success); }
.wizard-body { padding: 1.5rem 1.75rem; }
.sub-head { margin: 1.5rem 0 0.5rem; font-size: 0.95rem; }
.check-row { display: flex; align-items: center; gap: 0.5rem; margin: 0.5rem 0; }
.config-block { border-left: 2px solid var(--border); padding-left: 1rem; margin: 0.75rem 0 0; }
.fact-list { margin: 0.5rem 0; }
.fact-list > div { display: flex; gap: 0.75rem; padding: 0.3rem 0; }
.fact-list dt { color: var(--text-light); min-width: 9rem; margin: 0; }
.fact-list dd { margin: 0; font-weight: 600; }
.warn-note { border-left: 3px solid var(--warning, #b8860b); padding: 0.5rem 0.75rem; margin: 0.75rem 0; background: rgba(184, 134, 11, 0.08); }
.ok-note { border-left: 3px solid var(--success, #2e7d32); padding: 0.5rem 0.75rem; margin: 0.75rem 0; background: rgba(46, 125, 50, 0.08); }
.restart-note { border-left: 3px solid var(--warning, #b8860b); padding: 0.6rem 0.85rem; margin: 0 0 1rem; background: rgba(184, 134, 11, 0.08); }
.restart-note p { margin: 0.4rem 0 0.6rem; }
.seed-row { display: flex; gap: 0.6rem; flex-wrap: wrap; margin: 0.75rem 0; }
.check-list { list-style: none; padding: 0; margin: 0.5rem 0 0; }
.check-list li { display: grid; grid-template-columns: 5rem 11rem 1fr; gap: 0.6rem; align-items: baseline; padding: 0.35rem 0; border-bottom: 1px solid var(--border); }
.st-badge { font-size: 10px; text-transform: uppercase; letter-spacing: 0.4px; }
.st-todo .st-badge { color: var(--text-light); }
.st-partial .st-badge { color: var(--warning, #b8860b); font-weight: 700; }
.st-label { font-weight: 600; }
.st-detail { color: var(--text-light); font-size: 0.86rem; }
/* Logo preview. Deliberately NOT .wizard-map-thumb: that class hardcodes a
white background for blueprint previews, which made the shipped white logo
invisible. --sidebar-bg is dark in every theme, so preview against it. */
.logo-chip { margin: 0.5rem 0 0; width: fit-content; border: 1px solid var(--border); border-radius: 4px; padding: 0.6rem 0.9rem; background: var(--sidebar-bg, #00003d); display: flex; flex-direction: column; align-items: center; gap: 0.4rem; }
.logo-chip img { max-height: 54px; max-width: 170px; height: auto; display: block; }
.logo-chip figcaption { font-size: 11px; letter-spacing: 0.3px; text-transform: uppercase; color: rgba(255, 255, 255, 0.7); }
.finish-note { margin: 0.75rem 0 0; padding: 0.6rem 0.75rem; background: var(--bg); border-left: 3px solid var(--primary); border-radius: 4px; font-size: 0.85rem; }
.step-num { display: inline-flex; align-items: center; justify-content: center; width: 20px; height: 20px; border-radius: 50%; border: 1px solid currentColor; font-size: 0.72rem; }
.wizard-body { padding: 1.5rem 1.75rem; min-height: 220px; }
.wizard-body h2 { margin-top: 0; }
.hint { color: var(--text-light); font-size: 0.88rem; }
.form-group { margin-bottom: 1rem; }
.form-group label { display: block; margin-bottom: 0.3rem; font-weight: 500; }
.form-row { display: flex; gap: 1rem; }
.form-row .form-group { flex: 1; }
.plugin-list { display: flex; flex-direction: column; gap: 0.6rem; }
.plugin-item { border: 1px solid var(--border); border-radius: 6px; padding: 0.6rem 0.8rem; }
.plugin-item.recommended { border-left: 3px solid var(--primary); }
.state-tag { margin-left: 0.5rem; font-size: 10px; text-transform: uppercase; letter-spacing: 0.4px; font-weight: 600; }
.state-tag.starting { color: var(--warning, #b8860b); }
.state-tag.ready { color: var(--success, #2e7d32); }
.progress-line { display: flex; align-items: center; gap: 0.5rem; margin: 0.5rem 0 0; font-size: 0.86rem; }
.spinner { width: 12px; height: 12px; border: 2px solid var(--border); border-top-color: var(--primary); border-radius: 50%; display: inline-block; animation: spin 0.8s linear infinite; }
@keyframes spin { to { transform: rotate(360deg); } }
.rec-tag { margin-left: 0.45rem; font-size: 10px; text-transform: uppercase; letter-spacing: 0.4px; color: var(--primary); font-weight: 600; }
.rec-bar { display: flex; align-items: center; justify-content: space-between; gap: 0.75rem; flex-wrap: wrap; padding: 0.6rem 0.85rem; margin: 0 0 1rem; border-left: 3px solid var(--primary); background: var(--bg); border-radius: 4px; }
.plugin-row { display: grid; grid-template-columns: auto 10rem 1fr; gap: 0.6rem; align-items: baseline; }
.plugin-row { display: grid; grid-template-columns: auto auto 1fr; gap: 0.6rem; align-items: baseline; padding: 0.5rem 0.6rem; background: var(--bg); border-radius: 6px; }
.plugin-name { font-weight: 600; }
.plugin-desc { color: var(--text-light); font-size: 0.86rem; }
.plugin-mode { margin-top: 0.6rem; padding-left: 1.6rem; }
.mode-opt { display: flex; align-items: center; gap: 0.4rem; margin: 0.25rem 0; }
.provision-note { margin-top: 0.5rem; font-size: 0.86rem; color: var(--text-light); }
.provision-tables code, .provision-docs code { margin-right: 0.35rem; }
.secret-tag { margin-left: 0.4rem; font-size: 10px; text-transform: uppercase; letter-spacing: 0.4px; color: var(--warning, #b8860b); }
.plugin-desc { color: var(--text-light); font-size: 0.85rem; }
.plugin-mode { margin: 0.4rem 0 0.2rem 1.9rem; }
.mode-opt { display: block; font-size: 0.86rem; margin-bottom: 0.3rem; cursor: pointer; }
.mode-opt input { margin-right: 0.4rem; }
.plugin-mode .provision-note, .plugin-mode .config-block { margin-left: 0; }
.provision-note { margin: 0.3rem 0 0.2rem 1.9rem; padding: 0.6rem 0.75rem; background: var(--bg); border-left: 3px solid var(--warning); border-radius: 4px; font-size: 0.82rem; }
.finish-note { margin: 0.75rem 0 0; padding: 0.6rem 0.75rem; background: var(--bg); border-left: 3px solid var(--primary); border-radius: 4px; font-size: 0.85rem; }
.provision-note p { margin: 0 0 0.35rem; }
.provision-note p:last-child { margin-bottom: 0; }
.provision-tables code, .provision-docs code { background: var(--bg-card); border: 1px solid var(--border); border-radius: 3px; padding: 0 0.3rem; margin-right: 0.3rem; font-size: 0.78rem; }
.seed-result { margin-top: 0.75rem; color: var(--success); font-size: 0.88rem; }
.starter-actions { display: flex; flex-direction: column; gap: 1.1rem; }
.starter-actions .hint { margin: 0.35rem 0 0; }
.config-block { margin-top: 1.25rem; padding-top: 1rem; border-top: 1px solid var(--border); }
.config-title { margin: 0 0 0.75rem; font-size: 1rem; text-transform: capitalize; }
.secret-tag { margin-left: 0.5rem; font-size: 0.68rem; text-transform: uppercase; letter-spacing: 0.04em; color: var(--warning); border: 1px solid var(--warning); border-radius: 4px; padding: 0 0.3rem; }
.env-block { margin-top: 1rem; }
.env-pre { background: var(--bg); border: 1px solid var(--border); border-radius: 4px; padding: 0.6rem; overflow-x: auto; }
.env-pre { background: var(--bg); border: 1px solid var(--border); border-radius: 6px; padding: 0.75rem; overflow-x: auto; font-size: 0.82rem; margin: 0.5rem 0; }
.wizard-map-thumb { display: block; margin-top: 0.5rem; max-height: 90px; border: 1px solid var(--border); border-radius: 4px; background: #fff; }
.wizard-map-thumb.dark { background: #222; }
.wizard-foot { display: flex; justify-content: space-between; align-items: center; padding: 1rem 1.75rem; border-top: 1px solid var(--border); }
.foot-right { display: flex; align-items: center; gap: 0.75rem; }
.btn-text { background: none; border: none; color: var(--text-light); text-decoration: none; }

View File

@@ -10,17 +10,8 @@
<template v-else>
<p class="hint">
Enabling or disabling a plugin takes effect on its own: each worker
picks the change up after finishing what it is serving and restarts
itself, so the site stays up while they roll over. A plugin is usable
once its row reads <strong>ready</strong>.
Disabling a plugin removes its pages and API on the next app restart.
</p>
<div v-if="pending.size" class="rolling-note">
<span class="spinner" aria-hidden="true"></span>
Applying: <code>{{ [...pending].join(', ') }}</code>
<span class="muted">{{ readyWorkers.size }} of {{ REQUIRED_CLEAN_WORKERS }} workers confirmed</span>
<span v-if="elapsed">({{ elapsed }}s)</span>
</div>
<div class="table-container">
<table>
<thead>
@@ -33,19 +24,14 @@
</thead>
<tbody>
<tr v-for="p in plugins" :key="p.name">
<td>
{{ p.displayname || p.name }}
<span v-if="pending.has(p.name)" class="state-tag starting">starting{{ dots }}</span>
<span v-else-if="justReady[p.name]" class="state-tag ready">ready</span>
<span v-else-if="p.recommended" class="state-tag rec">recommended</span>
</td>
<td>{{ p.displayname || p.name }}</td>
<td class="mono">{{ p.version }}</td>
<td class="cell-truncate" :title="p.description">{{ p.description || '-' }}</td>
<td>
<button
class="toggle-btn"
:class="{ active: p.enabled }"
:disabled="pending.has(p.name)"
:disabled="saving"
@click="toggle(p)"
>
<span class="toggle-slider"></span>
@@ -63,21 +49,15 @@
</template>
<script setup>
import { ref, onMounted, onBeforeUnmount } from 'vue'
import { pluginsApi, setupApi } from '../../api'
import { ref, onMounted } from 'vue'
import { pluginsApi } from '../../api'
const plugins = ref([])
const contractVersion = ref('')
const loading = ref(true)
const saving = ref(false)
const error = ref('')
const message = ref('')
const pending = ref(new Set())
const justReady = ref({})
const dots = ref('')
const elapsed = ref(0)
const readyWorkers = ref(new Set())
let timer = null
let dotTimer = null
onMounted(() => load())
@@ -97,110 +77,20 @@ async function load() {
async function toggle(p) {
error.value = ''
message.value = ''
saving.value = true
try {
await pluginsApi.setEnabled(p.name, !p.enabled)
const response = await pluginsApi.setEnabled(p.name, !p.enabled)
p.enabled = !p.enabled
watchUntilLive(p.name)
message.value = response.data.message || 'Updated'
} catch (err) {
error.value = err.response?.data?.data?.error?.message || 'Failed to update plugin'
} finally {
saving.value = false
}
}
// The API answers as soon as the registry is written, but the plugin is not
// usable until every worker has recycled onto the new set. Poll /setup/health
// and hold the row in "starting" until it really is live, so the only state on
// screen is the one the fleet is actually in.
//
// Each reply names the worker that answered. Waiting for several DISTINCT
// workers to report clean is the point: counting replies instead let a single
// recycled worker answer repeatedly and declare the change complete while an
// idle worker still served the old routes.
const REQUIRED_CLEAN_WORKERS = 3
const MAX_WATCH_SECONDS = 120
const MAX_CONSECUTIVE_FAILURES = 10
function watchUntilLive(name) {
pending.value = new Set([...pending.value, name])
readyWorkers.value = new Set()
startDots()
if (timer) return
const startedAt = Date.now()
let failures = 0
const tick = async () => {
elapsed.value = Math.round((Date.now() - startedAt) / 1000)
// Checked BEFORE the request: a poll that keeps failing (expired token,
// API down) must still time out rather than spin with the spinner up.
if (elapsed.value > MAX_WATCH_SECONDS || failures >= MAX_CONSECUTIVE_FAILURES) {
error.value = 'Still applying. Restart the app if this does not clear.'
pending.value = new Set()
stopWatching()
return
}
let data = null
try {
data = (await setupApi.health()).data?.data || {}
failures = 0
} catch (err) {
failures += 1 // a worker mid-recycle may refuse one poll
}
if (data) {
const awaiting = new Set(data?.plugins?.awaitingrestart || [])
const stale = new Set(data?.plugins?.stillloaded || [])
// Both matter: enabling clears awaitingrestart, disabling clears
// stillloaded, and a toggle is not done until neither names anything.
const clean = awaiting.size === 0 && stale.size === 0
if (clean && data.worker) {
readyWorkers.value = new Set([...readyWorkers.value, data.worker])
} else if (!clean) {
readyWorkers.value = new Set()
}
if (readyWorkers.value.size >= REQUIRED_CLEAN_WORKERS) {
for (const n of pending.value) {
justReady.value[n] = true
setTimeout(() => { delete justReady.value[n] }, 6000)
}
pending.value = new Set()
stopWatching()
return
}
}
timer = setTimeout(tick, 2000)
}
timer = setTimeout(tick, 1500)
}
function startDots() {
if (dotTimer) return
dotTimer = setInterval(() => {
dots.value = dots.value.length >= 3 ? '' : dots.value + '.'
}, 500)
}
function stopWatching() {
if (timer) { clearTimeout(timer); timer = null }
if (dotTimer) { clearInterval(dotTimer); dotTimer = null }
dots.value = ''
elapsed.value = 0
}
onBeforeUnmount(stopWatching)
</script>
<style scoped>
.state-tag { margin-left: 0.5rem; font-size: 10px; text-transform: uppercase; letter-spacing: 0.4px; font-weight: 600; }
.state-tag.starting { color: var(--warning, #b8860b); }
.state-tag.ready { color: var(--success, #2e7d32); }
.state-tag.rec { color: var(--primary); }
.muted { color: var(--text-light); }
.rolling-note { display: flex; align-items: center; gap: 0.5rem; margin: 0 0 1rem; padding: 0.55rem 0.8rem;
border-left: 3px solid var(--primary); background: var(--bg); border-radius: 4px; font-size: 0.86rem; }
.spinner { width: 12px; height: 12px; border: 2px solid var(--border); border-top-color: var(--primary);
border-radius: 50%; display: inline-block; animation: spin 0.8s linear infinite; }
@keyframes spin { to { transform: rotate(360deg); } }
.mono {
font-family: 'SF Mono', 'Monaco', 'Consolas', monospace;
}

View File

@@ -1,113 +0,0 @@
<template>
<div class="setup-checklist">
<header class="page-head">
<div>
<h1>Setup checklist</h1>
<p class="hint">What this site has configured and what it has not. The first-run wizard
covers only what a site cannot run without; the rest lives here.</p>
</div>
<button class="btn btn-secondary" :disabled="loading" @click="load">
{{ loading ? 'Checking...' : 'Re-check' }}
</button>
</header>
<div v-if="counts" class="summary">
<span class="pill ok">{{ counts.ok || 0 }} done</span>
<span class="pill partial">{{ counts.partial || 0 }} partial</span>
<span class="pill todo">{{ counts.todo || 0 }} to do</span>
<span v-if="counts.na" class="pill na">{{ counts.na }} n/a</span>
</div>
<!-- A pending restart makes the rest of this page misleading: a plugin can
be enabled and still answer 404, so say it before anything else. -->
<div v-if="health.restartrequired" class="restart-note">
<strong>Applying changes.</strong>
<span v-if="(health.plugins?.awaitingrestart || []).length">
Enabled but not loaded: <code>{{ (health.plugins.awaitingrestart || []).join(', ') }}</code>.
</span>
<span v-if="(health.plugins?.stillloaded || []).length">
Disabled but still running: <code>{{ (health.plugins.stillloaded || []).join(', ') }}</code>.
</span>
<p>Those features come online within a few seconds as each worker restarts itself.</p>
</div>
<p v-if="error" class="warn-note">{{ error }}</p>
<table v-if="areas.length" class="table checklist-table">
<thead>
<tr><th>Status</th><th>Area</th><th>Detail</th><th></th></tr>
</thead>
<tbody>
<tr v-for="a in areas" :key="a.key" :class="'st-' + a.status">
<td><span class="st-badge">{{ statusLabel(a.status) }}</span></td>
<td class="st-label">{{ a.label }}<span v-if="a.inwizard" class="wiz-tag">wizard</span></td>
<td class="st-detail">{{ a.detail }}</td>
<td class="st-action">
<router-link v-if="a.status !== 'na'" :to="a.route" class="btn btn-secondary btn-sm">
{{ a.status === 'ok' ? 'Review' : 'Configure' }}
</router-link>
</td>
</tr>
</tbody>
</table>
<p v-else-if="!loading && !error" class="hint">Nothing to show.</p>
</div>
</template>
<script setup>
import { ref, onMounted } from 'vue'
import { setupApi } from '../../api'
import { apiError } from '../../utils/apiError'
const areas = ref([])
const counts = ref(null)
const health = ref({})
const loading = ref(false)
const error = ref('')
function statusLabel(status) {
return { ok: 'done', partial: 'partial', todo: 'to do', na: 'n/a' }[status] || status
}
async function load() {
loading.value = true
error.value = ''
try {
const [list, state] = await Promise.all([setupApi.checklist(), setupApi.health()])
areas.value = list.data?.data?.areas || []
counts.value = list.data?.data?.counts || null
health.value = state.data?.data || {}
} catch (err) {
error.value = apiError(err, 'Could not load the setup checklist')
} finally {
loading.value = false
}
}
onMounted(load)
</script>
<style scoped>
.setup-checklist { padding: 1.5rem; }
.page-head { display: flex; justify-content: space-between; align-items: flex-start; gap: 1rem; }
.page-head h1 { margin: 0; }
.hint { color: var(--text-light); margin: 0.25rem 0 0; max-width: 60ch; }
.summary { display: flex; gap: 0.5rem; margin: 1rem 0; flex-wrap: wrap; }
.pill { font-size: 11px; text-transform: uppercase; letter-spacing: 0.4px; padding: 0.2rem 0.55rem; border-radius: 999px; border: 1px solid var(--border); }
.pill.ok { color: var(--success, #2e7d32); }
.pill.partial { color: var(--warning, #b8860b); }
.pill.todo { color: var(--text-light); }
.pill.na { color: var(--text-light); opacity: 0.7; }
.restart-note { border-left: 3px solid var(--warning, #b8860b); padding: 0.6rem 0.85rem; margin: 0 0 1rem; background: rgba(184, 134, 11, 0.08); }
.restart-note p { margin: 0.4rem 0 0; }
.warn-note { border-left: 3px solid var(--warning, #b8860b); padding: 0.5rem 0.75rem; background: rgba(184, 134, 11, 0.08); }
.checklist-table { width: 100%; }
.st-badge { font-size: 10px; text-transform: uppercase; letter-spacing: 0.4px; }
.st-ok .st-badge { color: var(--success, #2e7d32); }
.st-partial .st-badge { color: var(--warning, #b8860b); font-weight: 700; }
.st-todo .st-badge, .st-na .st-badge { color: var(--text-light); }
.st-label { font-weight: 600; white-space: nowrap; }
.wiz-tag { margin-left: 0.4rem; font-size: 10px; text-transform: uppercase; letter-spacing: 0.4px; color: var(--text-light); font-weight: 400; }
.st-detail { color: var(--text-light); }
.st-action { text-align: right; white-space: nowrap; }
</style>

View File

@@ -4,7 +4,7 @@
// Order: site identity + the reference-data catalogs users touch daily come
// first, then the platform/system groups (integrations, communication, search,
// plugins, access) cluster together at the end.
import { Factory, MapPin, Tag, Package, Droplets, Monitor, MonitorSmartphone, Laptop, Cog, Building, Globe, Link, History, FileText, Users, Puzzle, Bell, Network, Home, Wrench, Printer, Router, Palette, SlidersHorizontal, Contact, Mail, ShieldCheck, KeyRound, Fingerprint, Search, Key, Upload, DatabaseBackup, ListChecks} from 'lucide-vue-next'
import { Factory, MapPin, Tag, Package, Droplets, Monitor, MonitorSmartphone, Laptop, Cog, Building, Globe, Link, History, FileText, Users, Puzzle, Bell, Network, Home, Wrench, Printer, Router, Palette, SlidersHorizontal, Contact, Mail, ShieldCheck, KeyRound, Fingerprint, Search, Key, Upload, DatabaseBackup } from 'lucide-vue-next'
export const settingsGroups = [
{
@@ -85,7 +85,6 @@ export const settingsGroups = [
{
title: 'System',
cards: [
{ to: '/settings/setupchecklist', icon: ListChecks, title: 'Setup Checklist', description: 'What this site has configured and what is still outstanding' },
{ to: '/settings/plugins', icon: Puzzle, title: 'Plugins', description: 'Enable or disable installed plugins' },
{ to: '/settings/shopdbbackups', icon: DatabaseBackup, title: 'Site Backups', description: 'Scheduled database and file backups with guided restore; keeps any day in the last 5' },
],

View File

@@ -39,19 +39,6 @@ depends_on = None
def upgrade():
# GUARD: this table belongs to the endpoints PLUGIN, not to core.
# PLUGIN_TABLE_OWNERS has always said so, and the foreign key below points
# at endpointcommands, which only the plugin's 0002 revision creates. Core
# runs first and a plugin chain only runs once the plugin is enabled, so on
# a fresh database endpointcommands does not exist yet and MySQL rejects
# the create with 1824. Skip here and let endpoints0025logcollections build
# it in the chain that owns it.
#
# Sites that already applied this revision have the table, so they are
# unaffected either way: the plugin revision is guarded and no-ops there.
if not sa.inspect(op.get_bind()).has_table('endpointcommands'):
return
op.create_table(
'endpointlogcollections',
sa.Column('endpointlogcollectionid', sa.Integer(), nullable=False),
@@ -96,11 +83,6 @@ def upgrade():
def downgrade():
# Mirror of the upgrade guard: on a fresh install this revision created
# nothing, so there is nothing here to drop.
if not sa.inspect(op.get_bind()).has_table('endpointlogcollections'):
return
op.drop_index('idx_endpointlogcollections_assetcollected',
table_name='endpointlogcollections')
op.drop_index('ix_endpointlogcollections_collectedat',

View File

@@ -2,7 +2,6 @@
"name": "backups",
"version": "1.0.0",
"description": "Per-asset configuration backups with revision history. Kind-pluggable: NTLARS/DNC registry, part marker, UDC, CMM. Parseable kinds store a JSON projection in ShopDB and re-render to their native format on download; opaque vendor formats stay on the SFLD share with ShopDB holding metadata and the UNC pointer.",
"recommended": true,
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.16.0,<1.0.0",

View File

@@ -12,7 +12,21 @@
has been configured.</p>
</div>
<!-- Split into two cards rather than one list with a Source column.
There is one shipped policy PER MONITOR, so shipped rows outnumber
site-authored ones and a single sorted list buries the handful of
policies the site actually wrote. The split also gives the shipped
group somewhere to carry its warning, which a per-row badge cannot
do. Both groups render the same row markup and the same editable
rule - the separation is presentational, not a permission boundary. -->
<div class="card">
<h3>Shipped policies</h3>
<p class="shipped-warning">
These ship with the product as starting points - one per monitor. You
can edit or delete them and your changes are kept, but an upgrade may
add new rows here. Deleting one does not stop the monitor it refers to;
it only removes this policy's opinion about it.
</p>
<div v-if="loading" class="loading">Loading...</div>
<template v-else>
<div class="table-container">
@@ -27,7 +41,49 @@
</tr>
</thead>
<tbody>
<tr v-for="p in policies" :key="p.configurationpolicyid">
<tr v-for="p in shippedPolicies" :key="p.configurationpolicyid">
<td>{{ p.name }}</td>
<td class="cell-truncate" :title="p.description">{{ p.description || '-' }}</td>
<td>
<span v-for="a in p.actions" :key="a.action" class="badge" style="margin-right:4px;">
{{ actionLabel(a.action) }} ({{ a.mode }})
</span>
<span v-if="!p.actions || p.actions.length === 0" style="color: var(--text-light);">none</span>
</td>
<td>
<span class="badge badge-warning">{{ p.source }}</span>
</td>
<td class="actions">
<button class="btn btn-secondary btn-sm" :disabled="!iseditable(p)" @click="openModal(p)">Edit</button>
<button class="btn btn-danger btn-sm" :disabled="!iseditable(p)" @click="remove(p)">Delete</button>
</td>
</tr>
<tr v-if="shippedPolicies.length === 0">
<td colspan="5" style="text-align:center; color: var(--text-light);">No shipped policies</td>
</tr>
</tbody>
</table>
</div>
</template>
</div>
<div class="card">
<h3>Site policies</h3>
<div v-if="loading" class="loading">Loading...</div>
<template v-else>
<div class="table-container">
<table>
<thead>
<tr>
<th>Name</th>
<th>Description</th>
<th>Actions in policy</th>
<th>Source</th>
<th></th>
</tr>
</thead>
<tbody>
<tr v-for="p in localPolicies" :key="p.configurationpolicyid">
<td>{{ p.name }}</td>
<td class="cell-truncate" :title="p.description">{{ p.description || '-' }}</td>
<td>
@@ -46,7 +102,7 @@
<button class="btn btn-danger btn-sm" :disabled="!iseditable(p)" @click="remove(p)">Delete</button>
</td>
</tr>
<tr v-if="policies.length === 0">
<tr v-if="localPolicies.length === 0">
<td colspan="5" style="text-align:center; color: var(--text-light);">No configuration policies</td>
</tr>
</tbody>
@@ -115,7 +171,7 @@
</template>
<script setup>
import { ref, onMounted } from 'vue'
import { ref, computed, onMounted } from 'vue'
import { computersApi, applicationsApi, endpointsApi } from '@/api'
import { useToast } from '@/composables/toast'
import { apiError } from '@/utils/apiError'
@@ -299,6 +355,17 @@ function iseditable(p) {
return EDITABLE_SOURCES.includes(p.source)
}
// Presentational split only. Anything that is not 'shipped' lands in the site
// list - including 'autoapp', which is generated rather than hand-written but
// is still this site's own row and carries its own badge there. Written as
// "not shipped" rather than "=== local" so a future source value shows up in a
// list instead of vanishing from both, which is the failure mode that made the
// old Edit buttons dead.
const shippedPolicies = computed(
() => policies.value.filter(p => p.source === 'shipped'))
const localPolicies = computed(
() => policies.value.filter(p => p.source !== 'shipped'))
async function remove(p) {
if (!confirm(`Delete configuration policy "${p.name}"?`)) return
try {
@@ -318,6 +385,14 @@ async function remove(p) {
.feature-off-notice p {
margin: 0.4rem 0 0 0;
}
/* Matches the feature-off notice rather than inventing a second warning
treatment: same left rule, same warning colour. */
.shipped-warning {
border-left: 3px solid var(--warning);
padding-left: 0.75rem;
margin: 0 0 1rem 0;
color: var(--text-light);
}
.modal-lg {
max-width: 760px;
}

View File

@@ -1834,8 +1834,30 @@ async function confirmRestart() {
const corporateTools = [
{
task: '\\ReportIP',
label: 'Report IP',
description: 'Re-runs the corporate IP reporting agent on this computer.',
label: 'Report IP - Scheduled Task',
description: 'Re-runs the corporate IP reporting agent on this computer '
+ 'by starting its registered scheduled task as SYSTEM.',
},
// Same tool, different delivery. The entry above starts the endpoint's own
// scheduled task; this one pushes a ShopDB-hosted script that reproduces
// double-clicking the desktop icon, for machines where the task is missing
// or not registered.
//
// It dispatches run_script rather than scheduledtask_run, so it carries a
// `script` field instead of a task path. routes.py allows this ONE script
// through on endpoints.corptools alone, matched by the stored script name in
// CORPTOOLS_PUSHABLE_SCRIPTS - so this string is load-bearing and must stay
// identical to the name the seeder writes. The name is sent rather than
// script:<id> because the id is assigned at seed time and differs per
// deployment; _resolve_run_script accepts either.
{
task: 'script:Report IP - Desktop Shortcut',
script: 'Report IP - Desktop Shortcut',
label: 'Report IP - Desktop Shortcut',
description: 'Re-reports this computer\'s IP address by running the same '
+ 'tool the desktop icon launches. Runs in the signed-in '
+ 'user\'s session - fails if nobody is logged on. Use when '
+ 'the scheduled task above is missing or fails.',
},
{
task: '\\SFLD-ApplyDSCConfig',
@@ -2274,17 +2296,32 @@ async function confirmRunTask(tool) {
if (typed !== 'RUN') return
} else if (!confirm(`Run "${tool.label}" on ${label}?\n\n`
+ `${tool.description}\n\n`
+ 'The task starts immediately on the endpoint.')) {
+ (tool.script
? 'The script is queued and runs at the next check-in.'
: 'The task starts immediately on the endpoint.'))) {
return
}
firingTask.value = tool.task
try {
await endpointsApi.enqueueCommand(
computer.value.assetid, 'scheduledtask_run', tool.task)
// "Started", not "finished": the agent reports success as soon as Task
// Scheduler accepts the task, so this never means the work is done. Any
// result the task produces comes back through telemetry, not this command.
toast.success(`${tool.label} started on ${label}`)
// Two dispatch shapes behind one list. Most entries start a task the
// endpoint already approved; the managed-script entries deliver a body
// ShopDB holds, which is a different action with a different server-side
// check. Branching here rather than in the list keeps the card markup and
// the confirm gate identical for both.
if (tool.script) {
await endpointsApi.enqueueCommand(
computer.value.assetid, 'run_script', tool.script)
// "Queued", not "started": run_script is collected on the agent's next
// check-in, unlike scheduledtask_run which the endpoint accepts live.
toast.success(`${tool.label} queued for ${label}`)
} else {
await endpointsApi.enqueueCommand(
computer.value.assetid, 'scheduledtask_run', tool.task)
// "Started", not "finished": the agent reports success as soon as Task
// Scheduler accepts the task, so this never means the work is done. Any
// result the task produces comes back through telemetry, not this command.
toast.success(`${tool.label} started on ${label}`)
}
} catch (error) {
toast.error(apiError(error, `Failed to queue ${tool.label}`))
} finally {

View File

@@ -20,8 +20,26 @@ describe('the configuration policy buttons follow the API guard', () => {
expect(view).not.toContain(":disabled=\"p.source !== 'local'\"")
})
// Counted in pairs rather than pinned to a fixed number: the list renders
// shipped and site policies as two separate tables, so there is one Edit and
// one Delete per table. What matters is that the two buttons always agree
// with each other and both defer to iseditable - not how many tables exist.
it('gates edit and delete on the same rule', () => {
const disabled = view.match(/:disabled="!iseditable\(p\)"/g) || []
expect(disabled).toHaveLength(2)
const edits = view.match(/@click="openModal\(p\)"/g) || []
const deletes = view.match(/@click="remove\(p\)"/g) || []
expect(disabled.length).toBe(edits.length + deletes.length)
expect(edits.length).toBe(deletes.length)
expect(edits.length).toBeGreaterThan(0)
})
// The split is presentational. Both groups must render the same buttons
// under the same rule, so a shipped row is never stranded behind a dead
// button the way it was when the buttons gated on source === 'local'.
it('renders both policy groups from the same editable rule', () => {
expect(view).toContain('shippedPolicies')
expect(view).toContain('localPolicies')
expect(view).toContain("p.source === 'shipped'")
expect(view).toContain("p.source !== 'shipped'")
})
})

View File

@@ -1,24 +1,23 @@
{
"name": "computers",
"version": "1.0.0",
"description": "Computer management plugin for PCs, servers, and workstations with software tracking",
"recommended": true,
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.20.0,<1.0.0",
"api_prefix": "/api/computers",
"provides": {
"asset_type": "computer",
"features": [
"computer_tracking",
"software_inventory",
"remote_access",
"os_management"
]
},
"settings": {
"enable_winrm": true,
"enable_vnc": true,
"auto_report_interval_hours": 24
}
"name": "computers",
"version": "1.0.0",
"description": "Computer management plugin for PCs, servers, and workstations with software tracking",
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.20.0,<1.0.0",
"api_prefix": "/api/computers",
"provides": {
"asset_type": "computer",
"features": [
"computer_tracking",
"software_inventory",
"remote_access",
"os_management"
]
},
"settings": {
"enable_winrm": true,
"enable_vnc": true,
"auto_report_interval_hours": 24
}
}

View File

@@ -28,22 +28,13 @@ from ..models import EndpointMonitor
from .routes import endpoints_bp
# Which PC type the shipped monitors attach to, as a SETTING rather than a
# literal (ADR-015). PC-type vocabulary is per-site seed data: a plant whose
# type is called "Shop Floor Workstation" got no shipped monitors at all when
# this was hardcoded, and a plant that happens to use the name for something
# else got all of them. The default matches the seeded type so nothing changes
# for a site that uses it; a blank setting attaches the shipped bundle to
# nothing, which is the right neutral behaviour for a site that does not want
# it rather than a silent mismatch.
# RETIRED. This setting named the PC type the shipped monitors auto-attached
# to, defaulting to 'Shopfloor PC'. That auto-attach path is gone - policy is
# now the only attachment route (see _resolve_pctype_monitors) - so nothing
# reads this. The key is kept named here, not deleted, so an existing row in a
# site's settings table is recognisable as dead rather than mysterious, and so
# the key is not reused for something else later.
SHIPPED_PCTYPE_SETTING = 'plugin.endpoints.shippedmonitorpctype'
SHIPPED_DEFAULT_PCTYPE = 'Shopfloor PC'
def shipped_pctype():
"""The PC type the shipped monitor bundle attaches to at this site."""
configured = Setting.get(SHIPPED_PCTYPE_SETTING, default=SHIPPED_DEFAULT_PCTYPE)
return (configured or '').strip()
_VALID_KINDS = ('process', 'service')
@@ -170,20 +161,25 @@ def delete_monitor(monitor_id):
def _resolve_pctype_monitors(pctype):
"""Monitor definitions a PC type should watch: shipped defaults + policy.
"""Monitor definitions a PC type should watch, from site policy alone.
Policy is the ONLY attachment route. Shipped monitors used to also attach
automatically to whichever PC type the shipped-pctype setting named, which
defaulted to 'Shopfloor PC' - so eDNC, UDC and MTConnect were expected
monitors on every endpoint of that type at every site, with no policy
anywhere and no way to see why. 0019 split the shipped bundle into
per-monitor policies and removed the shipped SITE policy, intending
"off by default"; this branch survived and kept them attached, which is
why dncMain.exe kept appearing in Fleet health for all endpoints.
Shipped monitors still exist and are still editable - a site composes them
through a Site Policy like anything else. What no longer happens is
attaching them because a PC type happens to carry a particular name.
Shipped monitors attach to the PC type named by the shipped-pctype setting;
policy-attached monitors come from the computers policy resolver by PC type.
Returns a list of EndpointMonitor rows (active only), de-duped by id.
"""
selected = {}
shipped_for = shipped_pctype()
if shipped_for and pctype == shipped_for:
for monitor in EndpointMonitor.query.filter_by(
source='shipped', isactive=True).all():
selected[monitor.endpointmonitorid] = monitor
computertypeid = _computertypeid_for(pctype)
if computertypeid is not None:
policy_ids = _policy_monitor_ids(computertypeid)

View File

@@ -0,0 +1,159 @@
"""Seed the Report IP desktop-shortcut script into the ShopDB script library.
run_script only delivers bodies that live in the script library, pinned by
SHA-256. The shortcut-equivalent tool therefore has to exist as a real Script
row with a stored body before Corporate Tools can push it.
The template is source-controlled at
plugins/endpoints/api/templates/report-ip-shortcut.ps1 and seeded from there
rather than typed into the library by hand, so the reviewed script and the one
endpoints actually run are the same artifact. Editing the row's body in the UI
is pointless - the next seed overwrites it.
That last point is load-bearing here in a way it is not for an ordinary script.
REPORT_IP_SCRIPT_NAME is named in CORPTOOLS_PUSHABLE_SCRIPTS, so holders of
endpoints.corptools may push this one script WITHOUT holding
endpoints.runscript. Granting on a name is only safe because the name maps to a
body that comes from source control and is digest-pinned on delivery: the name
cannot be made to mean something else. If this row were hand-authorable, the
allowance would become "corptools may run arbitrary code", which is exactly the
bundling that endpoints.runscript was split out to prevent.
It lives under the plugin, not under agent/, because the Docker image copies
shopdb/, plugins/, migrations/ and scripts/ but NOT agent/. A template read
from agent/ resolves fine on a developer checkout and then fails in the
deployed container, which is the worst place to discover it.
Unlike the agent self-upgrade seeder there is no placeholder to render: the
script resolves its own target on the endpoint (shortcut first, installed path
as fallback), so the body is shipped verbatim.
"""
import hashlib
import os
from shopdb.api import Script, db, script_body_location
# Reserved script name. Unique in the library, which makes seeding idempotent
# and keeps a hand-made script from colliding with the managed one. This exact
# string is what the enqueue route matches when deciding whether
# endpoints.corptools is sufficient, so it must stay in lockstep with
# CORPTOOLS_PUSHABLE_SCRIPTS in plugins/endpoints/commands.py - the test
# test_reserved_report_ip_name_matches_the_commands_constant pins them together.
REPORT_IP_SCRIPT_NAME = 'Report IP - Desktop Shortcut'
_REPORT_IP_DESCRIPTION = (
'Managed: reproduces double-clicking the Report IP icon on the Public '
'Desktop, re-reporting this computer\'s IP address. Runs in the signed-in '
'user\'s session and fails closed if nobody is logged on. '
'Seeded from source control - edits here are overwritten.'
)
_TEMPLATE_FILENAME = 'report-ip-shortcut.ps1'
def template_path():
return os.path.join(os.path.dirname(os.path.abspath(__file__)),
'templates', _TEMPLATE_FILENAME)
def _script_dir():
"""Where script bodies live. Resolved through the contract surface helper
so this cannot drift from the directory the download route serves from."""
directory, _prefix = script_body_location()
return directory
def render_template():
"""Return the script body verbatim.
No substitution: the endpoint resolves its own target at run time. Kept as
a function so the seeder reads the file through one place and a future
placeholder has an obvious home.
"""
with open(template_path(), 'r', encoding='utf-8') as handle:
return handle.read()
def seed_report_ip_script():
"""Create or refresh the managed Report IP script row and its body.
Returns a dict describing the seeded script. Idempotent: unchanged input
rewrites nothing and leaves the digest alone.
"""
body = render_template().encode('utf-8')
digest = hashlib.sha256(body).hexdigest()
script = Script.query.filter_by(scriptname=REPORT_IP_SCRIPT_NAME).first()
created = script is None
if created:
script = Script(
scriptname=REPORT_IP_SCRIPT_NAME,
scriptdescription=_REPORT_IP_DESCRIPTION,
# USER context, and this is not a preference - SYSTEM cannot do
# this job at all. Measured on a live endpoint: launched as SYSTEM
# the tool immediately re-entered the interactive session, leaving
# the work in a different process, session and account
# (pid 7240, session 1, owner ...\ShopFloor) while the SYSTEM-side
# launcher exited 0 straight away. The command then reported
# "completed (system context)" having observed nothing - a success
# that means "handed off", not "worked".
#
# This is the same session-0 boundary EndpointCollector already
# documents for mapped drives: session 0 has no access to a user's
# drive table, HKCU or profile, so a SYSTEM process cannot see -
# let alone report on - what happens in the user's session. The
# agent's user path crosses that boundary properly via
# CreateProcessAsUser and returns the child's REAL exit code.
#
# The tradeoff accepted here: user context fails closed when nobody
# is signed in. That is the correct failure - an honest "no one is
# logged on" beats a green result that observed nothing.
runcontext='user',
# Capture successful output too. The whole point of this tool is
# the response line it prints, and an operator who pushed it needs
# to read that line to know the IP was actually reported - a bare
# "succeeded" answers a different question.
captureoutput=True,
)
db.session.add(script)
db.session.flush()
# Body filename follows the library convention (script-<id>.ps1) so the
# collector download route serves it like any other script.
filename = 'script-{}.ps1'.format(script.scriptid)
path = os.path.join(_script_dir(), filename)
unchanged = (script.filename == filename and script.sha256 == digest
and os.path.isfile(path))
if not unchanged:
os.makedirs(_script_dir(), exist_ok=True)
# Temp-then-replace so an interrupted write cannot leave a truncated
# body at the path the row's digest already claims to describe.
temppath = path + '.tmp'
with open(temppath, 'wb') as handle:
handle.write(body)
os.replace(temppath, path)
script.filename = filename
script.sha256 = digest
# Re-assert the managed fields on every seed. A row edited in the UI drifts
# back to the shipped definition here, which is what "edits are
# overwritten" has to mean for the corptools allowance to stay honest: an
# operator must not be able to flip this to runcontext=system, or
# deactivate it, and leave the name still pushable. The context is part of
# what makes the result trustworthy, so it is re-asserted like the rest.
script.scriptdescription = _REPORT_IP_DESCRIPTION
script.runcontext = 'user'
script.captureoutput = True
script.isactive = True
db.session.commit()
return {
'scriptid': script.scriptid,
'scriptname': script.scriptname,
'created': created,
'updated': not unchanged,
'sha256': digest,
'bytes': len(body),
}

View File

@@ -31,7 +31,7 @@ from ..models import (
EndpointState, EndpointEvent, EndpointCommand, EndpointApplication,
EndpointHealthState, OperatorReport, EndpointLogCollection,
)
from ..commands import COMMAND_ALLOWLIST, is_allowed_action, ALLOWED_ACTIONS_FOR, row_actions_for, requires_target, action_targetkind, elevated_permission_for, minimum_agent_version_for, action_minimum_versions, parse_agent_version, agent_meets_minimum, PLACE_POLICY_HOLD, ACTIVE_POLICY_HOLDS, HOLDABLE_ACTIONS_FOR, RELEASE_POLICY_HOLD, ACTIVE_LOCK_FOR, PLACE_ENDPOINT_LOCK, RELEASE_ENDPOINT_LOCK
from ..commands import COMMAND_ALLOWLIST, is_allowed_action, ALLOWED_ACTIONS_FOR, row_actions_for, requires_target, action_targetkind, elevated_permission_for, corptools_may_push_script, minimum_agent_version_for, action_minimum_versions, parse_agent_version, agent_meets_minimum, PLACE_POLICY_HOLD, ACTIVE_POLICY_HOLDS, HOLDABLE_ACTIONS_FOR, RELEASE_POLICY_HOLD, ACTIVE_LOCK_FOR, PLACE_ENDPOINT_LOCK, RELEASE_ENDPOINT_LOCK
from . import agentstore
from . import agentauth
from ..eventcatalog import CRITICAL_EVENT_IDS, event_severity, event_title
@@ -2992,10 +2992,28 @@ def enqueue_endpoint_command(asset_id: int):
# require_permission exists to prevent, reintroduced by checking by hand.
elevated = elevated_permission_for(action)
if elevated and not caller_holds(elevated):
return error_response(
ErrorCodes.FORBIDDEN,
f'Action {action!r} requires the {elevated} permission',
http_code=403)
# One narrow exception: a SHIPPED script whose reviewed body is
# operationally equivalent to something the person could already do
# unaided at the machine (Report IP is an icon on every Public Desktop).
# Those are pushable with endpoints.corptools alone.
#
# The allowance is keyed on the script's STORED name, read off a row we
# resolve here - never on the caller-supplied target. Matching what the
# requester typed would let anyone name their way past the grant, and
# relaxing the run_script ACTION instead would hand every corptools
# holder the whole library (the script id rides in the request body),
# rebuilding the endpoints.deploy bundle the permission split removed.
allowed_by_shipped_script = False
if action == 'run_script' and caller_holds('endpoints.corptools'):
resolved, _resolve_error = _resolve_run_script(target)
if resolved is not None:
allowed_by_shipped_script = corptools_may_push_script(
resolved.get('scriptname'))
if not allowed_by_shipped_script:
return error_response(
ErrorCodes.FORBIDDEN,
f'Action {action!r} requires the {elevated} permission',
http_code=403)
# Break-glass: an admin may push past a Configuration Policy denial, but
# only by asking for it explicitly and saying why. The override is scoped to

View File

@@ -201,21 +201,39 @@ _BASE_APPSETTINGS = {
'adoptonfirstrun': True,
'utilitiespath': '',
'logpath': '',
# Shipped EMPTY, deliberately. The server decides what an endpoint watches:
# the agent fetches its watch list from /api/endpoints/monitors/resolved
# every telemetry cycle and REPLACES this list with the answer, so anything
# seeded here is overwritten within one cadence anyway.
#
# It used to seed dncMain.exe, udc.exe, agent.exe and the MTConnect service,
# which is why every endpoint in the fleet reported eDNC regardless of what
# it actually ran. Back when the agent UNIONED the two lists, a baked-in
# entry could never be retracted from the server, so detaching a monitor in
# the WebUI silently did nothing on deployed endpoints and Fleet health
# surfaced dncMain everywhere. Seeding anything here re-creates that class of
# bug for as long as the affected packages stay in the field.
#
# To watch a process or service, attach a monitor to the PC type in the
# WebUI. That reaches every endpoint with no rebuild and no redeploy.
'monitors': {
'processes': [
{'name': 'dncMain.exe', 'app': 'eDNC'},
{'name': 'udc.exe', 'app': 'Universal Data Collector'},
{'name': 'agent.exe', 'app': 'MTConnect'},
],
'services': [
{'name': 'MTConnect UDC Agent', 'app': 'MTConnect'},
],
},
'launchpaths': {
'DncMain.exe': 'C:\\Program Files (x86)\\Dnc\\bin\\DncMain.exe',
'UDC.exe': 'C:\\Program Files\\UDC\\UDC.exe',
'agent.exe': 'C:\\ProgramData\\udc\\mtconnect_udc\\agent\\run.bat',
'processes': [],
'services': [],
},
# Intentionally EMPTY, like monitors above, and for the same reason. Launch
# paths are server authoritative: they ride on the monitor rows the WebUI
# owns and reach the agent through /api/endpoints/monitors/resolved, so
# editing a path in the WebUI reaches every endpoint with no rebuild.
#
# A local entry here WINS over the server path AND bypasses the agent's
# LaunchPathPolicy validation (see CommandExecutor.ResolveLaunchPath), which
# is exactly how a stale pin outlives the configuration that justified it.
# eDNC, UDC and MTConnect all carry their launchpath on the shipped monitor
# rows instead.
#
# Seed an entry here only as a deliberate per-site override, accepting that
# the endpoint stops tracking WebUI changes for that image.
'launchpaths': {},
# Approved scheduled task paths for scheduledtask_run. Shipped empty, which
# means the agent falls back to ScheduledTaskPolicy.DefaultApprovedTaskPaths
# (the Corporate Tools set). Listing entries here REPLACES those defaults, so

View File

@@ -0,0 +1,74 @@
# Report IP - reproduces double-clicking the "Report IP" desktop icon.
#
# Seeded from source control by plugins/endpoints/api/reportipscript.py. Edits
# made to the library row are overwritten on the next seed: the corptools
# allowance grants this script BY NAME, so the name has to keep meaning this
# reviewed body.
#
# RUNS IN THE USER SESSION, and that is a hard requirement rather than a
# preference. Measured on a live endpoint, launching this tool as SYSTEM has it
# immediately re-enter the interactive session - the work ended up in a separate
# process under the signed-in account (session 1) while the SYSTEM-side launcher
# exited 0 within a second, having observed nothing. Every attempt to capture
# output or a verdict from the SYSTEM side therefore reported success for work
# it never saw.
#
# This is the same session-0 boundary EndpointCollector documents for mapped
# drives: a SYSTEM process has no view of a user's drive table, HKCU or profile.
# The agent's user-context path crosses it properly (CreateProcessAsUser into
# the active console session) and returns the child's real exit code, so the
# verdict below is trustworthy in a way the SYSTEM path could not be.
#
# Consequently this script stays deliberately thin: the agent owns launching,
# waiting and output capture. Re-implementing any of that here is what produced
# the false results.
$ErrorActionPreference = 'Stop'
# Resolved from the shortcut when present, so the arguments stay whatever the
# icon on the desktop actually uses. The fallback matches the installed tool.
$ShortcutPath = 'C:\Users\Public\Desktop\Report IP.lnk'
$Exe = 'C:\ProgramData\ReportIP\GE_ReportIP_3_v1.EXE'
$ExeArgs = @('/ForceUpdate=True', '/S')
# Working directory comes from the vendor's own scheduled task definition
# (\ReportIP), which runs the tool from its install folder.
$WorkDir = 'C:\ProgramData\ReportIP'
if (Test-Path -LiteralPath $ShortcutPath) {
try {
$link = (New-Object -ComObject WScript.Shell).CreateShortcut($ShortcutPath)
if ($link.TargetPath) {
$Exe = $link.TargetPath
if ($link.Arguments) { $ExeArgs = @($link.Arguments) }
if ($link.WorkingDirectory) { $WorkDir = $link.WorkingDirectory }
}
} catch {
Write-Output "could not read the shortcut, using the installed path: $($_.Exception.Message)"
}
}
if (-not (Test-Path -LiteralPath $Exe)) {
Write-Output "executable not found: $Exe"
exit 2
}
if (-not (Test-Path -LiteralPath $WorkDir)) { $WorkDir = Split-Path -Parent $Exe }
# Identity is logged because it is the thing most likely to be wrong: if this
# ever reports SYSTEM, the run is meaningless for the reasons in the header.
Write-Output "running as: $([Security.Principal.WindowsIdentity]::GetCurrent().Name)"
Write-Output "session: $((Get-Process -Id $PID).SessionId)"
Write-Output "launching: $Exe $($ExeArgs -join ' ')"
# -Wait, and the tool's own exit code is the verdict. No success-string scan:
# the tool reports its outcome through its own channel, and inferring success
# from console text was what produced both false failures and false successes
# in earlier revisions.
$process = Start-Process -FilePath $Exe `
-ArgumentList $ExeArgs `
-WorkingDirectory $WorkDir `
-PassThru -Wait
Write-Output "exit code: $($process.ExitCode)"
exit $process.ExitCode

View File

@@ -238,6 +238,44 @@ def elevated_permission_for(action):
return ELEVATED_ACTION_PERMISSIONS.get(action)
# Shipped scripts that endpoints.corptools may push without holding
# endpoints.runscript.
#
# This is an allowance on ONE NAMED SCRIPT, never on the run_script action. The
# distinction is the whole point: relaxing the action would let any corptools
# holder push ANY script in the library - including the agent self-upgrade, and
# anything an operator uploads later - because the script id rides in the
# caller-controlled request body. That would silently rebuild the endpoints.deploy
# bundle the permission split took apart, which is the one outcome this must not
# produce.
#
# Granting on a name is only defensible because these names are MANAGED: each is
# seeded from a source-controlled template, pinned by SHA-256 on delivery, and
# re-asserted on every seed (see reportipscript.py). The name therefore cannot be
# made to mean something else by editing a row. An entry here is a statement that
# the reviewed body is operationally equivalent to something the user could
# already do unaided - Report IP is an icon on every Public Desktop in the
# company, so pushing it grants no capability the person did not already have.
#
# Do NOT add a name here because a denial is inconvenient. The test for
# membership is that a non-privileged employee can already trigger the same work
# by hand on the machine in front of them. Anything else needs endpoints.runscript.
CORPTOOLS_PUSHABLE_SCRIPTS = frozenset({
'Report IP - Desktop Shortcut',
})
def corptools_may_push_script(scriptname):
"""True when endpoints.corptools alone is enough to push this script.
Matched on the exact managed name. Callers must resolve the target to a real
Script row FIRST and pass the stored scriptname, never a caller-supplied
string: matching what the requester typed would let anyone name their way
past the grant.
"""
return scriptname in CORPTOOLS_PUSHABLE_SCRIPTS
def minimum_agent_version_for(action):
"""The lowest agent version that implements an action, or None if any will do."""
return COMMAND_ALLOWLIST.get(action, {}).get('minagentversion')
@@ -449,6 +487,19 @@ POLICY_EXEMPT_ACTIONS = frozenset({
# still requires endpoints.updateapp via ELEVATED_ACTION_PERMISSIONS, and
# the agent still refuses UNC, traversal, directories and non-text files.
'collect_log',
# Script push is gated by WHO the caller is, not by what a PC type exposes.
# Configuration Policy describes machine shape; it has no view on whether a
# given person may execute code, so a policy that never mentions run_script
# must not be read as withdrawing it. Leaving it policy-gated forced sites
# to keep an all-PCs "allow run_script" policy row alive purely to avoid
# losing the action everywhere - a rule that encoded no real intent and
# whose deletion silently disarmed Script Push on every policied endpoint.
# Authority lives on the caller axis instead: run_script maps to
# endpoints.runscript in ELEVATED_ACTION_PERMISSIONS, checked with
# caller_holds at the enqueue route BEFORE policy is consulted, so a scoped
# personal token cannot inherit it from an admin owner. This exemption
# widens WHERE the action may run, never WHO may run it.
'run_script',
})

View File

@@ -1,101 +0,0 @@
"""Move endpointlogcollections into the chain that owns it.
Revision ID: endpoints0025logcollections
Revises: endpoints0024bootstraptokens
Create Date: 2026-09-19
WHY THIS REVISION EXISTS. endpointlogcollections was created by a CORE
migration (7d51_endpoint_logcollections) even though PLUGIN_TABLE_OWNERS has
always listed it, and endpointcommands, as owned by this plugin. The core table
carries a foreign key onto endpointcommands, which only 0002 creates. Core runs
before any plugin chain, and a plugin chain only runs once the plugin is
enabled, which cannot happen before the schema exists. So on a FRESH database
there is no ordering in which the core revision can succeed: MySQL rejects it
with 1824, "Failed to open the referenced table 'endpointcommands'".
Nothing caught it because TestingConfig runs on sqlite:///:memory:, and SQLite
accepts a foreign key onto a table that does not exist. The failure is
MySQL-only, and therefore invisible to the whole test suite.
Existing sites already ran the core revision and have the table, so this is
guarded and is a no-op there. 7d51 is guarded in the same change so it skips
when endpointcommands is absent, which is exactly the fresh-install case this
revision then handles.
Downgrade drops the table.
"""
from alembic import op
from shopdb.plugins.alembic_template import (
create_index_if_missing,
create_table_if_missing,
)
import sqlalchemy as sa
revision = 'endpoints0025logcollections'
down_revision = 'endpoints0024bootstraptokens'
branch_labels = None
depends_on = None
plugin_name = 'endpoints'
COLLECTIONS = 'endpointlogcollections'
def upgrade():
created = create_table_if_missing(
COLLECTIONS,
sa.Column('endpointlogcollectionid', sa.Integer(), nullable=False),
sa.Column('assetid', sa.Integer(), nullable=False),
sa.Column('endpointcommandid', sa.Integer(), nullable=True),
sa.Column('sourcepath', sa.String(length=512), nullable=False,
comment='Path as the AGENT resolved it, which may differ '
'from what the operator typed once environment '
'variables are expanded endpoint-side.'),
sa.Column('content', sa.Text(length=16777215), nullable=True),
sa.Column('filesizebytes', sa.BigInteger(), nullable=True,
comment='Size of the file on disk. Differs from contentbytes '
'when the stored copy is a truncated tail.'),
sa.Column('contentbytes', sa.Integer(), nullable=True),
sa.Column('linecount', sa.Integer(), nullable=True),
sa.Column('istruncated', sa.Boolean(), nullable=False,
server_default=sa.false(),
comment='True when content is only the tail of a larger file.'),
sa.Column('filemodifiedat', sa.DateTime(), nullable=True),
sa.Column('collectedbyuserid', sa.Integer(), nullable=True),
sa.Column('collectedbyusername', sa.String(length=100), nullable=True,
comment='Username snapshot; the id alone becomes unreadable '
'if the user is later removed.'),
sa.Column('collectedat', sa.DateTime(), nullable=False),
sa.ForeignKeyConstraint(['assetid'], ['assets.assetid'],
ondelete='CASCADE'),
# SET NULL rather than CASCADE: commands are reaped and expired over
# time, and a collected log an operator is still reading must not
# vanish because its originating command row was cleaned up.
sa.ForeignKeyConstraint(['endpointcommandid'],
['endpointcommands.endpointcommandid'],
ondelete='SET NULL'),
sa.ForeignKeyConstraint(['collectedbyuserid'], ['users.userid'],
ondelete='SET NULL'),
sa.PrimaryKeyConstraint('endpointlogcollectionid'),
)
if not created:
return
create_index_if_missing('ix_endpointlogcollections_assetid',
COLLECTIONS, ['assetid'])
create_index_if_missing('ix_endpointlogcollections_collectedat',
COLLECTIONS, ['collectedat'])
# Serves both hot paths: listing an asset's history newest-first, and
# finding that asset's oldest rows to prune on insert.
create_index_if_missing('idx_endpointlogcollections_assetcollected',
COLLECTIONS, ['assetid', 'collectedat'])
def downgrade():
from sqlalchemy import inspect
if not inspect(op.get_bind()).has_table(COLLECTIONS):
return
op.drop_table(COLLECTIONS)

View File

@@ -0,0 +1,162 @@
"""Seed the Office ProPlus LTSC monitor templates (x64 and x86).
Revision ID: endpoints0025officemonitors
Revises: endpoints0024bootstraptokens
Create Date: 2026-09-19
Adds shipped process-monitor definitions for the Office ProPlus LTSC suite at
both install roots:
C:\\Program Files\\Microsoft Office\\root\\Office16\\<app>.EXE (x64)
C:\\Program Files (x86)\\Microsoft Office\\root\\Office16\\<app>.EXE (x86)
Apps: WINWORD, EXCEL, POWERPNT, OUTLOOK, MSACCESS.
These are TEMPLATES, not an instruction to watch anything. Seeding a definition
only puts a row in the catalog; nothing is watched until a site policy attaches
it to a PC type. That separation is the whole point of the monitor rework - the
installer no longer bakes a watch list into appsettings.json, and the agent
replaces its list from the server each cycle - so adding rows here cannot
resurrect the "dncMain.exe on every endpoint" symptom.
x64 and x86 are SEPARATE rows rather than one row with two paths. The launchpath
column holds a single path, and the two roots are genuinely different installs;
a site pins whichever one it actually deploys. The kind/displayname unique
constraint is what forces the architecture into the display name, which is also
what an operator needs to see to pick the right one.
processname is the same image name for both (WINWORD.EXE and friends do not
differ by architecture), so these rows are distinguished by launchpath. A site
that runs both would attach only the row matching its install root.
Idempotent on (kind, displayname): a re-run inserts nothing. Unlike 0006 this
does NOT bail out when the table is non-empty - that guard would make this a
no-op on every existing site, which is precisely where these templates are
wanted. Guards on the table being present and no-ops if it is not.
Downgrade removes only the rows this added, and only when no policy action
references them, so a site that composed one into a policy does not lose it to a
rollback.
"""
from alembic import op
import sqlalchemy as sa
from datetime import datetime
# Alembic stores this in alembic_version.version_num, a VARCHAR(32).
revision = 'endpoints0025officemonitors'
down_revision = 'endpoints0024bootstraptokens'
branch_labels = None
depends_on = None
plugin_name = 'endpoints'
MONITORS = 'endpointmonitors'
_X64_ROOT = r'C:\Program Files\Microsoft Office\root\Office16'
_X86_ROOT = r'C:\Program Files (x86)\Microsoft Office\root\Office16'
# (label, image name). The image name is identical across architectures.
_OFFICE_APPS = [
('Word', 'WINWORD.EXE'),
('Excel', 'EXCEL.EXE'),
('PowerPoint', 'POWERPNT.EXE'),
('Outlook', 'OUTLOOK.EXE'),
('Access', 'MSACCESS.EXE'),
]
def _rows():
"""The shipped Office rows: every app at both install roots."""
for label, image in _OFFICE_APPS:
for architecture, root in (('x64', _X64_ROOT), ('x86', _X86_ROOT)):
yield {
'kind': 'process',
'displayname': 'Office {0} ({1})'.format(label, architecture),
'processname': image,
'servicename': None,
'launchpath': '{0}\\{1}'.format(root, image),
'datacollectorpath': None,
'logcollectorpath': None,
'source': 'shipped',
'isactive': True,
}
def _inspector():
from sqlalchemy import inspect
return inspect(op.get_bind())
def _has_table(name):
return _inspector().has_table(name)
def _existing(bind, kind, displayname):
return bind.execute(
sa.text('SELECT endpointmonitorid FROM endpointmonitors '
'WHERE kind = :kind AND displayname = :displayname'),
{'kind': kind, 'displayname': displayname}).scalar()
def upgrade():
if not _has_table(MONITORS):
return
bind = op.get_bind()
now = datetime.utcnow()
monitors = sa.table(
MONITORS,
sa.column('kind', sa.String),
sa.column('displayname', sa.String),
sa.column('processname', sa.String),
sa.column('servicename', sa.String),
sa.column('launchpath', sa.String),
sa.column('datacollectorpath', sa.String),
sa.column('logcollectorpath', sa.String),
sa.column('source', sa.String),
sa.column('isactive', sa.Boolean),
sa.column('createdat', sa.DateTime),
)
pending = []
for row in _rows():
if _existing(bind, row['kind'], row['displayname']) is not None:
continue
record = dict(row)
record['createdat'] = now
pending.append(record)
if pending:
op.bulk_insert(monitors, pending)
def downgrade():
if not _has_table(MONITORS):
return
bind = op.get_bind()
policies_present = _has_table('configurationpolicyactions')
for row in _rows():
monitorid = _existing(bind, row['kind'], row['displayname'])
if monitorid is None:
continue
# Leave behind anything a site has already composed into a policy:
# dropping it would silently break that policy's action.
if policies_present:
referenced = bind.execute(
sa.text('SELECT COUNT(*) FROM configurationpolicyactions '
'WHERE action = :action'),
{'action': 'monitor:{0}'.format(monitorid)}).scalar()
if referenced:
continue
bind.execute(
sa.text('DELETE FROM endpointmonitors '
'WHERE endpointmonitorid = :id'),
{'id': monitorid})

View File

@@ -1448,6 +1448,68 @@ class EndpointsPlugin(BasePlugin):
'(concurrent update?)'.format(stored, skipped, len(to_add)))
return stored
def _authoritative_services(self, assetid, reported):
"""Keep only reported monitors the SERVER expects this asset to watch.
The server is the authority on what is monitored. Without this, the
watch list is effectively whatever the endpoint says it is: the agent
merges its local appsettings.json into the server list, so anyone who
can edit that file on the box can add entries the server never
attached. Two attacks fall out of that, and the second is the one that
matters. Adding noise is merely annoying. Adding a NEVER-RUNNING entry
parks the asset at a permanent condition, and an operator who learns to
ignore a perpetually amber endpoint stops reading the signal that says
a real service died - the endpoint blinds the fleet view by talking,
not by going quiet.
Resolution is by wire name, case-insensitively, because that is the
identity the agent reports and the identity the resolver hands out.
An asset with no resolved monitors keeps nothing, which is the same
"no monitors, no conditions" contract _service_conditions already has.
Drops are logged rather than silent: an entry disappearing between the
endpoint and the page is exactly the kind of thing that is impossible
to diagnose from the UI alone.
"""
if not isinstance(reported, list):
return reported
from flask import current_app
from .api.routes import _resolved_monitors_for_asset, _service_name
try:
monitors = _resolved_monitors_for_asset(assetid)
except Exception:
# Never fail a check-in over the filter. A resolver error must not
# cost us the whole telemetry report, so fall back to storing what
# was reported and let the health layer apply its own rules.
current_app.logger.exception(
'[collector] asset_id=%s monitor_resolution_failed: '
'storing reported services unfiltered', assetid)
return reported
expected = set()
for monitor in monitors:
for wirename in (monitor.processname, monitor.servicename):
if wirename and wirename.strip():
expected.add(wirename.strip().lower())
kept, dropped = [], []
for entry in reported:
if not isinstance(entry, dict):
continue
name = (_service_name(entry) or '').strip().lower()
if name in expected:
kept.append(entry)
else:
dropped.append(name)
if dropped:
current_app.logger.info(
'[collector] asset_id=%s unmonitored_services_dropped: %s '
'(not attached by any site policy)', assetid, ', '.join(dropped))
return kept
def apply_collector_payload(self, payload: Dict) -> Dict:
"""Idempotently upsert one endpoint's state and append its events.
@@ -1526,7 +1588,8 @@ class EndpointsPlugin(BasePlugin):
if 'memorypercent' in payload:
state.memorypercent = payload.get('memorypercent')
if 'services' in payload:
state.servicesjson = _as_json(payload.get('services'))
state.servicesjson = _as_json(
self._authoritative_services(assetid, payload.get('services')))
if 'ports' in payload:
state.portsjson = _as_json(payload.get('ports'))
if 'interfaces' in payload:

View File

@@ -3,14 +3,11 @@
"display_name": "Guides",
"version": "1.0.0",
"description": "Step-by-step guides written and read inside ShopDB, with images, working checklists and printable output",
"recommended": true,
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.1.0,<1.0.0",
"api_prefix": "/api/guides",
"provides": {
"features": [
"guides"
]
"features": ["guides"]
}
}

View File

@@ -3,14 +3,11 @@
"display_name": "Knowledge Base",
"version": "1.0.0",
"description": "Knowledge Base articles linking to external resources",
"recommended": true,
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.1.0,<1.0.0",
"api_prefix": "/api/knowledgebase",
"provides": {
"features": [
"knowledgebase"
]
"features": ["knowledgebase"]
}
}

View File

@@ -1,23 +1,22 @@
{
"name": "machines",
"version": "1.0.0",
"description": "Machine management plugin for CNCs, CMMs, lathes, grinders, and other manufacturing machines",
"recommended": true,
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.20.0,<1.0.0",
"api_prefix": "/api/machines",
"provides": {
"asset_type": "machine",
"features": [
"machine_tracking",
"maintenance_scheduling",
"vendor_management",
"model_catalog"
]
},
"settings": {
"enable_maintenance_alerts": true,
"maintenance_alert_days": 30
}
"name": "machines",
"version": "1.0.0",
"description": "Machine management plugin for CNCs, CMMs, lathes, grinders, and other manufacturing machines",
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.20.0,<1.0.0",
"api_prefix": "/api/machines",
"provides": {
"asset_type": "machine",
"features": [
"machine_tracking",
"maintenance_scheduling",
"vendor_management",
"model_catalog"
]
},
"settings": {
"enable_maintenance_alerts": true,
"maintenance_alert_days": 30
}
}

View File

@@ -1,22 +1,22 @@
{
"name": "network",
"version": "1.0.0",
"description": "Network device management plugin for switches, APs, cameras, and IDFs",
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.20.0,<1.0.0",
"api_prefix": "/api/network",
"provides": {
"asset_type": "network_device",
"features": [
"network_device_tracking",
"port_management",
"firmware_tracking",
"poe_monitoring"
]
},
"settings": {
"enable_snmp_polling": false,
"snmp_community": "public"
}
"name": "network",
"version": "1.0.0",
"description": "Network device management plugin for switches, APs, cameras, and IDFs",
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.20.0,<1.0.0",
"api_prefix": "/api/network",
"provides": {
"asset_type": "network_device",
"features": [
"network_device_tracking",
"port_management",
"firmware_tracking",
"poe_monitoring"
]
},
"settings": {
"enable_snmp_polling": false,
"snmp_community": "public"
}
}

View File

@@ -1,16 +1,12 @@
{
"name": "notifications",
"version": "1.0.0",
"description": "Notifications and announcements management plugin",
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.1.0,<1.0.0",
"api_prefix": "/api/notifications",
"provides": {
"features": [
"notifications",
"announcements",
"calendar_events"
]
}
}
{
"name": "notifications",
"version": "1.0.0",
"description": "Notifications and announcements management plugin",
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.1.0,<1.0.0",
"api_prefix": "/api/notifications",
"provides": {
"features": ["notifications", "announcements", "calendar_events"]
}
}

View File

@@ -1,26 +1,25 @@
{
"name": "printers",
"version": "1.0.0",
"description": "Printer management plugin with Zabbix integration, supply tracking, and QR codes",
"recommended": true,
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.20.0,<1.0.0",
"api_prefix": "/api/printers",
"provides": {
"machine_category": "Printer",
"features": [
"printer_extensions",
"driver_management",
"supply_tracking",
"zabbix_integration",
"qr_codes"
]
},
"settings": {
"zabbix_url": "",
"zabbix_token": "",
"supply_alert_threshold": 10,
"default_driver_source": "internal"
}
"name": "printers",
"version": "1.0.0",
"description": "Printer management plugin with Zabbix integration, supply tracking, and QR codes",
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.20.0,<1.0.0",
"api_prefix": "/api/printers",
"provides": {
"machine_category": "Printer",
"features": [
"printer_extensions",
"driver_management",
"supply_tracking",
"zabbix_integration",
"qr_codes"
]
},
"settings": {
"zabbix_url": "",
"zabbix_token": "",
"supply_alert_threshold": 10,
"default_driver_source": "internal"
}
}

View File

@@ -23,7 +23,10 @@ from shopdb.api import (
)
from ..models import Warranty, WarrantyAsset
from ..services import get_provider, ProviderNotConfigured, WarrantyLookupError
from ..services import (
get_provider, ProviderNotConfigured, WarrantyLookupError,
run_dell_sync, ROUTE_SYNC_LIMIT,
)
warranty_bp = Blueprint('warranty', __name__)
@@ -281,14 +284,27 @@ def refresh_warranty(warrantyid):
warranty = db.session.get(Warranty, warrantyid)
if not warranty:
return error_response(ErrorCodes.NOT_FOUND, 'Warranty not found', http_code=404)
provider = get_provider(warranty.provider)
# Copy to locals and end the transaction before the vendor call: the lookup
# can block for the full HTTP timeout, and holding a pooled connection
# across it starves the pool under load. Reading ORM attributes after the
# commit would check a connection straight back out.
providername = warranty.provider
servicetag = warranty.servicetag
vendor = warranty.vendor
db.session.commit()
provider = get_provider(providername)
try:
result = provider.lookup(warranty.servicetag, warranty.vendor)
result = provider.lookup(servicetag, vendor)
except (ProviderNotConfigured, WarrantyLookupError) as exc:
return error_response(ErrorCodes.VALIDATION_ERROR, str(exc), http_code=400)
if not result:
return error_response(ErrorCodes.VALIDATION_ERROR,
'This warranty is manual - nothing to refresh.', http_code=400)
warranty = db.session.get(Warranty, warrantyid)
if not warranty:
return error_response(ErrorCodes.NOT_FOUND, 'Warranty not found', http_code=404)
if result.get('servicelevel'):
warranty.servicelevel = result['servicelevel']
if result.get('startdate'):
@@ -307,100 +323,36 @@ def refresh_warranty(warrantyid):
@warranty_bp.route('/sync/dell', methods=['POST'])
@require_permission('warranty.edit')
def sync_dell():
"""Look up every asset serial (= Dell service tag) that has no warranty yet
"""Look up asset serials (= Dell service tags) that have no warranty yet
and create warranties for the ones Dell recognizes.
Bounded to ROUTE_SYNC_LIMIT tags per call. Dell is queried 100 tags at a
time, so an unbounded fleet-sized sync is dozens of sequential HTTP calls
inside one request and a worker timeout or proxy kills it first. The
response carries `remaining` so the caller knows to run it again; nothing
drains it in the background. `flask warranty sync-dell` does the same work
from the command line for a large first-run backfill.
Only assets that lack a dated warranty are looked up, so re-running is cheap
and non-Dell serials are filtered out by Dell (they return no coverage).
"""
provider = get_provider('dell')
# ?all=true re-checks assets that already have a dated warranty too.
recheck_all = request.args.get('all', 'false').lower() == 'true'
# Assets already covered by a dated warranty - skipped unless recheck_all.
covered = set()
if not recheck_all:
for link in WarrantyAsset.query.all():
w = db.session.get(Warranty, link.warrantyid)
if w and w.isactive and w.enddate:
covered.add(link.assetid)
# Candidate assets: active, have a serial, not already covered.
candidates = (Asset.query
.filter(Asset.isactive.is_(True),
Asset.serialnumber.isnot(None),
Asset.serialnumber != '')
.all())
by_tag = {}
for asset in candidates:
if asset.assetid in covered:
continue
by_tag.setdefault(asset.serialnumber.strip().upper(), []).append(asset.assetid)
if not by_tag:
return success_response({
'candidates': 0, 'matched': 0, 'created': 0, 'updated': 0,
}, message='No PCs need a warranty lookup.')
try:
results = provider.bulk_lookup(list(by_tag.keys()))
summary = run_dell_sync(limit=ROUTE_SYNC_LIMIT, recheck_all=recheck_all)
except (ProviderNotConfigured, WarrantyLookupError) as exc:
return error_response(ErrorCodes.VALIDATION_ERROR, str(exc), http_code=400)
created = updated = matched = 0
now = datetime.now(timezone.utc).replace(tzinfo=None)
for tag, assetids in by_tag.items():
found = results.get(tag)
if not found:
continue
matched += 1
for assetid in assetids:
# Reuse an existing Dell warranty for this asset if there is one.
existing = None
for link in WarrantyAsset.query.filter_by(assetid=assetid).all():
candidate = db.session.get(Warranty, link.warrantyid)
# Reuse a warranty that is Dell by ANY signal, not just
# provider: a manually-added or imported Dell warranty carries
# provider 'manual' (create default) but the same service tag or
# a "Dell" vendor. Matching only provider=='dell' made re-check
# duplicate every one of those instead of updating it.
if candidate and (
candidate.provider == 'dell'
or (candidate.servicetag or '').strip().upper() == tag
or (candidate.vendor or '').strip().lower() == 'dell'):
existing = candidate
break
if existing:
# Canonicalize to Dell so later re-checks match by provider and
# never fall through to a duplicate.
existing.provider = 'dell'
existing.vendor = existing.vendor or 'Dell'
existing.servicelevel = found.get('servicelevel')
existing.startdate = _vendor_date(found.get('startdate'))
existing.enddate = _vendor_date(found.get('enddate'))
existing.servicetag = tag
existing.lastcheckeddate = now
updated += 1
else:
warranty = Warranty(
vendor='Dell', provider='dell', servicetag=tag,
servicelevel=found.get('servicelevel'),
startdate=_vendor_date(found.get('startdate')),
enddate=_vendor_date(found.get('enddate')),
lastcheckeddate=now,
)
warranty.links.append(WarrantyAsset(assetid=assetid))
db.session.add(warranty)
created += 1
if not summary['tags']:
return success_response(summary, message='No PCs need a warranty lookup.')
db.session.commit()
return success_response({
'candidates': sum(len(v) for v in by_tag.values()),
'tags': len(by_tag),
'matched': matched,
'created': created,
'updated': updated,
}, message=f'Dell sync: {created} added, {updated} updated, {matched} tags matched.')
message = (f"Dell sync: {summary['created']} added, {summary['updated']} updated, "
f"{summary['matched']} tags matched.")
if summary['remaining']:
message += (f" {summary['remaining']} still to check - run the sync again "
f"to continue.")
return success_response(summary, message=message)
# =============================================================================

View File

@@ -2,14 +2,11 @@
"name": "warranty",
"version": "1.0.0",
"description": "Asset warranty tracking - manual entry now, Dell/Lenovo/HP provider lookups later. Derived coverage status + report buckets.",
"recommended": true,
"author": "ShopDB Team",
"dependencies": [],
"core_version": ">=0.2.0,<1.0.0",
"api_prefix": "/api/warranty",
"provides": {
"features": [
"warranty-tracking"
]
"features": ["warranty-tracking"]
}
}

View File

@@ -10,12 +10,14 @@ import logging
from pathlib import Path
from typing import List, Dict, Optional, Type
import click
from flask import Flask, Blueprint
from shopdb.plugins.base import BasePlugin, PluginMeta
from .api import warranty_bp
from .models import Warranty, WarrantyAsset
from .services import CLI_SYNC_LIMIT
logger = logging.getLogger(__name__)
@@ -182,5 +184,44 @@ class WarrantyPlugin(BasePlugin):
"""
return ['warranty.view']
def get_cli_commands(self) -> List:
"""Return CLI commands for this plugin."""
@click.group('warranty')
def warrantycli():
"""Warranty plugin commands."""
pass
@warrantycli.command('sync-dell')
@click.option('--limit', default=None, type=int,
help=f'Tags to look up this run (default {CLI_SYNC_LIMIT}).')
@click.option('--all', 'recheck_all', is_flag=True,
help='Re-check assets that already have a dated warranty.')
def syncdell(limit, recheck_all):
"""Look up Dell warranties for a bounded chunk of pending assets.
Manual counterpart to the Dell sync button, for a large first-run
backfill where repeated clicking would be tedious. Nothing runs this
on a schedule. Each run takes one Dell call by default; assets that
already hold a dated warranty are skipped, so runs are cheap and
safely repeatable until `remaining` reaches zero.
"""
from flask import current_app
from .services import run_dell_sync, CLI_SYNC_LIMIT as default_limit
from .services import ProviderNotConfigured, WarrantyLookupError
with current_app.app_context():
try:
summary = run_dell_sync(limit=limit or default_limit,
recheck_all=recheck_all)
except (ProviderNotConfigured, WarrantyLookupError) as exc:
raise click.ClickException(str(exc))
click.echo(
f"Dell sync: {summary['tags']} tag(s) checked, "
f"{summary['matched']} matched, {summary['created']} added, "
f"{summary['updated']} updated, {summary['remaining']} remaining.")
return [warrantycli]
def init_app(self, app: Flask, db_instance) -> None:
logger.info(f"Warranty plugin initialized (v{self.meta.version})")

View File

@@ -6,5 +6,7 @@ from .providers import (
ProviderNotConfigured,
WarrantyLookupError,
)
from .dellsync import run_dell_sync, ROUTE_SYNC_LIMIT, CLI_SYNC_LIMIT
__all__ = ['get_provider', 'provider_names', 'ProviderNotConfigured', 'WarrantyLookupError']
__all__ = ['get_provider', 'provider_names', 'ProviderNotConfigured', 'WarrantyLookupError',
'run_dell_sync', 'ROUTE_SYNC_LIMIT', 'CLI_SYNC_LIMIT']

View File

@@ -0,0 +1,169 @@
"""Dell warranty sync, callable from a request or from the CLI.
Extracted from the POST /sync/dell route so the same logic can run in bounded
chunks off the request path. `bulk_lookup` calls Dell 100 tags at a time at
HTTP_TIMEOUT seconds each, so an unbounded fleet-sized sync is dozens of
sequential HTTP calls inside one web request - long enough for a gunicorn
worker timeout or a proxy to kill it first, leaving the user with a failure and
no idea what landed.
The sync is resumable by design: assets that already hold a dated warranty are
skipped, so a run that covers part of the fleet simply leaves the rest for the
next run. That is what makes chunking safe - no job queue, no progress table,
no state machine. Runs are operator-driven (the sync button, or
`flask warranty sync-dell` for a big first-run backfill); nothing drains the
remainder on a schedule, so callers report `remaining` and ask for another run.
"""
import random
from datetime import datetime, timezone
from shopdb.api import db, Asset, DateValueError, parse_date
from ..models import Warranty, WarrantyAsset
from .providers import get_provider
# One Dell call is 100 tags. The synchronous route stays a couple of calls deep
# so it cannot outlive a worker timeout; the CLI default is one call per run.
ROUTE_SYNC_LIMIT = 200
CLI_SYNC_LIMIT = 100
def _vendor_date(value):
"""A date off a VENDOR response, which this code does not control.
Kept lenient on purpose, unlike a date a person typed: a provider that
starts returning a shape we do not parse must not fail the refresh of an
otherwise good record. The field is simply left as it was.
"""
try:
return parse_date(value)
except DateValueError:
return None
def collect_pending_tags(recheck_all=False):
"""Service tags still wanting a Dell lookup, mapped to their asset ids."""
covered = set()
if not recheck_all:
for link in WarrantyAsset.query.all():
warranty = db.session.get(Warranty, link.warrantyid)
if warranty and warranty.isactive and warranty.enddate:
covered.add(link.assetid)
candidates = (Asset.query
.filter(Asset.isactive.is_(True),
Asset.serialnumber.isnot(None),
Asset.serialnumber != '')
.all())
by_tag = {}
for asset in candidates:
if asset.assetid in covered:
continue
by_tag.setdefault(asset.serialnumber.strip().upper(), []).append(asset.assetid)
return by_tag
def run_dell_sync(limit=None, recheck_all=False):
"""Look up service tags that have no warranty yet and record what Dell knows.
`limit` caps how many TAGS are sent to Dell in this run. Returns the same
counters the route has always returned plus `remaining`, so a caller can
tell the user how much of the fleet is still pending.
Raises ProviderNotConfigured / WarrantyLookupError from the provider; the
caller decides how to surface those.
"""
provider = get_provider('dell')
by_tag = collect_pending_tags(recheck_all=recheck_all)
pending = len(by_tag)
if not by_tag:
return {'candidates': 0, 'tags': 0, 'matched': 0, 'created': 0,
'updated': 0, 'remaining': 0}
tags = list(by_tag.keys())
if limit is not None and limit < len(tags):
# Shuffle before truncating. A serial Dell does not recognize never
# gains a warranty, so it stays pending forever - and under a stable
# order those same unrecognized tags would occupy every chunk and the
# sync would never reach the rest of the fleet. Random selection makes
# progress regardless of how many dead serials are in the set.
tags = random.sample(tags, limit)
by_tag = {tag: by_tag[tag] for tag in tags}
# by_tag is plain strings and ints, so it survives the commit. Release the
# connection before bulk_lookup: Dell is called 100 tags at a time and a
# fleet-sized sync would otherwise pin a pooled connection for minutes.
db.session.commit()
results = provider.bulk_lookup(tags)
# Preload every warranty already linked to a candidate asset in one query.
# Done per asset this was a query per asset plus one per link, which would
# just trade the long network hold for a long write transaction.
matched_assetids = [assetid
for tag, assetids in by_tag.items() if results.get(tag)
for assetid in assetids]
existing_by_assetid = {}
if matched_assetids:
rows = (db.session.query(WarrantyAsset.assetid, Warranty)
.join(Warranty, Warranty.warrantyid == WarrantyAsset.warrantyid)
.filter(WarrantyAsset.assetid.in_(matched_assetids))
.all())
for assetid, warranty in rows:
existing_by_assetid.setdefault(assetid, []).append(warranty)
created = updated = matched = 0
now = datetime.now(timezone.utc).replace(tzinfo=None)
for tag, assetids in by_tag.items():
found = results.get(tag)
if not found:
continue
matched += 1
for assetid in assetids:
# Reuse an existing Dell warranty for this asset if there is one.
existing = None
for candidate in existing_by_assetid.get(assetid, []):
# Reuse a warranty that is Dell by ANY signal, not just
# provider: a manually-added or imported Dell warranty carries
# provider 'manual' (create default) but the same service tag or
# a "Dell" vendor. Matching only provider=='dell' made re-check
# duplicate every one of those instead of updating it.
if (candidate.provider == 'dell'
or (candidate.servicetag or '').strip().upper() == tag
or (candidate.vendor or '').strip().lower() == 'dell'):
existing = candidate
break
if existing:
# Canonicalize to Dell so later re-checks match by provider and
# never fall through to a duplicate.
existing.provider = 'dell'
existing.vendor = existing.vendor or 'Dell'
existing.servicelevel = found.get('servicelevel')
existing.startdate = _vendor_date(found.get('startdate'))
existing.enddate = _vendor_date(found.get('enddate'))
existing.servicetag = tag
existing.lastcheckeddate = now
updated += 1
else:
warranty = Warranty(
vendor='Dell', provider='dell', servicetag=tag,
servicelevel=found.get('servicelevel'),
startdate=_vendor_date(found.get('startdate')),
enddate=_vendor_date(found.get('enddate')),
lastcheckeddate=now,
)
warranty.links.append(WarrantyAsset(assetid=assetid))
db.session.add(warranty)
created += 1
db.session.commit()
return {
'candidates': sum(len(v) for v in by_tag.values()),
'tags': len(by_tag),
'matched': matched,
'created': created,
'updated': updated,
'remaining': max(pending - len(by_tag), 0),
}

View File

@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""Compare installed package versions against the pinned lockfiles.
Editing a lockfile changes what a FUTURE install produces. It does not touch an
already-populated site-packages. On 2026-08-29 commit a8f8580 raised eight pins
carrying advisories; no install was re-run, so this workstation kept running
pyjwt 2.12.1 and idna 3.13 for three weeks while requirements.txt said 2.13.0
and 3.19. Every local test run in that window exercised different libraries than
a container build would have, and a vulnerability scan of the environment
disagreed with a scan of the lockfile.
MISSING is reported separately from MISMATCHED on purpose. The first pass at
this incident only compared versions of packages that were installed, which
silently skipped pytest-xdist and execnet - absent entirely. That surfaced later
as "pytest: error: unrecognized arguments: -n", which reads as a broken
pytest.ini rather than a stale environment, and invites deleting "-n auto"
instead of installing the plugin.
Exit codes:
0 environment matches the lockfiles
1 drift found
2 could not run the comparison (bad arguments, unreadable lockfile)
"""
import argparse
import importlib.metadata
import re
import sys
from pathlib import Path
PIN_PATTERN = re.compile(r'^([A-Za-z0-9._-]+)==([^\s\\;]+)')
DEFAULT_LOCKFILES = ('requirements.txt', 'requirements-dev.txt')
def normalize(name):
"""PEP 503 normalization, so Pillow/pillow and pytest_cov/pytest-cov agree."""
return re.sub(r'[-_.]+', '-', name).lower()
def read_pins(lockfile):
"""Map normalized package name to pinned version for one lockfile.
uv pip compile emits hashes on continuation lines and comments elsewhere;
only lines that START with name==version are pins, which is what the regex
anchors on.
"""
pins = {}
for line in lockfile.read_text(encoding='utf-8').splitlines():
match = PIN_PATTERN.match(line)
if match:
pins[normalize(match.group(1))] = match.group(2)
return pins
def compare(pins):
"""Split pinned packages into missing, mismatched, and satisfied."""
missing, mismatched, satisfied = [], [], 0
for name, wanted in sorted(pins.items()):
try:
installed = importlib.metadata.version(name)
except importlib.metadata.PackageNotFoundError:
missing.append((name, wanted))
continue
if normalize_version(installed) != normalize_version(wanted):
mismatched.append((name, installed, wanted))
else:
satisfied += 1
return missing, mismatched, satisfied
def normalize_version(version):
"""Trim a trailing .0 series so 1.2.0 and 1.2 compare equal."""
trimmed = version.strip()
while trimmed.endswith('.0'):
trimmed = trimmed[:-2]
return trimmed
def main():
parser = argparse.ArgumentParser(
description='Check installed packages against the pinned lockfiles.')
parser.add_argument('lockfiles', nargs='*', default=list(DEFAULT_LOCKFILES),
help='lockfiles to check (default: %s)'
% ' '.join(DEFAULT_LOCKFILES))
parser.add_argument('--quiet', action='store_true',
help='print only drift, not the clean summary')
arguments = parser.parse_args()
repo_root = Path(__file__).resolve().parent.parent
total_missing = total_mismatched = 0
report_lines = []
for name in arguments.lockfiles:
lockfile = Path(name)
if not lockfile.is_absolute():
lockfile = repo_root / lockfile
if not lockfile.is_file():
print('ERROR: lockfile not found: %s' % lockfile, file=sys.stderr)
return 2
pins = read_pins(lockfile)
if not pins:
print('ERROR: no pins parsed from %s' % lockfile, file=sys.stderr)
return 2
missing, mismatched, satisfied = compare(pins)
total_missing += len(missing)
total_mismatched += len(mismatched)
for package, wanted in missing:
report_lines.append(' MISSING %s (pinned %s) [%s]'
% (package, wanted, lockfile.name))
for package, installed, wanted in mismatched:
report_lines.append(' MISMATCH %s installed %s, pinned %s [%s]'
% (package, installed, wanted, lockfile.name))
if not arguments.quiet and not missing and not mismatched:
print('OK: %s - %d pins satisfied' % (lockfile.name, satisfied))
if report_lines:
print('FAIL: dependency drift - %d missing, %d mismatched:'
% (total_missing, total_mismatched))
for line in report_lines:
print(line)
print('')
print('Interpreter: %s' % sys.executable)
print('Fix with:')
for name in arguments.lockfiles:
print(' python -m pip install --require-hashes -r %s' % name)
return 1
return 0
if __name__ == '__main__':
sys.exit(main())

View File

@@ -608,6 +608,67 @@ if [ -n "$SITE_HITS" ]; then
fi
fi
# --- Dependency drift -------------------------------------------------------
#
# Not a naming rule, but it belongs to the same question: can this run be
# trusted. A stale environment makes every OTHER check here a statement about
# libraries that are not the ones the lockfile specifies.
#
# ADVISORY BY DEFAULT, and deliberately so. Set DEPENDENCY_DRIFT_ENFORCE=1 to
# fail the gate (CI should). Two considerations pull against each other here:
# a warning nobody reads is worthless, but a blocking check wedges anyone who
# is mid-upgrade and legitimately holds an unpinned version for an afternoon.
# The tie-breaker is the comment above the site-literal scan: this file has
# twice acquired a reasonable-looking flag that silently disabled a check, once
# for months with the gate green. A check that ANNOUNCES it is advisory cannot
# become that, because its output changes the moment drift appears.
#
# Skipped when python is absent rather than failing: this script is also run on
# machines that only build the frontend.
#
# Each candidate is TEST-EXECUTED, not merely located with command -v. On
# Windows, "python3" resolves to the Microsoft Store app-execution alias - a
# zero-byte stub that exists on PATH, satisfies command -v, and then prints
# "Python was not found; run without arguments to install from the Microsoft
# Store" to stderr and exits non-zero. Locating it and trusting it reported the
# drift check as failing on a machine whose environment was in fact clean.
DEPENDENCY_DRIFT_ENFORCE=${DEPENDENCY_DRIFT_ENFORCE:-0}
echo "==> Checking installed packages against the lockfiles..."
DRIFT_PYTHON=""
for CANDIDATE in python python3 py; do
if command -v "$CANDIDATE" >/dev/null 2>&1 \
&& "$CANDIDATE" -c 'import sys' >/dev/null 2>&1; then
DRIFT_PYTHON="$CANDIDATE"
break
fi
done
# The checker and the lockfiles must both be present. They are not when the gate
# is copied alone into a throwaway tree, which is exactly what
# tests/test_naming_gate_migrations.py does to exercise the non-git fallback
# branch. Absent inputs mean "nothing to compare", not "drift".
if [ ! -f scripts/check-dependency-drift.py ] || [ ! -f requirements.txt ]; then
DRIFT_PYTHON=""
fi
if [ -z "$DRIFT_PYTHON" ]; then
echo " python not found, skipping dependency drift check"
echo
else
DRIFT_OUTPUT=$("$DRIFT_PYTHON" scripts/check-dependency-drift.py --quiet 2>&1) || DRIFT_STATUS=$?
DRIFT_STATUS=${DRIFT_STATUS:-0}
if [ "$DRIFT_STATUS" -ne 0 ]; then
if [ "$DEPENDENCY_DRIFT_ENFORCE" = "1" ]; then
echo "$DRIFT_OUTPUT"
echo
VIOLATIONS=$((VIOLATIONS + 1))
else
echo " dependency drift found (report only, not failing):"
echo "$DRIFT_OUTPUT" | sed 's/^/ /'
echo " Set DEPENDENCY_DRIFT_ENFORCE=1 to make this fail the gate."
echo
fi
fi
fi
if [ "$VIOLATIONS" -gt 0 ]; then
echo "=================================================="
echo "$VIOLATIONS naming/style violation(s) found."

View File

@@ -1199,7 +1199,7 @@ def _inventory_app():
raise SystemExit(
'these bundled plugins are disabled, so their routes would be '
'missing\nfrom the inventory: %s\nenable them and re-run:\n'
' FLASK_APP=shopdb venv/bin/flask plugin enable %s'
' FLASK_APP=shopdb venv/bin/flask plugins enable %s'
% (', '.join(off), off[0]))
return app

View File

@@ -264,11 +264,6 @@ def create_app(config_name: str = None) -> Flask:
pass # DB not ready yet, skip
plugin_manager.init_app(app, db)
# Recycle this worker when the enabled-plugin set changes underneath it, so
# a toggle takes effect without the operator restarting anything by hand.
from shopdb.plugins.recycle import init_recycle
init_recycle(app)
# Register core blueprints
register_blueprints(app)

View File

@@ -859,6 +859,38 @@ def seed_settings():
click.echo(click.style(f"{created} default settings created.", fg='green'))
@seed_cli.command('shipped-scripts')
@with_appcontext
def seed_shipped_scripts():
"""Seed managed scripts that ship with the build into the script library.
Idempotent: an unchanged template rewrites nothing and leaves the stored
digest alone. Safe to run on every deploy, which is the point - a managed
script row edited in the UI drifts back to the shipped definition here.
Guarded: a lean build without the endpoints plugin has no shipped scripts
to seed, so the command reports that and exits rather than crashing the
seed chain it runs in.
"""
try:
from plugins.endpoints.api import reportipscript
except ImportError:
click.echo('Endpoints plugin is not installed - no shipped scripts to seed.')
return
seeded = reportipscript.seed_report_ip_script()
if seeded['created']:
state = 'created'
elif seeded['updated']:
state = 'updated'
else:
state = 'already current'
click.echo(click.style(
f"{seeded['scriptname']}: {state} "
f"(scriptid={seeded['scriptid']}, {seeded['bytes']} bytes)",
fg='green'))
# Demo assets carry this assetnumber prefix so a re-run skips what it made and
# an operator can bulk-delete them without touching imported/real rows.
DEMO_PREFIX = 'DEMO-'

View File

@@ -185,6 +185,21 @@ class Config:
AUTH_RATELIMIT_WINDOW_SECONDS = int(
os.environ.get('AUTH_RATELIMIT_WINDOW_SECONDS', 300))
# Swagger UI "try it" page at /api/docs/try, beside the read-only Redoc
# page at /api/docs. One spec, two renderers.
#
# Off unless a site turns it on, and that is the point rather than caution:
# Swagger UI's Execute button fires a REAL request at THIS server from the
# operator's browser, carrying whatever session that browser already holds.
# On a production box that is a one-click way to POST against live fleet
# data from a page anyone can open. Redoc, which cannot send anything, stays
# the default and is always on.
#
# When off the routes are never bound at all, so /api/docs/try is Flask's
# own 404 and the Redoc page does not mention it. See
# shopdb/core/api/docs.py.
API_DOCS_TRYIT_ENABLED = os.environ.get(
'API_DOCS_TRYIT_ENABLED', 'false').lower() == 'true'
DEFAULT_PAGE_SIZE = 20
MAX_PAGE_SIZE = 100
@@ -218,6 +233,8 @@ class DevelopmentConfig(Config):
# On in development: firing the request is the whole reason a developer
# opens the API docs. Still read from the environment, so a shared dev
# instance can turn it back off without editing this file.
API_DOCS_TRYIT_ENABLED = os.environ.get(
'API_DOCS_TRYIT_ENABLED', 'true').lower() == 'true'
SQLALCHEMY_DATABASE_URI = os.environ.get(
'DATABASE_URL',
@@ -248,6 +265,7 @@ class TestingConfig(Config):
# keeps describing what production actually exposes; the tests that ARE
# about the page build their own app with the flag on
# (tests/test_core/test_api_docs_tryit.py).
API_DOCS_TRYIT_ENABLED = False
SQLALCHEMY_DATABASE_URI = 'sqlite:///:memory:'
SQLALCHEMY_ENGINE_OPTIONS = {
'connect_args': {'check_same_thread': False},

View File

@@ -6,8 +6,11 @@ The spec is docs/openapi.json in the repo (regenerate with
scripts/gen_openapi.py after API changes).
Two renderers over ONE spec. Redoc is read-only by design and is the default,
because what is published is a contract.
because what is published is a contract. Swagger UI at /api/docs/try adds the
one thing Redoc will not do: fire a real request from the page, against this
server, with whatever session the browser is already carrying. That is its
whole value and the whole reason it is off unless a site turns it on - see
API_DOCS_TRYIT_ENABLED in shopdb/config.py.
"""
import json
@@ -26,6 +29,43 @@ _SPEC = os.path.join(_REPO_ROOT, 'docs', 'openapi.json')
_LLMS = os.path.join(_REPO_ROOT, 'docs', 'llms.txt')
_REDOC = os.path.join(_HERE, 'staticdocs', 'redoc.standalone.js')
# Vendored from swagger-ui-dist 5.33.0. TWO files out of the ~30 in that dist,
# because those are the two the page below actually loads:
# swagger-ui.css - the stylesheet. Its images are inline data: URIs,
# so it pulls nothing else. The trailing
# sourceMappingURL comment was stripped: the .map
# is not vendored, and left in place it turns every
# devtools open into a 404 on a box with no
# internet to fall back on.
# swagger-ui-bundle.min.js - the renderer plus the core presets, self-
# contained. NOT vendored beside it:
# swagger-ui-standalone-preset.js (the topbar with
# the "load any spec URL" box - this page serves
# one spec on purpose), the ES bundles (module
# builds of the same code), the .map files (~10 MB
# of source maps that only devtools reads),
# oauth2-redirect.html (no OAuth flow here), and
# the favicons.
# The upstream name is swagger-ui-bundle.js; the vendored copy carries .min.js,
# which is what it is - a minified single-line build. That suffix is also what
# scripts/check-naming-and-style.sh already filters vendored bundles by, so the
# style gate does not report thousands of hits in code nobody here wrote.
# swagger-ui.LICENSE.txt is the Apache-2.0 text that ships with the dist. It is
# never served; it is here because redistributing the bundle requires it.
_SWAGGER_CSS = os.path.join(_HERE, 'staticdocs', 'swagger-ui.css')
_SWAGGER_JS = os.path.join(_HERE, 'staticdocs', 'swagger-ui-bundle.min.js')
_TRYIT_ENDPOINT = 'docs.swagger_tryit'
def _tryit_is_mounted():
"""True when the try-it page exists on THIS app.
Reads the url map rather than the config flag on purpose: the routes are
bound once, at blueprint registration, so a config value changed after boot
would have the Redoc page advertise a link that answers 404.
"""
return _TRYIT_ENDPOINT in current_app.view_functions
@docs_bp.route('/', strict_slashes=False)
@@ -33,6 +73,16 @@ def docs_index():
"""Redoc page. url_for keeps the asset URLs correct under any mount."""
spec_url = url_for('docs.openapi_spec')
redoc_url = url_for('docs.redoc_js')
# Mentioned only when it is actually there. An unconditional note would be
# an advert for a 404 on every box that ships the shipped default.
tryit_note = ''
if _tryit_is_mounted():
tryit_note = (
'<p class="docs-note">'
'This page is read-only. To send a real request against this '
'server, use <a href="%s">the try-it view</a>.</p>'
% url_for(_TRYIT_ENDPOINT)
)
# DARK MODE. Redoc does not read prefers-color-scheme on its own, and
# the <redoc spec-url> element takes no theme - so the page goes through
# Redoc.init instead, which does. The app has a dark theme and this was
@@ -47,163 +97,52 @@ def docs_index():
'<meta charset="utf-8">'
'<meta name="viewport" content="width=device-width, initial-scale=1">'
'<style>'
# color-scheme follows the SAME attribute Redoc is themed from, so the
# OS never paints dark scrollbars under a page the user set to light.
':root{color-scheme:light}'
':root[data-theme="dark"]{color-scheme:dark}'
':root{color-scheme:light dark}'
'body{margin:0;padding:0;background:#fff}'
'.docs-note{font:14px/1.6 system-ui,-apple-system,Segoe UI,sans-serif;'
'margin:0;padding:10px 16px;background:#f5f5f5;color:#1a1a2e;'
'border-bottom:1px solid #dee2e6}'
'.docs-note a{color:#2d6ce0}'
# The ONLY Redoc elements with no theme key. Each is a colour hard-coded
# in the vendored bundle, so it cannot be reached from the theme object:
# h5 section headings ("query Parameters"),
# rgba(38,50,56,.5) - fails AA on light too
# td[colspan] > div nested schema tables at depth 2 are
# painted #ffffff whatever nestedBackground
# says, and the response envelopes nest 3
# deep, so it shows on every operation
# .react-tabs__tab--selected right-panel tab paints text.primary on
# rightPanel.textColor; both are light in
# dark mode, so the selected tab label
# vanished into its own background
# Selectors use Redoc's own DOM - tag, colspan attribute, react-tabs
# class - never generated class hashes, which change on every build.
'#redoc h5{color:#5c636a;border-bottom-color:#dee2e6}'
':root[data-theme="dark"] body{background:#0a0a1a}'
':root[data-theme="dark"] .docs-note{background:#0d0d2b;color:#e6e6e8;'
'border-bottom-color:rgba(65,129,255,0.2)}'
':root[data-theme="dark"] .docs-note a{color:#4181ff}'
':root[data-theme="dark"] #redoc h5{color:#9d9da3;'
'border-bottom-color:rgba(255,255,255,0.2)}'
':root[data-theme="dark"] #redoc td[colspan="2"]>div td[colspan="2"]>div'
'{background:#0a0a1a}'
':root[data-theme="dark"] #redoc td[colspan="2"]>div td[colspan="2"]>div '
'td[colspan="2"]>div{background:#0d0d2b}'
':root[data-theme="dark"] #redoc .react-tabs__tab--selected{color:#0a0a1a}'
# The content-type control is a native <select>. Redoc never styles it,
# so it keeps the OS default (black text, transparent background) and
# reads 1.07:1 on the dark right panel. Name both halves; the border
# keeps it looking like a control rather than stray text.
# The content-type label ("application/json") is a span INSIDE that
# same h5, carrying its own hard-coded black, so theming the h5 alone
# left it at 1.07:1 on the dark panel. Inherit rather than restate, so
# it tracks whatever the heading colour is.
'#redoc h5 span{color:inherit}'
# Redoc's footer badge links to redocly.com and its logo was replaced
# with a blank GIF so an air-gapped box stops reaching for a CDN, so
# what remains is a dead external link on a shop-floor kiosk. The MIT
# licence requires the notice be retained in the DISTRIBUTION, which
# staticdocs/redoc.standalone.js.LICENSE.txt covers; it does not
# require rendering a badge.
'#redoc a[href*="redocly.com"]{display:none}'
'.docs-note{font:14px/1.4 system-ui,sans-serif;margin:0;'
'padding:10px 16px;background:#eef2f7;color:#0f172a;'
'border-bottom:1px solid #cbd5e1}'
'.docs-note a{color:#1d4ed8}'
'@media (prefers-color-scheme: dark){'
'body{background:#0f172a}'
'.docs-note{background:#1e293b;color:#e2e8f0;'
'border-bottom-color:#334155}'
'.docs-note a{color:#7dd3fc}'
'}'
'</style></head><body>'
'%s'
'<div id="redoc"></div>'
'<script src="%s"></script>'
'<script>'
'var SPEC = %s;'
# Product type: the app's body stack and its mono stack. Inter is
# bundled into the SPA under a hashed filename this standalone page
# cannot reference, so system-ui is the honest choice (Segoe UI on the
# Windows kiosks).
'var FONT="system-ui,-apple-system,Segoe UI,sans-serif";'
'var MONO="SF Mono,Monaco,Consolas,monospace";'
# The product chrome is the same navy with white text in BOTH themes,
# so one object serves both. Redoc deep-merges the theme into a fresh
# copy of its defaults and snapshots it, so sharing is safe.
'var SIDEBAR={backgroundColor:"#00003d",'
'textColor:"rgba(255,255,255,0.85)",activeTextColor:"#ffffff",'
'groupItems:{activeBackgroundColor:"rgba(255,255,255,0.1)"},'
'level1Items:{activeBackgroundColor:"rgba(255,255,255,0.1)"},'
'arrow:{color:"rgba(255,255,255,0.85)"}};'
# Light. The middle column is the product CARD white. primary is
# --primary-dark rather than --primary, because --primary is 3.6:1 on
# white and fails AA; success and error are a shade darker than the
# product tokens for the same reason - those tokens are designed as
# fills, not as text on the nested panels where type names and required
# labels live. tabTextColor is pinned because the response tabs sit on
# the DARK code block in both themes, and Redoc would otherwise derive
# them from the light middle-panel colours.
'var LIGHT={colors:{primary:{main:"#2d6ce0",light:"#2d6ce0"},'
'success:{main:"#0b7a3b"},'
'warning:{main:"#ff9500",contrastText:"#1a1a2e"},'
'error:{main:"#c1273a"},'
'text:{primary:"#1a1a2e",secondary:"#5c636a"},'
'border:{dark:"#dee2e6"},'
'gray:{50:"#f5f5f5",100:"#f5f5f5"},'
# tabTextColor is deliberately NOT pinned here. It was, on the
# assumption that these tabs sit on the dark code block in both
# themes; measuring the rendered page showed the light-theme tab list
# on the white panel, where the bright green read 1.95:1. Unset, Redoc
# derives it from success.main, which is chosen to pass on white.
'responses:{}},'
'schema:{nestedBackground:"#f5f5f5"},'
'typography:{fontSize:"15px",lineHeight:"1.6em",fontFamily:FONT,'
'fontWeightBold:"600",headings:{fontFamily:FONT,fontWeight:"600"},'
'code:{fontFamily:MONO,color:"#1a1a2e",'
'backgroundColor:"rgba(26,26,46,0.06)"},'
'links:{hover:"#4181ff"}},'
'sidebar:SIDEBAR,'
'rightPanel:{backgroundColor:"#00003d",textColor:"#ffffff",'
'servers:{overlay:{backgroundColor:"#f5f5f5",textColor:"#1a1a2e"},'
'url:{backgroundColor:"#ffffff"}}},'
'codeBlock:{backgroundColor:"#0a0a1a"}};'
# Dark. Product tokens verbatim, with the two alpha text tokens
# flattened onto the page background so Redoc's own lighten and
# transparentize derivations stay predictable. The response accordion
# tints are named because Redoc derives a 7% tint that is invisible on
# a near-black page.
'var DARK={colors:{primary:{main:"#4181ff"},'
'success:{main:"#0ad64f"},'
'warning:{main:"#ff9500",contrastText:"#1a1a2e"},'
'error:{main:"#f5365c"},'
'text:{primary:"#e6e6e8",secondary:"#9d9da3"},'
'border:{dark:"rgba(65,129,255,0.2)"},'
'gray:{50:"#0d0d2b",100:"#0d0d2b"},'
'responses:{success:{backgroundColor:"rgba(10,214,79,0.12)"},'
'error:{backgroundColor:"rgba(245,54,92,0.12)"}}},'
'schema:{nestedBackground:"#0d0d2b"},'
'typography:{fontSize:"15px",lineHeight:"1.6em",fontFamily:FONT,'
'fontWeightBold:"600",headings:{fontFamily:FONT,fontWeight:"600"},'
'code:{fontFamily:MONO,color:"#e6e6e8",'
'backgroundColor:"rgba(255,255,255,0.08)"},'
'links:{hover:"#7aa8ff"}},'
'sidebar:SIDEBAR,'
'rightPanel:{backgroundColor:"#00003d",textColor:"#ffffff",'
'servers:{overlay:{backgroundColor:"#0d0d2b",textColor:"#e6e6e8"},'
'url:{backgroundColor:"#0a0a1a"}}},'
'codeBlock:{backgroundColor:"#0a0a1a"}};'
# THEME SOURCE. The app stores an explicit choice under this key and
# stamps <html data-theme>. This page is served from the same origin,
# so it reads the same key rather than guessing from the OS - picking
# light in the app on a dark-OS kiosk used to give a dark docs page.
# The OS scheme is only the fallback, exactly as in the app's store.
# localStorage can throw outright on a locked-down kiosk browser.
'var KEY="shopdb-theme";'
# Redoc computes text and the right-hand sample panel from the
# background and gets both wrong on a dark one, so they are named
# rather than left to it.
'var DARK={colors:{primary:{main:"#7dd3fc"},'
'text:{primary:"#e2e8f0",secondary:"#94a3b8"},'
'border:{dark:"#334155",light:"#1e293b"}},'
'sidebar:{backgroundColor:"#111827",textColor:"#e2e8f0",'
'activeTextColor:"#7dd3fc"},'
'rightPanel:{backgroundColor:"#020617",textColor:"#e2e8f0"},'
'codeBlock:{backgroundColor:"#0b1220"},'
'schema:{nestedBackground:"#0b1220"},'
'typography:{fontFamily:"system-ui,sans-serif",'
'code:{color:"#e2e8f0",backgroundColor:"#0b1220"}}};'
'var LIGHT={colors:{primary:{main:"#1d4ed8"}},'
'typography:{fontFamily:"system-ui,sans-serif"}};'
'var q=window.matchMedia("(prefers-color-scheme: dark)");'
'var el=document.getElementById("redoc");'
'function pick(){var t=null;'
'try{t=window.localStorage.getItem(KEY);}catch(e){}'
'if(t!=="dark"&&t!=="light"){t=q.matches?"dark":"light";}'
'return t;}'
# Re-initialise rather than restyle: Redoc bakes colours into
# styled-components at init. data-theme is stamped first so the CSS
# above and the Redoc theme can never disagree about which one is live.
'function draw(){var t=pick();'
'document.documentElement.setAttribute("data-theme",t);'
'el.innerHTML="";'
'Redoc.init(SPEC,{theme:t==="dark"?DARK:LIGHT},el);}'
'function draw(){'
'document.getElementById("redoc").innerHTML="";'
'Redoc.init(SPEC,{theme:q.matches?DARK:LIGHT},'
'document.getElementById("redoc"));'
'}'
'draw();'
# addEventListener is missing from MediaQueryList in older WebViews,
# and a shop-floor kiosk browser is exactly where this gets read.
'if(q.addEventListener){q.addEventListener("change",draw);}'
'else if(q.addListener){q.addListener(draw);}'
# Fires in THIS tab when the app in another tab toggles the theme.
'window.addEventListener("storage",function(e){'
'if(e.key===KEY){draw();}});'
'</script>'
'</body></html>' % (redoc_url, json.dumps(spec_url))
'</body></html>' % (tryit_note, redoc_url, json.dumps(spec_url))
)
return Response(page, mimetype='text/html')
@@ -246,107 +185,12 @@ def _spec_with_server():
@docs_bp.route('/openapi.json')
def openapi_spec():
"""The generated OpenAPI 3.1 spec (machine + LLM readable).
?tag=core-assets narrows it to one tag group. The whole spec is ~700 KB
across 483 operations, which is more than a caller generating a client for
one area needs and more than most model context windows should spend. The
filtered copy keeps components intact, because a pruned schema graph is
worse than a large one: a $ref that resolves to nothing turns a working
generator into a confusing failure.
"""
"""The generated OpenAPI 3.1 spec (machine + LLM readable)."""
if not os.path.isfile(_SPEC):
return Response('{"error":"openapi.json not generated"}',
status=404, mimetype='application/json')
spec = _spec_with_server()
wanted = (request.args.get('tag') or '').strip()
if wanted:
paths = {}
for path, methods in (spec.get('paths') or {}).items():
kept = {m: op for m, op in methods.items()
if not isinstance(op, dict) or wanted in (op.get('tags') or [])}
# A path whose only survivors are shared keys (parameters, summary)
# carries no operation and would read as an endpoint that exists
# and does nothing.
if any(isinstance(op, dict) and op.get('tags') for op in kept.values()):
paths[path] = kept
if not paths:
return Response(
json.dumps({'error': f'no operations tagged {wanted!r}',
'tags': sorted(_operation_tags(spec))}),
status=404, mimetype='application/json')
spec = dict(spec, paths=paths)
return Response(json.dumps(spec), mimetype='application/json')
def _operation_tags(spec):
"""Every tag that actually carries an operation."""
tags = set()
for methods in (spec.get('paths') or {}).values():
for op in methods.values():
if isinstance(op, dict):
tags.update(op.get('tags') or [])
return tags
@docs_bp.route('/operations.txt')
def operations_index():
"""Every operation, one line each: method, path, tag, auth, summary.
WHY THIS EXISTS. The spec is the reference and it is ~700 KB; this is the
index, and it is ~40 KB. A caller - a person or an agent writing a script -
can hold every one of the 483 operations at once here, find the four it
needs, and only then fetch their schemas from openapi.json?tag=... Handing
over the whole spec instead means either truncating it or spending a context
window on 479 operations nobody asked about.
Plain text on purpose: it is meant to be read and grepped, not parsed.
"""
if not os.path.isfile(_SPEC):
return Response('openapi.json not generated', status=404,
mimetype='text/plain')
spec = _spec_with_server()
servers = spec.get('servers') or [{'url': '/'}]
rows = []
for path, methods in sorted((spec.get('paths') or {}).items()):
for method, op in sorted(methods.items()):
if not isinstance(op, dict) or method.upper() not in _HTTP_METHODS:
continue
tag = (op.get('tags') or ['-'])[0]
# security == [] means public. A list containing an EMPTY dict
# means the scheme is optional - the route works signed out and
# returns more when authenticated - and 14 routes here are that
# shape. Collapsing them to the scheme name told a caller reading
# only the index that it needed a token for a route that does not
# require one.
security = op.get('security')
if security == []:
auth = 'public'
else:
entries = security or []
schemes = sorted({k for entry in entries for k in entry})
optional = any(not entry for entry in entries)
auth = ','.join(schemes) or 'bearer'
if optional:
auth += ',optional'
summary = ' '.join((op.get('summary') or '').split())
rows.append(f'{method.upper():<6} {path:<52} [{tag}] ({auth}) {summary}')
header = (
'# ShopDB API - operation index\n'
f'# {len(rows)} operations. Base URL: {servers[0].get("url", "/")}\n'
'#\n'
'# This is the INDEX. Schemas, parameters and examples are in the spec:\n'
'# /api/docs/openapi.json full (large)\n'
'# /api/docs/openapi.json?tag=<tag> one group\n'
'# Orientation for an agent, including the response envelope and auth:\n'
'# /api/docs/llms.txt\n'
'#\n'
'# METHOD path [tag] (auth) summary\n'
)
return Response(header + '\n'.join(rows) + '\n', mimetype='text/plain')
_HTTP_METHODS = {'GET', 'POST', 'PUT', 'PATCH', 'DELETE', 'HEAD', 'OPTIONS'}
return Response(json.dumps(_spec_with_server()),
mimetype='application/json')
@docs_bp.route('/redoc.standalone.js')
@@ -361,3 +205,92 @@ def llms_txt():
if not os.path.isfile(_LLMS):
return Response('llms.txt not found', status=404, mimetype='text/plain')
return send_file(_LLMS, mimetype='text/plain')
def swagger_tryit():
"""Swagger UI over the SAME spec the Redoc page renders.
Every URL on the page comes from url_for, so the spec and both assets stay
same-origin and correct under a mount path (prod runs under /shopdb).
"""
spec_url = url_for('docs.openapi_spec')
css_url = url_for('docs.swagger_css')
js_url = url_for('docs.swagger_js')
redoc_page_url = url_for('docs.docs_index')
page = (
'<!doctype html><html><head>'
'<title>ShopDB API - try it</title>'
'<meta charset="utf-8">'
'<meta name="viewport" content="width=device-width, initial-scale=1">'
'<link rel="stylesheet" href="%(css)s">'
'<style>body{margin:0;padding:0}'
'.tryit-warn{font:14px/1.4 system-ui,sans-serif;background:#7f1d1d;'
'color:#fff;margin:0;padding:10px 16px}'
'.tryit-warn a{color:#fff}</style>'
'</head><body>'
'<p class="tryit-warn"><b>Execute sends a real request to this '
'server</b>, as whoever is signed in on this browser. A write '
'operation changes live data. The read-only contract is at '
'<a href="%(redoc)s">/api/docs</a>.</p>'
'<div id="swagger-ui"></div>'
'<script src="%(js)s"></script>'
'<script>\n'
'window.ui = SwaggerUIBundle({\n'
# json.dumps, NOT a bare "%s". SCRIPT_NAME is proxy-controlled and
# ProxyFix honours X-Forwarded-Prefix, so a mount path carrying a quote
# closed this string literal and executed what followed: SCRIPT_NAME of
# /a"+alert(1)+" rendered url: "/a"+alert(1)+"/api/docs/openapi.json".
# Escaping it as JSON is what makes the value data rather than code.
' url: %(spec)s,\n'
' dom_id: "#swagger-ui",\n'
' presets: [SwaggerUIBundle.presets.apis],\n'
' layout: "BaseLayout",\n'
' deepLinking: true,\n'
' filter: true,\n'
# 573 operations. Expanded by default, the page spends many seconds
# laying out before it answers a scroll.
' docExpansion: "none",\n'
' withCredentials: true,\n'
# THE air-gap setting. Left unset, Swagger UI renders a "valid" badge
# by handing this spec to the public swagger validator - an outbound
# request that cannot succeed here, and that would ship the internal
# API surface off the box if it ever could.
' validatorUrl: null\n'
'});\n'
'</script>'
'</body></html>'
% {'css': escape(css_url), 'js': escape(js_url),
'spec': json.dumps(spec_url),
'redoc': escape(redoc_page_url)}
)
return Response(page, mimetype='text/html')
def swagger_css():
"""Vendored Swagger UI stylesheet (offline)."""
return send_file(_SWAGGER_CSS, mimetype='text/css')
def swagger_js():
"""Vendored Swagger UI bundle (offline)."""
return send_file(_SWAGGER_JS, mimetype='application/javascript')
@docs_bp.record_once
def _mount_tryit(state):
"""Bind the try-it routes only when the flag is on for this app.
Binding at registration rather than checking the flag inside the view, so
that when it is off the routes DO NOT EXIST: the URL gets Flask's own 404
and confirms nothing, and the unauthenticated-GET sweep in
tests/test_core/test_public_read_surface.py never has to make an exception
for three routes that are only meant to be reachable on a dev box.
"""
if not state.app.config.get('API_DOCS_TRYIT_ENABLED'):
return
state.add_url_rule('/try', endpoint='swagger_tryit',
view_func=swagger_tryit, strict_slashes=False)
state.add_url_rule('/swagger-ui.css', endpoint='swagger_css',
view_func=swagger_css)
state.add_url_rule('/swagger-ui-bundle.min.js', endpoint='swagger_js',
view_func=swagger_js)

View File

@@ -123,10 +123,6 @@ def set_plugin_enabled(name: str):
return success_response(
{'name': name, 'enabled': want},
# No longer "restart to apply": each worker notices the change to
# plugins.json and recycles itself, so the routes follow on their own
# within seconds. Callers poll /setup/health to know when that has
# finished across the pool.
message=f"Plugin {'enabled' if want else 'disabled'}. "
f"It comes online as the workers pick up the change."
message=f"Plugin {'enabled' if want else 'disabled'} "
f"(restart to apply route changes)"
)

View File

@@ -155,367 +155,6 @@ def seed_starter():
)
def _setting_values():
"""Every setting as {key: value}, read once.
The checklist inspects roughly forty keys across a dozen categories, so it
reads the table once rather than issuing a query per item.
"""
return {row.key: (row.value or '') for row in Setting.query.all()}
def _truthy(value):
return str(value or '').strip().lower() in ('true', '1', 'yes', 'on')
def _filled(values, *keys):
"""True only when every named key has a non-empty value."""
return all(str(values.get(k) or '').strip() for k in keys)
# Values that are SEEDED, not chosen. A setting holding one of these is still
# at its shipped default, so reporting it as configured would tell a site it had
# done something it has not: site_logo ships as the GE mark and the floor map
# ships as a placeholder image, and both are non-empty from the first boot.
SHIPPED_DEFAULTS = {
'site_logo': ('/ge-aerospace-logo.svg',),
'map_blueprint_light': ('/static/images/floorplan-placeholder.svg',),
'map_blueprint_dark': ('/static/images/floorplan-placeholder.svg',),
}
def _customised(values, key):
"""True when a key holds something other than what shipped with it."""
value = str(values.get(key) or '').strip()
if not value:
return False
if value in SHIPPED_DEFAULTS.get(key, ()):
return False
# Defensive: a renamed placeholder asset should still read as unconfigured.
return 'placeholder' not in value.lower()
def _area(key, label, status, detail, route, wizard=False):
return {'key': key, 'label': label, 'status': status,
'detail': detail, 'route': route, 'inwizard': wizard}
def _enabled_plugin_names():
"""Plugins the registry says are on. Empty when plugins are unavailable."""
from shopdb.plugins import plugin_manager
registry = getattr(plugin_manager, 'registry', None)
if registry is None:
return set()
try:
return set(registry.get_enabled_plugins())
except Exception: # a corrupt registry must not break the checklist
return set()
def _loaded_plugin_names():
"""Plugins actually loaded into THIS process."""
from shopdb.plugins import plugin_manager
loader = getattr(plugin_manager, 'loader', None)
if loader is None:
return set()
try:
return set(loader.get_all_loaded().keys())
except Exception:
return set()
def _manager_call(method):
"""Call a plugin-manager method by name, tolerating its absence."""
from shopdb.plugins import plugin_manager
getter = getattr(plugin_manager, method, None)
if getter is None:
return set()
try:
return set(getter())
except Exception:
return set()
def _enabled_on_disk():
"""Enabled plugins per plugins.json, or None when it cannot be read.
None and the empty set mean different things: "I could not tell you" versus
"nothing is enabled". Collapsing them made an all-disabled site fall back to
a stale in-memory view.
"""
from shopdb.plugins import plugin_manager
getter = getattr(plugin_manager, 'enabled_on_disk', None)
if getter is None:
return None
try:
return set(getter())
except Exception:
return None
def _booted_with_names():
return _manager_call('get_booted_with')
def _deferred_registration_names():
"""Plugins whose routes are waiting on a restart in this process."""
from shopdb.plugins import plugin_manager
getter = getattr(plugin_manager, 'get_deferred_registrations', None)
if getter is None:
return set()
try:
return set(getter())
except Exception:
return set()
def _build_checklist(values):
"""Configured-vs-not for every area a site is expected to set up.
WHY THIS EXISTS. The wizard covers only what a site cannot run without;
everything else would otherwise be invisible until someone went looking for
it in Settings. There are well over a hundred settings, and a wizard that
asked for all of them would be worse than no wizard. This is the other half:
a standing list of what is still unconfigured, with a link straight to the
page that configures it.
Status is deliberately three-valued. 'partial' matters because several areas
have an enable flag plus the fields that flag needs - SAML switched on with
no IdP metadata is a broken site, and reporting that as 'ok' would hide it.
"""
enabled = _enabled_plugin_names()
areas = []
# --- identity -------------------------------------------------------
# Only facility_name is genuinely required. The wizard offers "blank = use
# the browsing origin" for site_base_url and "Use the server timezone" for
# site_timezone, and saves '' for both - so demanding them here told an
# operator who took an option the wizard offered that they had not finished.
if _filled(values, 'facility_name'):
status, detail = 'ok', values.get('facility_name', '')
else:
status, detail = 'todo', 'Facility name is unset'
areas.append(_area('site_identity', 'Site identity', status, detail,
'/settings/siteconfig/site', wizard=True))
# --- authentication -------------------------------------------------
if _truthy(values.get('saml_enabled')):
if _filled(values, 'saml_entity_id', 'saml_acs_url', 'saml_idp_metadata_url'):
status, detail = 'ok', 'SAML configured'
else:
status, detail = 'partial', 'SAML is on but the IdP fields are incomplete'
else:
status, detail = 'todo', 'Local accounts only; SAML is off'
areas.append(_area('authentication', 'Authentication', status, detail,
'/settings/security/authentication', wizard=True))
# --- email ----------------------------------------------------------
if _truthy(values.get('smtp_enabled')):
if _filled(values, 'smtp_host', 'smtp_from_address'):
status, detail = 'ok', values.get('smtp_host', '')
else:
status, detail = 'partial', 'SMTP is on but host or from-address is unset'
else:
status, detail = 'todo', 'No mail server; alerts and notifications cannot send'
areas.append(_area('email', 'Email / SMTP', status, detail,
'/settings/email', wizard=True))
# --- backup destination ---------------------------------------------
kind = (values.get('backup_destination_kind') or '').strip().lower()
if kind == 'smb' and _filled(values, 'backup_smb_host', 'backup_smb_share'):
status, detail = 'ok', f"SMB: {values.get('backup_smb_host')}"
elif kind in ('aws', 's3') and _filled(values, 'backup_aws_bucket'):
status, detail = 'ok', f"S3: {values.get('backup_aws_bucket')}"
elif kind in ('smb', 'aws', 's3'):
status, detail = 'partial', f'{kind} selected but its fields are incomplete'
else:
# 'local' is the seeded default and a legitimate choice, but it means
# no copy leaves this host, so a site should make that decision rather
# than inherit it.
status, detail = 'todo', 'Backups stay on this host; no offsite copy'
areas.append(_area('backup', 'Backup destination', status, detail,
'/settings/shopdbbackups', wizard=True))
# --- branding -------------------------------------------------------
if _customised(values, 'site_logo'):
status, detail = 'ok', 'Custom logo set'
else:
status, detail = 'todo', 'Using the shipped logo and colours'
areas.append(_area('branding', 'Branding', status, detail,
'/settings/siteconfig/branding'))
# --- floor map ------------------------------------------------------
has_blueprint = _customised(values, 'map_blueprint_light')
if has_blueprint and _filled(values, 'map_width', 'map_height'):
status, detail = 'ok', f"{values.get('map_width')} x {values.get('map_height')}"
elif has_blueprint:
status, detail = 'partial', 'Blueprint uploaded but dimensions are unset'
else:
status, detail = 'todo', 'Still on the shipped placeholder blueprint'
areas.append(_area('floormap', 'Floor map', status, detail,
'/settings/siteconfig/floormap'))
# --- integrations ---------------------------------------------------
live = [name for flag, name in (
('zabbix_enabled', 'Zabbix'),
('warranty_dell_enabled', 'Dell warranty'),
('servicenow_enabled', 'ServiceNow'),
) if _truthy(values.get(flag))]
if live:
status, detail = 'ok', ', '.join(live)
else:
status, detail = 'todo', 'Zabbix, Dell warranty and ServiceNow are all off'
areas.append(_area('integrations', 'Integrations', status, detail,
'/settings/integrations'))
# --- identifiers ----------------------------------------------------
# Every identifier_* key is seeded 'true' (settings.py build_default_settings),
# so "any enabled" was true on every fresh site and this area could never
# read anything but ok. Report it as a review item instead of pretending a
# decision was made.
ident_total = [k for k in values if k.startswith('identifier_')]
ident_on = [k for k in ident_total if _truthy(values[k])]
if ident_total and len(ident_on) < len(ident_total):
status = 'ok'
detail = f'{len(ident_on)} of {len(ident_total)} identifier fields enabled'
else:
status = 'todo'
detail = 'All identifier fields are on by default; trim to what this site uses'
areas.append(_area('identifiers', 'Asset identifiers', status, detail,
'/settings/assetconfig/identifiers'))
# --- printing / QR ---------------------------------------------------
qr = [k for k in values if k.startswith('qr_target_') and _filled(values, k)]
if qr:
status, detail = 'ok', f'{len(qr)} QR target(s) set'
else:
status, detail = 'todo', 'QR codes will fall back to defaults'
areas.append(_area('printing', 'Printing and labels', status, detail,
'/settings/printing'))
# --- endpoint agent policy (only when the plugin is on) --------------
if 'endpoints' in enabled:
# The runtime default is PERMISSIVE: shared_key_allowed() in
# plugins/endpoints/api/agentauth.py reads the setting with default=True,
# so an absent row means the fleet-wide key is accepted. Treating absent
# as "off" told an admin the key was refused on exactly the sites that
# still accept it, which is the opposite of the truth about a credential
# readable by any local user.
raw = values.get('plugin.endpoints.sharedkeyallowed')
shared = True if raw in (None, '') else _truthy(raw)
if shared:
status = 'partial'
detail = ('The fleet-wide shared key is still accepted'
+ ('' if raw not in (None, '') else ' (setting unset, defaults to allowed)'))
else:
status, detail = 'ok', 'Shared key refused; per-PC credentials only'
else:
status, detail = 'na', 'Endpoints plugin is not enabled'
areas.append(_area('endpoint_policy', 'Endpoint agent policy', status, detail,
'/settings/plugins'))
# --- locations -------------------------------------------------------
# Location is a top-level export, so the import cannot fail; only the query
# can. Swallowing that reported "no locations" as a data fact AND left the
# session in a failed transaction, so the next count() below raised and the
# whole endpoint 500'd with a misleading detail already in the payload.
from shopdb.core.models import Location
try:
count = Location.query.count()
except Exception:
db.session.rollback()
count = None
if count is None:
status, detail = 'todo', 'Could not read locations'
else:
status = 'ok' if count else 'todo'
detail = f'{count} location(s)' if count else 'No locations defined yet'
areas.append(_area('locations', 'Locations', status, detail,
'/settings/locations/list'))
# --- reference data ---------------------------------------------------
vendors = Vendor.query.count()
status = 'ok' if vendors else 'todo'
detail = f'{vendors} vendor(s)' if vendors else 'No vendors seeded'
areas.append(_area('reference_data', 'Reference data', status, detail,
'/settings/catalog/vendors', wizard=True))
return areas
@setup_bp.route('/checklist', methods=['GET'])
@jwt_required()
@require_role('admin')
def checklist():
"""What is configured and what is not, with a link to each fix."""
values = _setting_values()
areas = _build_checklist(values)
counts = {'ok': 0, 'partial': 0, 'todo': 0, 'na': 0}
for area in areas:
counts[area['status']] = counts.get(area['status'], 0) + 1
return success_response({
'areas': areas,
'counts': counts,
'complete': _setup_already_complete(),
})
@setup_bp.route('/health', methods=['GET'])
@jwt_required()
@require_role('admin')
def health():
"""Post-setup reality check, including whether a restart is outstanding.
RESTART DETECTION IS THE POINT. Plugins are read from plugins.json at app
startup, so enabling one in the wizard changes the file but not the running
process: every route that plugin contributes answers 404 until the app is
restarted, with nothing on screen to say so. Comparing what the registry
says is enabled against what this process actually loaded turns that into
something the wizard can state plainly and then re-verify.
"""
# Intent comes from plugins.json re-read from disk, because that file is
# shared by every gunicorn worker while each worker's in-memory registry is
# its own. Reality is the set of plugins THIS process registered routes for
# at boot. Two things make loader.get_all_loaded() the wrong comparison:
# installing a plugin instantiates it (so it looks loaded while its
# blueprint was skipped), and a plugin enabled through another worker is
# invisible here. The boot snapshot has neither problem - a plugin enabled
# after start is missing from every worker's snapshot.
# `or` would treat "nothing enabled" as "could not read", falling back to
# this worker's stale boot view and hiding every stale-loaded plugin.
enabled = _enabled_on_disk()
if enabled is None:
enabled = _enabled_plugin_names()
booted = _booted_with_names()
pending = sorted(enabled - booted)
stale = sorted(booted - enabled)
loaded = _loaded_plugin_names()
# role.users is a dynamic relationship (AppenderQuery), so it answers to
# .count() and NOT to len(). len() raised here, the exception was swallowed,
# and the endpoint reported zero admins on a site that had one.
admin_role = Role.query.filter_by(rolename='admin').first()
admins = admin_role.users.count() if admin_role else 0
return success_response({
'database': True, # reaching here means the session works
# Which worker answered. Each one recycles independently when the
# enabled set changes, so a single clean reply only proves THAT worker
# has rolled. A caller that sees clean replies from several distinct
# pids knows the change has actually taken hold across the pool.
'worker': os.getpid(),
'users': User.query.count(),
'admins': admins,
'plugins': {
'enabled': sorted(enabled),
'loaded': sorted(loaded),
'awaitingrestart': pending,
'stillloaded': stale,
},
'restartrequired': bool(pending or stale),
})
@setup_bp.route('/complete', methods=['POST'])
@jwt_required()
@require_role('admin')

File diff suppressed because one or more lines are too long

View File

@@ -1,37 +0,0 @@
Redoc (redoc.standalone.js) - vendored third-party bundle
=========================================================
Redoc is distributed under the MIT License.
WHY THIS FILE IS A PLACEHOLDER
------------------------------
The vendored bundle opens with:
/*! For license information please see redoc.standalone.js.LICENSE.txt */
That companion file is produced by webpack and carries the extracted licence
headers of Redoc AND of every third-party package compiled into the bundle. It
ships alongside redoc.standalone.js in the upstream dist and was not copied when
this bundle was vendored, so the pointer above led nowhere.
The contents are NOT reproduced from memory here, because doing so would mean
inventing attribution for packages whose notices cannot be read back out of a
minified bundle. Do not treat this file as a substitute for the real one.
WHAT TO DO
----------
When next re-vendoring Redoc, copy BOTH files out of the upstream dist:
redoc.standalone.js
redoc.standalone.js.LICENSE.txt <- replaces this placeholder
and record the version and source URL in the same commit, so the next person
does not have to work out what is installed by reading the minified source.
The version of the currently vendored bundle could not be determined: it has no
version string, and no commit or document in this repository records it.
RELATED
-------
Swagger UI is vendored in this same directory and DOES carry its upstream
notice (swagger-ui.LICENSE.txt). That is the pattern this file should follow.

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

File diff suppressed because one or more lines are too long

View File

@@ -39,14 +39,6 @@ class PluginManager:
def __init__(self):
self.registry: Optional[PluginRegistry] = None
self.loader: Optional[PluginLoader] = None
# Plugins installed while the app was already serving, whose blueprint
# and CLI registration had to wait for a restart. The loader counts them
# as loaded (it instantiated them), so this is the only record that
# their ROUTES are not live yet.
self._deferred_registration: set = set()
# Plugins whose routes this process registered at startup. See
# _load_enabled_plugins.
self._booted_with: set = set()
self.migration_manager: Optional[PluginMigrationManager] = None
self.verifier = None
self._app: Optional[Flask] = None
@@ -108,15 +100,6 @@ class PluginManager:
def _load_enabled_plugins(self) -> None:
"""Load and register all enabled plugins."""
plugins = self.loader.load_enabled_plugins(self._app, self._db)
# What THIS process actually registered routes for, recorded at boot.
# Everything else is per-process and cannot answer "does the fleet need
# a restart": under gunicorn each worker has its own manager, its own
# loader and its own copy of the registry, so a plugin enabled through
# one worker is invisible to the other three. Comparing this snapshot
# against plugins.json re-read from disk is correct in every worker,
# because a plugin enabled after boot is absent from every worker's
# snapshot no matter which one served the enable.
self._booted_with = set(plugins.keys())
for name, plugin in plugins.items():
self._register_plugin_components(plugin)
@@ -174,7 +157,6 @@ class PluginManager:
results[name] = 'no-migrations'
continue
try:
self._release_session_locks()
ok = self.migration_manager.run_plugin_migrations(name)
results[name] = 'ok' if ok else 'failed'
except Exception as ex:
@@ -270,72 +252,8 @@ class PluginManager:
return {key for key, entry in self.get_declared_plugin_settings().items()
if entry.get('public')}
def _release_session_locks(self) -> None:
"""Close this request's transaction before any plugin DDL runs.
SELF-DEADLOCK, AND WHY IT TAKES THE WHOLE SITE DOWN. Enabling a plugin
over HTTP arrives inside a request that has already authenticated, and
@jwt_required does a SELECT on users - so the request's session is
sitting in an open transaction holding a SHARED_READ metadata lock on
users. The plugin's migrations then run through Alembic, which builds
its OWN connection; a plugin table with a foreign key to users (
endpointlocks, policyactionholds, endpointlogcollections and others)
needs an EXCLUSIVE metadata lock on it. That second connection waits on
the first, the first cannot commit because the request is blocked on the
second, and it never resolves.
It does not stop at the one request. MySQL grants metadata locks in
request order, so every later SELECT on users queues BEHIND the pending
exclusive even though those reads are compatible with the shared lock
already held. Every login on the site then hangs: observed as a total
outage from one plugin toggle.
Committing first ends the transaction and drops the lock, so the DDL
takes its exclusive lock immediately. The session stays usable - the
next statement simply begins a new transaction.
"""
if self._db is None:
return
try:
self._db.session.commit()
except Exception:
try:
self._db.session.rollback()
except Exception:
pass
def _register_plugin_components(self, plugin: BasePlugin) -> None:
"""Register plugin's blueprint, models, CLI commands, etc.
NOT POSSIBLE ON A LIVE APP, and that is fine. Flask refuses
register_blueprint once the app has served its first request, raising
AssertionError. Installing a plugin at runtime - which is exactly what
enabling one from the setup wizard or the plugins page does - therefore
used to run the plugin's migrations, update the registry, and THEN blow
up with a 500, leaving the operator with an error over a change that had
in fact been recorded.
Routes only ever take effect on restart anyway; the UI has always said
so. So on a live app, skip the parts that mutate app setup and let the
restart register them properly. The registry entry, the migrations, the
permissions and the settings - everything that belongs in the database -
still happen here.
"""
# Flask renamed this: 2.x exposed `got_first_request` publicly, 3.x
# keeps it as `_got_first_request`. Check both, because defaulting to
# False on the wrong name silently disables this guard - which is how
# the 500 survived the first attempt at this fix.
served = getattr(self._app, 'got_first_request', None)
if served is None:
served = getattr(self._app, '_got_first_request', False)
if served:
self._deferred_registration.add(plugin.meta.name)
logger.info(
"%s installed while the app is serving; its blueprint and CLI "
"commands register on the next restart (Flask does not allow "
"either after the first request).", plugin.meta.name)
return
"""Register plugin's blueprint, models, CLI commands, etc."""
# Register blueprint
blueprint = plugin.get_blueprint()
if blueprint:
@@ -356,40 +274,6 @@ class PluginManager:
for cmd in plugin.get_cli_commands():
self._app.cli.add_command(cmd)
def get_booted_with(self) -> set:
"""Plugins this process registered routes for at startup."""
return set(self._booted_with)
def enabled_on_disk(self) -> set:
"""Enabled plugins per plugins.json, re-read now.
The in-memory registry is per-worker and goes stale the moment another
worker enables something, so a restart check must go back to the file.
"""
import json
try:
path = self.registry.state_file
with open(path) as handle:
data = json.load(handle)
return {name for name, state in (data.get('plugins') or {}).items()
if state.get('enabled')}
except FileNotFoundError:
return set()
except Exception:
# Unreadable or corrupt: fall back to this worker's view rather
# than claiming nothing is enabled.
return set(self.registry.get_enabled_plugins())
def get_deferred_registrations(self) -> set:
"""Plugins enabled in this process whose routes await a restart.
get_all_loaded() is not a substitute: install_plugin instantiates the
plugin through the loader, so a plugin whose blueprint was skipped still
reports as loaded. Without this, a health check says no restart is
needed while every one of that plugin's routes answers 404.
"""
return set(self._deferred_registration)
def discover_available(self) -> List[Dict]:
"""
Get list of all available plugins (installed or not).
@@ -438,10 +322,6 @@ class PluginManager:
'config_schema': config_schema,
'provisioning_note': provisioning_note,
'default_enabled': manifest.get('default_enabled', True),
# Declared per plugin in its manifest, not held as a
# list in the wizard, so adding a plugin cannot leave
# the recommendation out of date in another file.
'recommended': bool(manifest.get('recommended', False)),
})
except Exception as e:
logger.warning(f"Error inspecting plugin {name}: {e}")
@@ -482,7 +362,6 @@ class PluginManager:
# Run migrations
if run_migrations:
self._release_session_locks()
success = self.migration_manager.run_plugin_migrations(name)
if not success:
logger.error(f"Failed to run migrations for {name}")
@@ -670,10 +549,6 @@ class PluginManager:
logger.error(f"Invalid plugin name: {name}")
return False
# Decisions below read the registry, so start from what is on disk
# rather than this worker's boot snapshot.
self.registry.refresh()
# Auto-install if available but not installed (UX: enabling should just work)
if not self.registry.is_installed(name):
logger.info(f"Plugin {name} not installed, installing first")
@@ -726,7 +601,6 @@ class PluginManager:
def disable_plugin(self, name: str) -> bool:
"""Disable an enabled plugin."""
self.registry.refresh()
if not self.registry.is_enabled(name):
logger.info(f"Plugin {name} is already disabled")
return True

View File

@@ -44,10 +44,6 @@ import shopdb.utils.mysql_charset # noqa: F401,E402
logger = logging.getLogger('alembic.env.plugin')
# Seconds a migration will wait for a metadata lock before failing. Long enough
# to ride out a normal request, far short of MySQL's one-year default.
MIGRATION_LOCK_WAIT_SECONDS = 20
# Explicit table-ownership map. Adding tables to a plugin requires updating
# this dict so the per-plugin migration knows which tables to include.
PLUGIN_TABLE_OWNERS: dict[str, Iterable[str]] = {
@@ -254,25 +250,7 @@ def run_migrations():
# config.get_section returns an empty dict and engine_from_config would
# find no sqlalchemy.url. db_url is already resolved above.
from sqlalchemy import create_engine
# Bound how long a migration waits for a metadata lock.
#
# THIS connection is the one that blocks. A plugin table with a foreign
# key to a core table (users, assets) needs an EXCLUSIVE metadata lock
# on it, and MySQL's lock_wait_timeout defaults to a YEAR, so the DDL
# waits behind any open transaction effectively forever. Worse, MySQL
# grants metadata locks in request order: every later reader queues
# behind the pending exclusive, so one stuck migration stops the whole
# site answering. Failing after a bounded wait turns a site-wide outage
# into one failed enable the operator can retry.
#
# MySQL only - SQLite has no such statement and would reject it.
connect_args = {}
if db_url.startswith('mysql'):
connect_args['init_command'] = (
f'SET SESSION lock_wait_timeout = {MIGRATION_LOCK_WAIT_SECONDS}')
connectable = create_engine(db_url, poolclass=pool.NullPool,
connect_args=connect_args)
connectable = create_engine(db_url, poolclass=pool.NullPool)
with connectable.connect() as connection:
context.configure(
connection=connection,

View File

@@ -75,21 +75,6 @@ class PluginMigrationManager:
config = Config()
config.set_main_option('script_location', str(migrations_dir))
config.set_main_option('sqlalchemy.url', self.database_url)
# The lock-wait cap lives with the engine in alembic_template, which
# builds its own; see MIGRATION_LOCK_WAIT_SECONDS there.
_unused = """Bound how long this migration will wait for a metadata lock.
#
# THIS connection is the one that blocks. A plugin table with a
# foreign key to a core table (users, assets) needs an EXCLUSIVE
# metadata lock on it, and MySQL's lock_wait_timeout defaults to a
# YEAR - so a DDL behind any long-running transaction waits
# effectively forever. Worse, MySQL grants metadata locks in request
# order, so every later reader queues behind the pending exclusive
# and the whole site stops answering, which is what took this
# instance down twice. Failing the migration after a bounded wait
# turns a site-wide outage into one failed enable the operator can
# retry.
""" # noqa: F841
# Use plugin-specific version table
config.set_main_option(

View File

@@ -1,179 +0,0 @@
"""Recycle a worker when the set of enabled plugins changes underneath it.
WHY THIS EXISTS. Enabling a plugin cannot make its routes live in a running
process: Flask refuses register_blueprint once the app has served a request, and
even if it did not, each gunicorn worker holds its own app object, so registering
in the worker that served the toggle would leave the others answering 404 for the
same URL. The operator therefore had to restart the app by hand, and the UI had
to tell them so.
The fix is the primitive gunicorn already uses for --reload: a worker does not
ask anyone to reload it, it EXITS, and the supervisor replaces it. Applied to a
single file that is cheap to stat, that is production-grade rather than a dev
convenience.
WHY NOT SIGNAL THE MASTER. Sending SIGHUP to os.getppid() assumes gunicorn, a
POSIX host, and that the parent is in fact the master; under the dev server that
parent is the operator's shell. Exiting instead needs none of those assumptions:
whatever started the process decides what replaces it. It is also ROLLING - each
worker recycles when it next serves a request, so the others keep serving - where
signalling the master drains every worker at once.
SCOPE. This runs under gunicorn only (see SUPERVISED). The Windows/IIS path
waitress behind HttpPlatformHandler is single-process and has never been
exercised here, so it keeps the manual restart and the UI keeps saying so.
WHAT IT FIXES BEYOND THE RESTART. Every worker converges on plugins.json within
one request of a change, so the per-worker registry skew that caused enables to
be lost and dependency checks to be refused stops being a standing condition.
"""
import hashlib
import logging
import os
import random
import signal
import threading
import time
logger = logging.getLogger(__name__)
# Do not recycle a worker younger than this. A worker that exits immediately on
# boot cannot serve anything, and a fingerprint that is somehow permanently
# "changed" would otherwise spin the supervisor respawning workers forever.
MIN_UPTIME_SECONDS = 15
# Ignore a change newer than this. plugins.json is written under an exclusive
# lock and replaced atomically, but a burst of enables (the wizard's "enable all
# recommended") rewrites it once per plugin. Waiting lets the burst settle so a
# worker recycles once at the end rather than after every plugin.
DEBOUNCE_SECONDS = 3
# Per-worker spread, on top of the debounce. Without it every worker that
# serves a request in the same second clears the same threshold and they all
# exit together, so the pool is briefly empty and the restart is not rolling at
# all - it is a synchronised outage. This is the same reason gunicorn pairs
# --max-requests with --max-requests-jitter. Chosen once per process.
JITTER_CEILING_SECONDS = 10
# Servers whose supervisor will bring the process back AND where the exit path
# below has been exercised. Anything else - the Werkzeug dev server above all -
# must never be exited from under the developer.
#
# WAITRESS IS DELIBERATELY ABSENT. The IIS path (HttpPlatformHandler running
# waitress) is single-process, so there is no worker to recycle: the process
# itself would have to exit and be relaunched. That is plausible but UNPROVEN -
# it has never run on Windows here - and HttpPlatformHandler stops relaunching
# after rapidFailsPerMinute (default 10), which would take the site down rather
# than reload it. Until it is tested on the Windows VM, IIS keeps the honest
# behaviour it always had: the operator restarts, and the UI says so. Claiming
# otherwise there would be worse than the manual step it replaces.
SUPERVISED = ('gunicorn/',)
def _fingerprint(path):
"""Content hash of plugins.json, or None when it is absent or unreadable.
Content, not mtime: an atomic replace gives the new file its own mtime even
when the contents are identical (every _save rewrites the whole file), so
mtime alone would recycle workers over writes that changed nothing.
"""
try:
with open(path, 'rb') as handle:
return hashlib.sha256(handle.read()).hexdigest()
except FileNotFoundError:
return None
except OSError:
# Unreadable is not "changed". Returning None on a transient error and
# a hash the next time would recycle the worker for no reason.
return 'unreadable'
def _supervised(environ):
"""True when something will restart this process after it exits."""
server = str(environ.get('SERVER_SOFTWARE') or '')
return server.startswith(SUPERVISED)
def _exit_gracefully():
"""Leave, letting in-flight requests finish, and let the supervisor return.
SIGTERM to our own pid is exactly what the gunicorn master sends for a
graceful worker shutdown: the worker stops accepting, drains what it is
holding, exits, and is respawned. Only reachable under gunicorn, because
_supervised() gates on it.
Runs on a short timer rather than inline so the response that triggered it
is already on its way out.
"""
def _leave():
# Not load-bearing under gunicorn, which finishes in-flight requests
# during its graceful drain regardless. It only buys a moment for the
# triggering response to reach the wire before anything changes.
time.sleep(0.5)
try:
# SIGTERM to our own pid is precisely what the gunicorn master
# sends for a graceful worker shutdown: stop accepting, drain what
# we are holding, exit, and be respawned. _supervised() has already
# established that we are under gunicorn, so this branch is the
# only one that can be reached.
os.kill(os.getpid(), signal.SIGTERM)
except Exception:
# Last resort only. sys.exit() would be wrong here: raising
# SystemExit in a non-main thread ends the THREAD, not the process,
# and the process would carry on believing it had recycled.
os._exit(0)
threading.Thread(target=_leave, daemon=True, name='plugin-recycle').start()
def init_recycle(app):
"""Watch plugins.json and recycle this worker when it changes.
Registered as an after_request hook: the check is a stat and a hash of a
file measured in kilobytes, and it only runs once the response is built.
"""
if app.config.get('TESTING'):
return
state_file = os.path.join(app.instance_path, 'plugins.json')
booted_fingerprint = _fingerprint(state_file)
booted_at = time.monotonic()
settle_after = DEBOUNCE_SECONDS + random.uniform(0, JITTER_CEILING_SECONDS)
# One worker, one exit. Without this two concurrent requests can both see
# the change and both start the timer.
leaving = threading.Event()
@app.after_request
def _recycle_if_plugins_changed(response):
if leaving.is_set():
return response
if time.monotonic() - booted_at < MIN_UPTIME_SECONDS:
return response
try:
from flask import request
if not _supervised(request.environ):
return response
except Exception:
return response
current = _fingerprint(state_file)
if current == booted_fingerprint or current == 'unreadable':
return response
try:
age = time.time() - os.path.getmtime(state_file)
except OSError:
return response
if age < settle_after:
return response
if leaving.is_set():
return response
leaving.set()
logger.warning(
'Enabled plugins changed on disk; recycling this worker so the '
'change takes effect. In-flight requests are finished first and '
'the supervisor starts a replacement.')
_exit_gracefully()
return response

View File

@@ -1,14 +1,11 @@
"""Plugin registry for tracking installed and enabled plugins."""
import json
import logging
from pathlib import Path
from typing import Dict, List, Optional
from dataclasses import dataclass, field, asdict
from datetime import datetime, timezone
logger = logging.getLogger(__name__)
@dataclass
class PluginState:
@@ -34,35 +31,16 @@ class PluginRegistry:
self._load()
def _load(self) -> None:
"""Replace this instance's view with what is on disk.
REPLACE, not merge. Assigning into the existing dict left entries that
the file no longer has, so a plugin uninstalled through another worker
stayed 'installed' here forever: enable_plugin would then skip
install_plugin (and its migrations) and _save would write the ghost back
out, resurrecting a plugin whose tables were never created. refresh()
exists precisely to end stale reads, and it could not while this merged.
"""
loaded = {}
"""Load registry from file."""
if self.state_file.exists():
try:
with open(self.state_file, 'r') as f:
data = json.load(f)
except (json.JSONDecodeError, OSError):
data = None
for name, state_data in ((data or {}).get('plugins') or {}).items():
try:
loaded[name] = PluginState(**state_data)
except TypeError:
# A key written by a newer build. Skip THIS entry and keep
# the rest: discarding the whole registry made every
# subsequent enable a silent no-op, because enable() only
# acts on names it can see and still answered 200.
logger.warning(
'plugins.json entry %r has unexpected fields; ignoring '
'it in this process', name)
continue
self._plugins = loaded
for name, state_data in data.get('plugins', {}).items():
self._plugins[name] = PluginState(**state_data)
except (json.JSONDecodeError, TypeError):
# Corrupted file, start fresh
self._plugins = {}
self._migrate_renamed_plugins()
def _migrate_renamed_plugins(self) -> None:
@@ -80,93 +58,10 @@ class PluginRegistry:
state = self._plugins.pop('equipment')
state.name = 'machines'
self._plugins['machines'] = state
self._save(only={'machines'}, remove={'equipment'})
self._save()
def _save(self, remove: set = None, only: set = None) -> None:
"""Persist this registry, merging rather than overwriting.
LOST UPDATES, AND WHY A PLAIN WRITE IS WRONG. Every gunicorn worker
builds its own PluginRegistry and loads plugins.json once at boot, so
each holds a snapshot that goes stale the moment another worker changes
anything. Rewriting the whole file from that snapshot therefore erases
whatever the other workers recorded: enabling seven plugins across four
workers persisted three, and the API answered 200 for all seven. Silent
data loss, with no error anywhere to hint at it.
So take an exclusive lock, re-read what is on disk, apply only the
entries this instance actually holds on top of it, and replace the file
atomically. Concurrent writers then serialise instead of clobbering, and
a crash mid-write cannot leave a half-written registry.
"""
import os
import tempfile
self.state_file.parent.mkdir(parents=True, exist_ok=True)
lock_path = self.state_file.with_suffix('.lock')
try:
import fcntl
except ImportError: # non-POSIX: fall back to the plain write
fcntl = None
lock_handle = None
try:
if fcntl is not None:
lock_handle = open(lock_path, 'w')
fcntl.flock(lock_handle, fcntl.LOCK_EX)
merged = {}
if self.state_file.exists():
try:
with open(self.state_file) as handle:
merged = (json.load(handle) or {}).get('plugins', {}) or {}
except (json.JSONDecodeError, TypeError, OSError):
merged = {}
# Overlay ONLY the entries this call is changing. Writing every
# entry this instance holds is what loses other workers' updates:
# a snapshot taken at boot still says X is disabled, so saving an
# unrelated change to Y drags X back down with it. `only` is the
# mutation actually being persisted.
for name, state in self._plugins.items():
if only is not None and name not in only:
continue
merged[name] = asdict(state)
# A delete cannot be expressed by merging - the entry is simply
# absent from self._plugins, and the on-disk copy would survive and
# read as a resurrection. Name it instead.
for name in (remove or set()):
merged.pop(name, None)
# Adopt the merged view so this instance stops carrying a stale
# snapshot that a later save would write back out.
rebuilt = {}
for name, data in merged.items():
try:
rebuilt[name] = PluginState(**data)
except TypeError:
# An unknown key from a newer build must not turn every
# enable into a 500; keep the row on disk, skip it here.
continue
self._plugins = rebuilt
handle = tempfile.NamedTemporaryFile(
'w', dir=str(self.state_file.parent), delete=False)
try:
json.dump({'plugins': merged}, handle, indent=2)
handle.flush()
os.fsync(handle.fileno())
finally:
handle.close()
os.replace(handle.name, self.state_file)
return
finally:
if lock_handle is not None:
try:
fcntl.flock(lock_handle, fcntl.LOCK_UN)
lock_handle.close()
except Exception:
pass
def _save_unmerged(self) -> None:
"""Original whole-file write. Kept for callers that own the file."""
def _save(self) -> None:
"""Save registry to file."""
self.state_file.parent.mkdir(parents=True, exist_ok=True)
with open(self.state_file, 'w') as f:
json.dump({
@@ -185,27 +80,15 @@ class PluginRegistry:
installed_at=datetime.now(timezone.utc).replace(tzinfo=None).isoformat(),
enabled=enabled
)
self._load() # another worker may have changed the file
self._plugins[name] = state
self._save(only={name})
self._save()
return state
def unregister(self, name: str) -> None:
"""Remove plugin from registry."""
self._load() # another worker may have registered it since boot
self._plugins.pop(name, None)
self._save(remove={name})
def refresh(self) -> None:
"""Re-read plugins.json before making a decision from it.
Every worker loads this file once at boot, so any read used to DECIDE
something - is it installed, is a dependency enabled - is answering from
a snapshot that another worker may have moved on from. That is how
enabling endpoints returned 409 "computers is not enabled" moments after
computers had in fact been enabled through a different worker.
"""
self._load()
if name in self._plugins:
del self._plugins[name]
self._save()
def get(self, name: str) -> Optional[PluginState]:
"""Get plugin state."""
@@ -221,23 +104,16 @@ class PluginRegistry:
return state.enabled if state else False
def enable(self, name: str) -> None:
"""Enable a plugin.
Re-reads first: this worker's snapshot dates from boot, so a plugin
another worker installed since is absent here and the old
`if name in self._plugins` guard silently did nothing.
"""
self._load()
"""Enable a plugin."""
if name in self._plugins:
self._plugins[name].enabled = True
self._save(only={name})
self._save()
def disable(self, name: str) -> None:
"""Disable a plugin. Re-reads first, for the same reason as enable."""
self._load()
"""Disable a plugin."""
if name in self._plugins:
self._plugins[name].enabled = False
self._save(only={name})
self._save()
def get_enabled_plugins(self) -> List[str]:
"""Get list of enabled plugin names."""
@@ -248,11 +124,10 @@ class PluginRegistry:
def add_migration(self, name: str, revision: str) -> None:
"""Record that a migration was applied."""
self._load()
if name in self._plugins:
if revision not in self._plugins[name].migrations_applied:
self._plugins[name].migrations_applied.append(revision)
self._save(only={name})
self._save()
def get_all(self) -> Dict[str, PluginState]:
"""Get all registered plugins."""
@@ -260,7 +135,6 @@ class PluginRegistry:
def update_config(self, name: str, config: Dict) -> None:
"""Update plugin configuration."""
self._load()
if name in self._plugins:
self._plugins[name].config.update(config)
self._save(only={name})
self._save()

View File

@@ -1,179 +0,0 @@
"""The API documentation page at /api/docs.
What is being proved is the thing that would hurt on the production box:
nothing the page loads comes from the internet. Production is air-gapped, so a
CDN reference, a webfont, or a source map pointing at a public host is not a
slow page - it is a page that never renders and a request that hangs.
The Swagger "try it" page this file used to cover was removed: it shipped
1.77 MB of vendored assets to every site in order to provide an Execute button
that only development configs ever enabled, and an agent now has the MCP
server's shopdb_call while a human has the spec's own worked examples.
These build their own Flask app around the docs blueprint rather than using the
session app from conftest. The blueprint touches no database, and a second
create_app() would rebind the SQLAlchemy extension and break whatever module
runs next (see the note on the gated_app fixture in test_public_read_surface).
"""
import os
import re
import pytest
from flask import Flask
from shopdb.core.api.docs import docs_bp
_STATICDOCS = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))),
'shopdb', 'core', 'api', 'staticdocs')
# The asset a browser actually fetches from this directory. It carried no
# air-gap test at all while this file parametrised over the two Swagger files -
# and the one real leak ever found here was Redoc reaching for cdn.redoc.ly.
_VENDORED = ('redoc.standalone.js',)
def _docs_app():
"""A bare app carrying only the docs blueprint, mounted where the real app
mounts it (register_blueprints uses /api/<name>)."""
application = Flask(__name__)
application.register_blueprint(docs_bp, url_prefix='/api/docs')
return application
@pytest.fixture
def client():
return _docs_app().test_client()
@pytest.mark.parametrize('filename', _VENDORED)
def test_a_vendored_asset_fetches_nothing_from_the_internet(filename):
"""The assets themselves, not just the page.
Scoped to what a BROWSER would actually request. The Redoc bundle
carries around a hundred http(s) strings that are prose - RFC and
json-schema citations, licence URLs, the example.com host in the spec's own
documentation - and none of them is fetched, so a blanket "no http in this
file" assertion would be a lie that fails on the next upstream bump. What
would really produce a request is a CDN or webfont reference, a stylesheet
url() pointing off-box, or a source map: hence these.
"""
body = open(os.path.join(_STATICDOCS, filename),
encoding='utf-8', errors='replace').read()
# NOT a host allowlist. The first version of this named five CDNs, so an
# upstream bump to esm.sh or cdn.skypack.dev walked straight through it, and
# the one real leak in this directory was cdn.redoc.ly - which was on no
# list. Match the SHAPES a browser fetches instead, whatever the host:
# a stylesheet import, a css url(), a script/link/img source.
fetching = [
(r'@import\s+(?:url\()?[\'"]?(?:https?:)?//', 'an @import from another origin'),
(r'url\(\s*[\'"]?(?:https?:)?//', 'a css url() on another origin'),
(r'(?:src|href)\s*=\s*[\'"](?:https?:)?//', 'an element loaded from another origin'),
(r'@font-face', 'a webfont, which is a fetch unless it is a data: URI'),
]
for pattern, what in fetching:
for match in re.finditer(pattern, body):
window = body[max(0, match.start() - 60):match.start() + 120]
# data: URIs are inline, not fetched, and this CSS is full of them.
if 'data:' in window[:len(window) // 2 + 40]:
continue
raise AssertionError(
'%s carries %s: ...%s...' % (filename, what, window.strip()[:140]))
# A map reference is a request too. The upstream .map files are not
# vendored (they are tens of megabytes and only devtools reads them), so
# any surviving pointer is a guaranteed 404 on a box that cannot reach the
# internet to try elsewhere.
# The TRAILING pragma only. Redoc's own code contains the same string three
# times - a data: URI it assembles and a worker blob - and those are source,
# not a directive. A bare "not in body" assertion fails on correct code, and
# trying to strip every occurrence destroyed half the bundle: the file is
# minified onto long lines, so a line-anchored pattern eats the rest of it.
trailing = body.rstrip().rsplit('\n', 1)[-1]
assert not trailing.startswith('//# sourceMappingURL'), (
'%s ends with a source-map pragma, and the .map is not vendored: every '
'devtools open is then a 404 on a box with no internet' % filename)
# Stylesheet resources: a vendored stylesheet must inline its images, and
# anything else would be an off-box fetch on first paint. CSS only - in the
# minified bundle "url(" is a function call, not a resource.
if filename.endswith('.css'):
for match in re.findall(r'url\(\s*["\']?([^"\')]+)', body):
assert match.startswith('data:'), (
'%s loads %s rather than an inline data: URI'
% (filename, match))
# ---------------------------------------------------------------------------
# The spec has to say where this instance actually lives
# ---------------------------------------------------------------------------
def test_the_spec_declares_the_mount_it_is_served_from(client):
"""Production runs under /shopdb, and the generated file cannot know that.
gen_openapi.py writes servers: [{"url": "/"}] - correct for a root install
and wrong for every subpath one - and wsgi.py only moves MOUNT_PATH into
SCRIPT_NAME at runtime. Served as written, both renderers built
https://host/api/... for an app mounted at /shopdb: a URL an operator copied
out of the docs page and got a 404 from, and the address Swagger's try-it
fired at.
"""
import json as _json
at_root = _json.loads(
client.get('/api/docs/openapi.json').get_data(as_text=True))
assert at_root['servers'] == [{'url': '/', 'description': 'this instance'}]
mounted = _json.loads(client.get(
'/api/docs/openapi.json',
environ_overrides={'SCRIPT_NAME': '/shopdb'}).get_data(as_text=True))
assert mounted['servers'] == [
{'url': '/shopdb', 'description': 'this instance'}]
# The paths themselves stay mount-free: the server prefix is what varies,
# and duplicating it into every path would double it under a subpath.
assert '/api/apitokens' in mounted['paths']
assert '/shopdb/api/apitokens' not in mounted['paths']
# ---------------------------------------------------------------------------
# The read-only page follows the OS theme
# ---------------------------------------------------------------------------
def test_the_redoc_page_follows_the_system_theme(client):
"""The app has a dark theme; /api/docs was the one page that stayed white.
Redoc ignores prefers-color-scheme and the <redoc spec-url> element takes no
theme, so the page has to go through Redoc.init - which is also why this
test pins the mechanism and not just the CSS. A future edit that puts the
element back would lose dark mode without changing a single colour.
"""
page = client.get('/api/docs/').get_data(as_text=True)
assert 'Redoc.init' in page
assert '<redoc spec-url' not in page, 'the element takes no theme'
# The page now follows the app's own stored choice first and only falls
# back to the OS, so assert the whole chain rather than the media query
# alone: the storage key, the attribute both the CSS and the Redoc theme
# key off, and the fallback.
assert '"shopdb-theme"' in page
assert 'data-theme' in page
assert 'prefers-color-scheme: dark' in page
assert 'var DARK=' in page and 'var LIGHT=' in page
# Re-render on a live scheme change, with the fallback for the older
# MediaQueryList that a kiosk browser may still have.
assert 'addEventListener("change",draw)' in page
assert 'q.addListener(draw)' in page
def test_the_docs_page_still_loads_nothing_from_another_origin(client):
"""Theming is the kind of change that quietly introduces a font or a CDN."""
import re as _re
page = client.get('/api/docs/').get_data(as_text=True)
assert not _re.findall(r'(?:src|href)\s*=\s*[\'"](?:https?:)?//', page)

View File

@@ -0,0 +1,283 @@
"""The Swagger UI "try it" page at /api/docs/try.
Two things are being proved, and they are the two that would hurt on the
production box:
1. It is OFF unless a site turns it on, and off means the URL does not exist.
The page fires real requests at this server with the browser's own session,
so a disabled deployment must not serve it, advertise it, or confirm it is
a thing by answering anything other than 404.
2. Nothing it loads comes from the internet. Production is air-gapped: a CDN
reference, a webfont, or a source map pointing at a public host is not a
slow page, it is a page that never renders and a request that hangs.
These build their own Flask app around the docs blueprint rather than using the
session app from conftest. The blueprint touches no database, and a second
create_app() would rebind the SQLAlchemy extension and break whatever module
runs next (see the note on the gated_app fixture in test_public_read_surface).
It also lets one test see the flag on and the next see it off, which is the
whole behaviour under test.
"""
import os
import re
import pytest
from flask import Flask
from shopdb.core.api.docs import docs_bp
# Everything the try-it page is allowed to load, and the files behind them.
_CSS_PATH = '/api/docs/swagger-ui.css'
_JS_PATH = '/api/docs/swagger-ui-bundle.min.js'
_STATICDOCS = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))),
'shopdb', 'core', 'api', 'staticdocs')
_VENDORED = ('swagger-ui.css', 'swagger-ui-bundle.min.js')
def _docs_app(tryit_enabled):
"""A bare app carrying only the docs blueprint, mounted where the real app
mounts it (register_blueprints uses /api/<name>)."""
application = Flask(__name__)
application.config['API_DOCS_TRYIT_ENABLED'] = tryit_enabled
application.register_blueprint(docs_bp, url_prefix='/api/docs')
return application
@pytest.fixture
def enabled_client():
return _docs_app(True).test_client()
@pytest.fixture
def disabled_client():
return _docs_app(False).test_client()
def test_enabled_page_renders_swagger_over_the_published_spec(enabled_client):
response = enabled_client.get('/api/docs/try')
assert response.status_code == 200
page = response.get_data(as_text=True)
assert 'SwaggerUIBundle(' in page
assert 'id="swagger-ui"' in page
# One spec, two renderers: the try-it page must read the same document the
# Redoc page serves, not a second copy that could drift from it.
assert '/api/docs/openapi.json' in page
def test_the_page_references_no_other_origin_at_all(enabled_client):
"""Absolute AND protocol-relative.
The first version of this scanned for https?://, so
<script src="//cdn.jsdelivr.net/npm/x.js"> - the form a copy-paste from any
CDN's own docs produces - matched nothing and passed.
"""
page = enabled_client.get('/api/docs/try').get_data(as_text=True)
offenders = re.findall(r'(?:src|href)\s*=\s*[\'"]((?:https?:)?//[^\'"]+)', page)
assert not offenders, 'the try-it page loads from another origin: %s' % offenders
def test_the_redoc_page_survives_try_it_being_unmounted(app):
"""The banner linking to the try-it page is built with url_for, and url_for
raises BuildError when those routes are not registered - which is every
production box. An unconditional link therefore 500s /api/docs/ in exactly
the deployment nobody tests locally.
"""
# The testing config is the unmounted case, and it is the one available
# here: ProductionConfig refuses to build without a real SECRET_KEY, which
# is correct of it and makes it useless as a fixture.
page = app.test_client().get('/api/docs/')
assert page.status_code == 200, page.get_data(as_text=True)[:200]
body = page.get_data(as_text=True)
assert 'redoc' in body.lower()
assert '/api/docs/try' not in body
def test_enabled_page_loads_only_the_vendored_assets(enabled_client):
page = enabled_client.get('/api/docs/try').get_data(as_text=True)
assert 'href="%s"' % _CSS_PATH in page
assert 'src="%s"' % _JS_PATH in page
for path, expected_type in ((_CSS_PATH, 'text/css'),
(_JS_PATH, 'application/javascript')):
asset = enabled_client.get(path)
assert asset.status_code == 200, path
assert expected_type in asset.headers['Content-Type']
assert asset.headers.get('Content-Length', '0') != '0'
def test_the_page_names_no_external_host(enabled_client):
"""Every URL on the page has to be same-origin. On an air-gapped box an
absolute http(s) URL is a request that hangs until it times out."""
page = enabled_client.get('/api/docs/try').get_data(as_text=True)
external = re.findall(r'https?://[^\s"\'<>)]+', page)
assert not external, 'try-it page references external hosts: %s' % external
def test_the_spec_validator_is_turned_off(enabled_client):
"""Swagger UI's one outbound default: unset, it posts the spec to a public
validator to draw a badge. That is a hung request here, and it would be an
upload of the internal API surface anywhere it worked."""
page = enabled_client.get('/api/docs/try').get_data(as_text=True)
assert re.search(r'validatorUrl:\s*null', page), (
'validatorUrl is not disabled; Swagger UI will call the public '
'validator')
def test_disabled_deployment_has_no_such_route(disabled_client):
"""404, not a rendered page explaining that it is disabled. The route is
never bound, so the URL confirms nothing."""
for path in ('/api/docs/try', '/api/docs/try/', _CSS_PATH, _JS_PATH):
assert disabled_client.get(path).status_code == 404, path
def test_the_redoc_page_advertises_try_it_only_when_it_exists(
enabled_client, disabled_client):
enabled_page = enabled_client.get('/api/docs/').get_data(as_text=True)
assert '/api/docs/try' in enabled_page
disabled_page = disabled_client.get('/api/docs/').get_data(as_text=True)
assert '/api/docs/try' not in disabled_page
# The word too, not just the href: a note about a view that 404s is worse
# than no note.
assert 'try-it' not in disabled_page
def test_every_url_follows_the_mount_path(enabled_client):
"""Prod runs under /shopdb. url_for has to carry the prefix onto the spec
and both assets, or the page loads nothing and can validate nothing."""
page = enabled_client.get(
'/api/docs/try',
environ_overrides={'SCRIPT_NAME': '/shopdb'}).get_data(as_text=True)
assert '/shopdb/api/docs/openapi.json' in page
assert '/shopdb%s' % _CSS_PATH in page
assert '/shopdb%s' % _JS_PATH in page
# No unprefixed twin left behind by a hardcoded path.
assert 'href="%s"' % _CSS_PATH not in page
assert 'src="%s"' % _JS_PATH not in page
redoc_page = enabled_client.get(
'/api/docs/',
environ_overrides={'SCRIPT_NAME': '/shopdb'}).get_data(as_text=True)
assert '/shopdb/api/docs/try' in redoc_page
@pytest.mark.parametrize('filename', _VENDORED)
def test_a_vendored_asset_fetches_nothing_from_the_internet(filename):
"""The assets themselves, not just the page.
Scoped to what a BROWSER would actually request. The swagger-ui bundle
carries around a hundred http(s) strings that are prose - RFC and
json-schema citations, licence URLs, the example.com host in the spec's own
documentation - and none of them is fetched, so a blanket "no http in this
file" assertion would be a lie that fails on the next upstream bump. What
would really produce a request is a CDN or webfont reference, a stylesheet
url() pointing off-box, or a source map: hence these.
"""
body = open(os.path.join(_STATICDOCS, filename),
encoding='utf-8', errors='replace').read()
# NOT a host allowlist. The first version of this named five CDNs, so an
# upstream bump to esm.sh or cdn.skypack.dev walked straight through it, and
# the one real leak in this directory was cdn.redoc.ly - which was on no
# list. Match the SHAPES a browser fetches instead, whatever the host:
# a stylesheet import, a css url(), a script/link/img source.
fetching = [
(r'@import\s+(?:url\()?[\'"]?(?:https?:)?//', 'an @import from another origin'),
(r'url\(\s*[\'"]?(?:https?:)?//', 'a css url() on another origin'),
(r'(?:src|href)\s*=\s*[\'"](?:https?:)?//', 'an element loaded from another origin'),
(r'@font-face', 'a webfont, which is a fetch unless it is a data: URI'),
]
for pattern, what in fetching:
for match in re.finditer(pattern, body):
window = body[max(0, match.start() - 60):match.start() + 120]
# data: URIs are inline, not fetched, and this CSS is full of them.
if 'data:' in window[:len(window) // 2 + 40]:
continue
raise AssertionError(
'%s carries %s: ...%s...' % (filename, what, window.strip()[:140]))
# A map reference is a request too. The upstream .map files are not
# vendored (they are tens of megabytes and only devtools reads them), so
# any surviving pointer is a guaranteed 404 on a box that cannot reach the
# internet to try elsewhere.
assert 'sourceMappingURL' not in body, (
'%s still points at a source map that is not vendored' % filename)
# Stylesheet resources: swagger-ui inlines its images as data: URIs, and
# anything else would be an off-box fetch on first paint. CSS only - in the
# minified bundle "url(" is a function call, not a resource.
if filename.endswith('.css'):
for match in re.findall(r'url\(\s*["\']?([^"\')]+)', body):
assert match.startswith('data:'), (
'%s loads %s rather than an inline data: URI'
% (filename, match))
# ---------------------------------------------------------------------------
# The spec has to say where this instance actually lives
# ---------------------------------------------------------------------------
def test_the_spec_declares_the_mount_it_is_served_from(enabled_client):
"""Production runs under /shopdb, and the generated file cannot know that.
gen_openapi.py writes servers: [{"url": "/"}] - correct for a root install
and wrong for every subpath one - and wsgi.py only moves MOUNT_PATH into
SCRIPT_NAME at runtime. Served as written, both renderers built
https://host/api/... for an app mounted at /shopdb: a URL an operator copied
out of the docs page and got a 404 from, and the address Swagger's try-it
fired at.
"""
import json as _json
at_root = _json.loads(
enabled_client.get('/api/docs/openapi.json').get_data(as_text=True))
assert at_root['servers'] == [{'url': '/', 'description': 'this instance'}]
mounted = _json.loads(enabled_client.get(
'/api/docs/openapi.json',
environ_overrides={'SCRIPT_NAME': '/shopdb'}).get_data(as_text=True))
assert mounted['servers'] == [
{'url': '/shopdb', 'description': 'this instance'}]
# The paths themselves stay mount-free: the server prefix is what varies,
# and duplicating it into every path would double it under a subpath.
assert '/api/apitokens' in mounted['paths']
assert '/shopdb/api/apitokens' not in mounted['paths']
# ---------------------------------------------------------------------------
# The read-only page follows the OS theme
# ---------------------------------------------------------------------------
def test_the_redoc_page_follows_the_system_theme(client):
"""The app has a dark theme; /api/docs was the one page that stayed white.
Redoc ignores prefers-color-scheme and the <redoc spec-url> element takes no
theme, so the page has to go through Redoc.init - which is also why this
test pins the mechanism and not just the CSS. A future edit that puts the
element back would lose dark mode without changing a single colour.
"""
page = client.get('/api/docs/').get_data(as_text=True)
assert 'Redoc.init' in page
assert '<redoc spec-url' not in page, 'the element takes no theme'
assert '@media (prefers-color-scheme: dark)' in page
assert 'var DARK=' in page and 'var LIGHT=' in page
# Re-render on a live scheme change, with the fallback for the older
# MediaQueryList that a kiosk browser may still have.
assert 'addEventListener("change",draw)' in page
assert 'q.addListener(draw)' in page
def test_the_docs_page_still_loads_nothing_from_another_origin(client):
"""Theming is the kind of change that quietly introduces a font or a CDN."""
import re as _re
page = client.get('/api/docs/').get_data(as_text=True)
assert not _re.findall(r'(?:src|href)\s*=\s*[\'"](?:https?:)?//', page)

View File

@@ -59,9 +59,6 @@ PUBLIC_ENDPOINTS = {
# API documentation: shapes, not data. The docs page loads its own script.
'docs.docs_index', 'docs.openapi_spec', 'docs.redoc_js', 'docs.llms_txt',
# The operation index is the same shape-not-data as the spec it indexes:
# method, path, tag and summary for routes the spec already publishes.
'docs.operations_index',
# Collector ingest, authenticated by the per-plugin collector key.
'collector.collector_schemas', 'collector.collector_status',

View File

@@ -26,6 +26,56 @@ import pytest
REPO_ROOT = Path(__file__).resolve().parent.parent
GATE = REPO_ROOT / 'scripts' / 'check-naming-and-style.sh'
def _find_bash():
"""Locate a bash that can actually run a POSIX script.
shutil.which('bash') is not enough on Windows. The WSL shim at
System32\\bash.exe is on PATH by default, satisfies which(), and then fails
every invocation with
WSL (...) ERROR: CreateProcessCommon:818: execvpe(/bin/bash) failed:
No such file or directory
when no distribution is installed - because the path handed to it is a
Windows path that means nothing inside the WSL namespace. That is the same
trap CONTRIBUTING.md documents for running the gate by hand, and it is why
these 13 tests were recorded as "environmental failures" rather than being
fixed: they were reporting the shim, not the gate.
Git Bash is preferred explicitly, then any candidate is proven by running a
trivial script through it before being trusted.
"""
candidates = []
git_bash = shutil.which('git')
if git_bash:
# <git>/cmd/git.exe -> <git>/bin/bash.exe
candidates.append(str(Path(git_bash).resolve().parent.parent / 'bin'
/ 'bash.exe'))
candidates.extend([
r'C:\Program Files\Git\bin\bash.exe',
r'C:\Program Files (x86)\Git\bin\bash.exe',
])
found = shutil.which('bash')
if found:
candidates.append(found)
for candidate in candidates:
if not candidate or not Path(candidate).exists():
continue
try:
probe = subprocess.run([candidate, '-c', 'exit 0'],
capture_output=True, text=True, timeout=30)
except (OSError, subprocess.SubprocessError):
continue
if probe.returncode == 0 and 'execvpe' not in (probe.stderr or ''):
return candidate
return None
BASH = _find_bash()
needs_bash = pytest.mark.skipif(BASH is None, reason='needs a working bash')
# Same shapes the gate matches: the name literal of a created column, positional
# or keyword, on the sa.Column( line or on the line after it, plus the target of
# a rename.
@@ -98,7 +148,7 @@ def test_no_migration_creates_a_badly_named_column():
+ '\n '.join(offenders))
@pytest.mark.skipif(shutil.which('bash') is None, reason='needs bash')
@needs_bash
def test_the_gate_fails_on_a_planted_migration_column(tmp_path):
"""Plant one, and require the gate to say so.
@@ -116,20 +166,20 @@ def test_the_gate_fails_on_a_planted_migration_column(tmp_path):
planted = versions / 'zz01_planted.py'
planted.write_text(PLANTED, encoding='utf-8')
dirty = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
dirty = subprocess.run([BASH, str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert dirty.returncode == 1, (
'the gate passed a snake_case migration column:\n' + dirty.stdout)
assert 'planted_column' in dirty.stdout
planted.unlink()
clean = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
clean = subprocess.run([BASH, str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert clean.returncode == 0, (
'the gate fails with nothing planted:\n' + clean.stdout)
@pytest.mark.skipif(shutil.which('bash') is None, reason='needs bash')
@needs_bash
def test_the_gate_fails_on_a_site_literal_in_the_agent(tmp_path):
"""agent/ is in the ADR-015 scan, and has to stay in it.
@@ -146,14 +196,14 @@ def test_the_gate_fails_on_a_site_literal_in_the_agent(tmp_path):
planted = agent / 'Probe.cs'
planted.write_text(PLANTED_AGENT, encoding='utf-8')
dirty = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
dirty = subprocess.run([BASH, str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert dirty.returncode == 1, (
'the gate passed a site hostname in the agent:\n' + dirty.stdout)
assert 'Probe.cs' in dirty.stdout
planted.unlink()
clean = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
clean = subprocess.run([BASH, str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert clean.returncode == 0, (
'the gate fails with nothing planted:\n' + clean.stdout)
@@ -224,7 +274,7 @@ def _gate_tree(tmp_path):
return gate
@pytest.mark.skipif(shutil.which('bash') is None, reason='needs bash')
@needs_bash
@pytest.mark.parametrize('shape', sorted(PLANTED_COLUMN_SHAPES))
def test_the_gate_catches_every_column_shape(tmp_path, shape):
"""Plant one shape at a time and require the gate to name it.
@@ -240,7 +290,7 @@ def test_the_gate_catches_every_column_shape(tmp_path, shape):
planted = versions / 'zz01_planted.py'
planted.write_text(_migration_module(body), encoding='utf-8')
dirty = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
dirty = subprocess.run([BASH, str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert dirty.returncode == 1, (
'the gate passed a {0} column:\n{1}'.format(shape, dirty.stdout))
@@ -249,7 +299,7 @@ def test_the_gate_catches_every_column_shape(tmp_path, shape):
expected, dirty.stdout))
@pytest.mark.skipif(shutil.which('bash') is None, reason='needs bash')
@needs_bash
def test_the_gate_passes_well_named_columns(tmp_path):
"""The other half: a gate that fails on everything is no better.
@@ -260,13 +310,13 @@ def test_the_gate_passes_well_named_columns(tmp_path):
gate = _gate_tree(tmp_path)
(versions / 'zz01_clean.py').write_text(
_migration_module(PLANTED_CLEAN_BODY), encoding='utf-8')
clean = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
clean = subprocess.run([BASH, str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert clean.returncode == 0, (
'the gate failed on lowercase concatenated names:\n' + clean.stdout)
@pytest.mark.skipif(shutil.which('bash') is None, reason='needs bash')
@needs_bash
def test_the_site_literal_scan_honours_its_include_list(tmp_path):
"""The ADR-015 scan must read the file types it declares, and no others.
@@ -289,7 +339,7 @@ def test_the_site_literal_scan_honours_its_include_list(tmp_path):
(agent / name).write_text(
'shipped to ' + PLANTED_SITE_HOST + '\n', encoding='utf-8')
result = subprocess.run(['bash', str(gate)], cwd=str(tmp_path),
result = subprocess.run([BASH, str(gate)], cwd=str(tmp_path),
capture_output=True, text=True)
assert result.returncode == 0, (
'the scan read a file type it does not declare:\n' + result.stdout)

View File

@@ -297,7 +297,19 @@ def test_import_scan_covers_symlinked_plugins(tmp_path):
(external / 'plugin.py').write_text('from shopdb.extensions import db\n')
link = root / 'zzsymlinkdemo'
link.symlink_to(external, target_is_directory=True)
try:
link.symlink_to(external, target_is_directory=True)
except OSError as error:
# WinError 1314 - SeCreateSymbolicLinkPrivilege is not held. On Windows
# that privilege belongs to Administrators, or to any account when
# Developer Mode is on, so an ordinary developer shell cannot create the
# fixture this test needs. Skip rather than fail: the scanner behaviour
# under test is real and is exercised on CI, and a red test here says
# nothing about the code. Only this specific cause is skipped - any
# other OSError still fails.
if getattr(error, 'winerror', None) == 1314:
pytest.skip('needs symlink privilege (Developer Mode or admin)')
raise
try:
scanned = {str(p) for p in _plugin_source_files()}
assert any('symlinkdemo' in s for s in scanned), (

View File

@@ -123,7 +123,9 @@ EXPECTED_HEAD_REVISION['warranty'] = 'warranty0002proof'
# endpointcredentials, endpointenrollrequests and endpointmigrationwaves.
# 0024 adds endpointbootstraptokens: the short-lived, admin-minted tokens behind
# the one-line bootstrap installer.
EXPECTED_HEAD_REVISION['endpoints'] = 'endpoints0025logcollections'
# 0025 seeds the Office ProPlus LTSC monitor templates (x64 and x86). Catalog
# rows only: nothing is watched until a site policy attaches one.
EXPECTED_HEAD_REVISION['endpoints'] = 'endpoints0025officemonitors'
# Plugins built after the cutover: their 0001 baseline really creates tables the
# core chain never owned.

View File

@@ -7,6 +7,8 @@ different PC is refused (ASSET_MISMATCH), the shared key is refused for that PC
(CREDENTIAL_INVALID) so the agent knows to enroll again.
"""
import shutil
import pytest
from shopdb.core.models import Asset, AssetType, AuditLog, Setting
@@ -882,6 +884,8 @@ def test_approve_refuses_an_enrolled_computer(client, db, app, keyheaders,
# -- server certificate pin --------------------------------------------------
@pytest.mark.skipif(shutil.which('openssl') is None,
reason='needs the openssl binary on PATH')
def test_server_cert_reports_the_spki_pin(client, db, auth_headers, tmp_path,
monkeypatch):
import hashlib

View File

@@ -0,0 +1,207 @@
"""The server decides what is monitored, not the endpoint.
The agent merges its local appsettings.json watch list into the server-resolved
one, so what an endpoint reports is not by itself evidence that the server ever
asked for it. Anyone able to edit that file on the box - or an old package with
monitors baked in by the installer - can put entries into the report that no
site policy attached.
The interesting failure is not noise. It is a never-running entry that parks an
asset at a permanent condition: an operator who learns to ignore a perpetually
amber endpoint stops reading the signal that says a real service died. So the
endpoint can blind the fleet view by TALKING, not only by going quiet.
These tests pin that reported entries are filtered against the server's own
resolved monitor list before they are stored.
"""
import json
from shopdb.core.models import Asset, AssetType, Setting
def _computer(db, hostname, computertype):
from plugins.computers.models import Computer, ComputerType
atype = AssetType.query.filter_by(assettype='computer').first()
if not atype:
atype = AssetType(assettype='computer')
db.session.add(atype)
db.session.flush()
ctype = ComputerType.query.filter_by(computertype=computertype).first()
if not ctype:
ctype = ComputerType(computertype=computertype)
db.session.add(ctype)
db.session.flush()
asset = Asset(assetnumber=hostname, assettypeid=atype.assettypeid,
isactive=True)
db.session.add(asset)
db.session.flush()
comp = Computer(assetid=asset.assetid, hostname=hostname,
computertypeid=ctype.computertypeid)
db.session.add(comp)
db.session.flush()
return asset, ctype
def _attach_monitor_to_pctype(db, monitor, computertypeid):
from plugins.computers.models import (
ConfigurationPolicy, ConfigurationPolicyAction,
BusinessPolicy, BusinessPolicyItem,
)
if not Setting.query.filter_by(
key='computers_configpolicies_enabled').first():
db.session.add(
Setting(key='computers_configpolicies_enabled', value='true'))
policy = ConfigurationPolicy(
name='Coverage {}'.format(monitor.endpointmonitorid),
scopejson={'type': 'pctype', 'id': computertypeid},
)
db.session.add(policy)
db.session.flush()
db.session.add(ConfigurationPolicyAction(
configurationpolicyid=policy.configurationpolicyid,
action='monitor:{}'.format(monitor.endpointmonitorid),
mode='gate',
))
business = BusinessPolicy(name='Site {}'.format(
monitor.endpointmonitorid), priority=100)
db.session.add(business)
db.session.flush()
db.session.add(BusinessPolicyItem(
businesspolicyid=business.businesspolicyid,
configurationpolicyid=policy.configurationpolicyid,
sortorder=0,
))
db.session.commit()
def _stored_services(db, assetid):
from plugins.endpoints.models import EndpointState
state = EndpointState.query.filter_by(assetid=assetid).first()
if state is None or not state.servicesjson:
return []
return json.loads(state.servicesjson)
def _report(app, hostname, services):
from plugins.endpoints.plugin import EndpointsPlugin
plugin = EndpointsPlugin()
return plugin.apply_collector_payload({
'hostname': hostname,
'services': services,
})
def test_an_unmonitored_reported_service_is_not_stored(client, db, app):
"""The baked-in installer case: the endpoint reports dncMain.exe, but no
site policy attaches it, so it never reaches the stored snapshot."""
asset, _ctype = _computer(db, 'INGEST1', 'Shopfloor PC')
db.session.commit()
_report(app, 'INGEST1', [
{'name': 'dncMain.exe', 'kind': 'process', 'state': 'stopped'},
])
db.session.commit()
assert _stored_services(db, asset.assetid) == []
def test_a_policy_attached_service_is_stored(client, db, app):
"""The control case: filtering must not throw away legitimate reports."""
from plugins.endpoints.models import EndpointMonitor
asset, ctype = _computer(db, 'INGEST2', 'Shopfloor PC')
monitor = EndpointMonitor(
kind='process', displayname='eDNC', processname='dncMain.exe',
source='shipped', isactive=True,
)
db.session.add(monitor)
db.session.flush()
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
_report(app, 'INGEST2', [
{'name': 'dncMain.exe', 'kind': 'process', 'state': 'stopped'},
])
db.session.commit()
stored = _stored_services(db, asset.assetid)
assert [entry['name'] for entry in stored] == ['dncMain.exe']
def test_a_spoofed_never_running_service_cannot_park_the_asset_amber(
client, db, app):
"""The attack that matters: an entry the server never attached cannot
manufacture a standing health condition."""
from plugins.endpoints.services.health import _service_conditions
from plugins.endpoints.models import EndpointState
asset, _ctype = _computer(db, 'INGEST3', 'Shopfloor PC')
db.session.commit()
_report(app, 'INGEST3', [
{'name': 'totally-made-up.exe', 'kind': 'process', 'state': 'stopped'},
])
db.session.commit()
state = EndpointState.query.filter_by(assetid=asset.assetid).first()
assert _service_conditions(state, None) == []
def test_matching_ignores_case_on_the_wire_name(client, db, app):
"""The agent reports whatever Windows hands it; 0006 seeds 'dncMain.exe'
while the launch path says 'DncMain.exe'. Identity is case-insensitive or
the filter would silently drop real monitors."""
from plugins.endpoints.models import EndpointMonitor
asset, ctype = _computer(db, 'INGEST4', 'Shopfloor PC')
monitor = EndpointMonitor(
kind='process', displayname='eDNC', processname='dncMain.exe',
source='shipped', isactive=True,
)
db.session.add(monitor)
db.session.flush()
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
_report(app, 'INGEST4', [
{'name': 'DNCMAIN.EXE', 'kind': 'process', 'state': 'running'},
])
db.session.commit()
stored = _stored_services(db, asset.assetid)
assert [entry['name'] for entry in stored] == ['DNCMAIN.EXE']
def test_an_omitted_services_key_does_not_wipe_the_last_snapshot(
client, db, app):
"""Filtering sits inside the existing presence guard: a report that omits
services leaves the previous snapshot standing."""
from plugins.endpoints.models import EndpointMonitor
asset, ctype = _computer(db, 'INGEST5', 'Shopfloor PC')
monitor = EndpointMonitor(
kind='service', displayname='MTConnect', servicename='MTConnect UDC Agent',
source='shipped', isactive=True,
)
db.session.add(monitor)
db.session.flush()
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
_report(app, 'INGEST5', [
{'name': 'MTConnect UDC Agent', 'kind': 'service', 'state': 'running'},
])
db.session.commit()
from plugins.endpoints.plugin import EndpointsPlugin
EndpointsPlugin().apply_collector_payload({'hostname': 'INGEST5'})
db.session.commit()
stored = _stored_services(db, asset.assetid)
assert [entry['name'] for entry in stored] == ['MTConnect UDC Agent']

View File

@@ -0,0 +1,172 @@
"""The Office ProPlus LTSC monitor templates seeded by endpoints 0025.
Runs the migration against a bare engine and checks the rows it produces, rather
than asserting against the module's constants - a test that reads the same list
the code writes proves nothing about what lands in the table.
The behaviour worth pinning here is that 0025 seeds on a site that ALREADY has
monitors. 0006 deliberately bails out when the catalog is non-empty (it is a
first-install seed), and copying that guard into 0025 would have made it a no-op
everywhere the templates are actually wanted.
"""
import importlib.util
from pathlib import Path
from sqlalchemy import create_engine, text
from alembic.migration import MigrationContext
from alembic.operations import Operations
_ROOT = Path(__file__).resolve().parent.parent.parent
_ENDPOINTS_VERSIONS = _ROOT / 'plugins' / 'endpoints' / 'migrations' / 'versions'
OFFICE_MIGRATION = '0025_office_monitor_templates.py'
_EXPECTED_NAMES = {
'Office Word (x64)', 'Office Word (x86)',
'Office Excel (x64)', 'Office Excel (x86)',
'Office PowerPoint (x64)', 'Office PowerPoint (x86)',
'Office Outlook (x64)', 'Office Outlook (x86)',
'Office Access (x64)', 'Office Access (x86)',
}
def _load(path, prefix):
spec = importlib.util.spec_from_file_location(prefix + path.stem, str(path))
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def _run(connection, *loaded):
operations = Operations(MigrationContext.configure(connection))
operations._install_proxy()
try:
for module in loaded:
module.upgrade()
finally:
operations._remove_proxy()
connection.commit()
def _office_module():
return _load(_ENDPOINTS_VERSIONS / OFFICE_MIGRATION, 'endpoints_mig_0025_')
def _seeded(connection):
_run(
connection,
_load(_ENDPOINTS_VERSIONS / '0006_endpoint_monitors.py',
'endpoints_mig_0006_'),
_office_module(),
)
def _monitors(connection):
rows = connection.execute(text(
'SELECT displayname, kind, processname, launchpath, source '
'FROM endpointmonitors')).all()
return {row[0]: row for row in rows}
def _connection():
return create_engine('sqlite://').connect()
def test_every_office_app_is_seeded_at_both_install_roots():
with _connection() as connection:
_seeded(connection)
by_name = _monitors(connection)
assert _EXPECTED_NAMES.issubset(set(by_name))
def test_access_is_included():
"""Called out explicitly because it is the one that gets forgotten."""
with _connection() as connection:
_seeded(connection)
by_name = _monitors(connection)
assert by_name['Office Access (x64)'][2] == 'MSACCESS.EXE'
assert by_name['Office Access (x86)'][2] == 'MSACCESS.EXE'
def test_the_architectures_point_at_different_install_roots():
with _connection() as connection:
_seeded(connection)
by_name = _monitors(connection)
x64 = by_name['Office Excel (x64)'][3]
x86 = by_name['Office Excel (x86)'][3]
assert x64 == r'C:\Program Files\Microsoft Office\root\Office16\EXCEL.EXE'
assert x86 == (r'C:\Program Files (x86)\Microsoft Office\root\Office16'
r'\EXCEL.EXE')
def test_the_rows_are_shipped_process_monitors():
with _connection() as connection:
_seeded(connection)
by_name = _monitors(connection)
for name in _EXPECTED_NAMES:
assert by_name[name][1] == 'process', name
assert by_name[name][4] == 'shipped', name
def test_seeding_is_idempotent():
"""The kind/displayname unique constraint would raise on a re-run."""
with _connection() as connection:
_seeded(connection)
_run(connection, _office_module())
by_name = _monitors(connection)
assert len([name for name in by_name if name.startswith('Office ')]) == 10
def test_templates_are_seeded_on_a_site_that_already_has_monitors():
"""0006 bails out when the catalog is non-empty; 0025 must not.
Every existing site has the four shipped monitors from 0006, so copying that
guard would make this migration a no-op exactly where it is needed.
"""
with _connection() as connection:
_run(connection,
_load(_ENDPOINTS_VERSIONS / '0006_endpoint_monitors.py',
'endpoints_mig_0006_'))
before = _monitors(connection)
assert before, 'expected the 0006 shipped rows to be present'
_run(connection, _office_module())
after = _monitors(connection)
assert _EXPECTED_NAMES.issubset(set(after))
assert 'eDNC' in after
def test_seeding_does_not_disturb_the_existing_shipped_monitors():
with _connection() as connection:
_seeded(connection)
by_name = _monitors(connection)
assert by_name['eDNC'][2] == 'dncMain.exe'
assert by_name['Universal Data Collector'][2] == 'udc.exe'
def test_downgrade_removes_the_seeded_templates():
module = _office_module()
with _connection() as connection:
_seeded(connection)
operations = Operations(MigrationContext.configure(connection))
operations._install_proxy()
try:
module.downgrade()
finally:
operations._remove_proxy()
connection.commit()
by_name = _monitors(connection)
assert not _EXPECTED_NAMES & set(by_name)
assert 'eDNC' in by_name

View File

@@ -12,7 +12,8 @@ from plugins.endpoints.eventcatalog import EVENT_CATALOG, CRITICAL_EVENT_IDS
from plugins.endpoints.commands import (
COMMAND_ALLOWLIST, is_allowed_action, elevated_permission_for,
requires_target, minimum_agent_version_for, agent_meets_minimum,
parse_agent_version,
parse_agent_version, corptools_may_push_script,
CORPTOOLS_PUSHABLE_SCRIPTS,
)
@@ -168,6 +169,55 @@ def test_endpoints_declares_the_restart_permission():
assert 'endpoints.command' in permissions
def test_run_script_still_requires_the_runscript_permission():
"""The corptools allowance is an exception for one named script, not a
relaxation of run_script. The action itself must keep demanding
endpoints.runscript, otherwise a corptools holder could push anything in
the library."""
assert elevated_permission_for('run_script') == 'endpoints.runscript'
def test_corptools_may_push_only_the_shipped_report_ip_script():
"""Corporate Tools can push the managed Report IP script by name, and
nothing else. A site-authored script that happens to be selected from the
same tab gets no allowance."""
assert corptools_may_push_script('Report IP - Desktop Shortcut') is True
assert corptools_may_push_script('Uninstall Everything') is False
assert corptools_may_push_script(None) is False
assert corptools_may_push_script('') is False
def test_corptools_pushable_scripts_stays_a_single_managed_entry():
"""Guards the blast radius of the exception. Growing this set widens what
endpoints.corptools can deliver without endpoints.runscript, so adding an
entry should be a deliberate change that fails this test first."""
assert CORPTOOLS_PUSHABLE_SCRIPTS == {'Report IP - Desktop Shortcut'}
def test_corptools_allowance_name_matches_the_seeder():
"""The allowance matches on the stored script name, so the seeder and the
allowlist must agree exactly. If either string is edited alone the
Corporate Tools button silently 403s."""
from plugins.endpoints.api.reportipscript import REPORT_IP_SCRIPT_NAME
assert REPORT_IP_SCRIPT_NAME in CORPTOOLS_PUSHABLE_SCRIPTS
def test_report_ip_script_body_does_not_relaunch_itself():
"""Pins the session-0 lesson. Launched as SYSTEM this tool re-enters the
interactive session, so the launcher observes nothing and exits 0 - a
success that means 'handed off', not 'worked'. The body must therefore
stay thin and let the agent's user-context path own launching and the exit
code; re-introducing a cmd.exe wrapper or a success-string scan is what
produced the false results."""
from plugins.endpoints.api.reportipscript import template_path
with open(template_path(), 'r', encoding='utf-8') as handle:
body = handle.read()
assert 'ComSpec' not in body
assert 'completed successfully' not in body
assert '-Wait' in body
assert 'exit $process.ExitCode' in body
def test_collect_now_is_allowlisted_targetless_and_routine():
"""collect_now forces an off-cadence telemetry report. It is allowlisted so
the enqueue route accepts it, carries no wire target (fixed action), and
@@ -217,6 +267,62 @@ def test_collect_log_survives_a_policy_that_never_mentions_it():
assert elevated_permission_for('collect_log') == 'endpoints.updateapp'
def test_run_script_survives_a_policy_that_never_mentions_it(monkeypatch):
"""Script Push is gated by caller permission, not by machine shape.
Before this exemption the action stayed reachable only because a site kept
an all-PCs "allow run_script" policy row alive. That row expressed no real
configuration intent, and deleting it silently removed Script Push from
every policied endpoint at once - a failure that reads as authorization
("not permitted by policy") while the caller's grant was never in doubt.
Authority belongs on the caller axis, so a policy that is silent about
run_script must not be read as withdrawing it.
"""
import plugins.endpoints.commands as commands
class _Effective:
def allowed_actions(self):
# A real, unrelated policy: it shapes services and says nothing
# whatsoever about running scripts.
return {'service_start', 'service_stop'}
monkeypatch.setattr(
'plugins.computers.services.policyresolver.effective_policy_for',
lambda assetid: _Effective(),
)
allowed = commands.ALLOWED_ACTIONS_FOR(assetid=1)
assert 'run_script' in allowed
# Exemption from POLICY is not exemption from PERMISSION: the grant must
# survive the exemption rather than be loosened by it. The enqueue route
# checks this with caller_holds BEFORE consulting policy, so a scoped
# personal token cannot inherit it from an admin owner.
assert elevated_permission_for('run_script') == 'endpoints.runscript'
def test_policy_exemption_never_widens_past_the_code_allowlist(monkeypatch):
"""Exempt actions are re-added, but the base allowlist still bounds them.
The exemption set is unioned in after the policy intersection, so a bug
there could hand back an action the command layer does not implement.
"""
import plugins.endpoints.commands as commands
class _Effective:
def allowed_actions(self):
return {'service_start'}
monkeypatch.setattr(
'plugins.computers.services.policyresolver.effective_policy_for',
lambda assetid: _Effective(),
)
allowed = commands.ALLOWED_ACTIONS_FOR(assetid=1)
assert allowed <= set(commands.COMMAND_ALLOWLIST.keys())
def test_collect_log_declares_the_agent_version_that_implements_it():
"""collect_log names its minimum agent version.

View File

@@ -253,30 +253,69 @@ def test_a_shipped_monitor_keeps_the_identity_the_policy_matches_on(
assert monitor.displayname == 'eDNC'
# --- which PC type the shipped bundle attaches to is a setting ---------------
# --- enabling a monitor must never require a new agent ----------------------
def test_shipped_monitors_follow_the_configured_pctype(client, db, app):
from plugins.endpoints.api.monitors import (
SHIPPED_PCTYPE_SETTING, _resolve_pctype_monitors)
def test_a_monitor_created_today_reaches_the_agent_watch_list(client, db, app,
auth_headers):
"""Create a monitor over the API, attach it by policy, and it is in the
agent's watch list on the next fetch.
This is the no-redeploy contract, and it is the property most easily lost
in a refactor. The agent reads this route every cycle and merges the result
over appsettings.json, so a monitor that appears here is live on the floor
without shipping a new exe. Nothing in this test touches the agent.
"""
asset, ctype = _computer(db, 'NOREDEPLOY1', 'CMM PC')
created = client.post('/api/endpoints/monitors', json={
'kind': 'service', 'displayname': 'Print Spooler',
'servicename': 'Spooler',
}, headers=auth_headers)
assert created.status_code in (200, 201)
from plugins.endpoints.models import EndpointMonitor
monitor = EndpointMonitor.query.filter_by(servicename='Spooler').first()
assert monitor is not None
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
response = client.get(RESOLVED_URL, query_string={'hostname': 'NOREDEPLOY1'},
headers=_collector_headers(app))
assert response.status_code == 200
data = response.get_json()['data']
assert {'name': 'Spooler', 'app': 'Print Spooler'} in data['services']
# --- shipped monitors attach by POLICY ONLY ---------------------------------
def test_shipped_monitors_are_not_attached_by_pc_type_name(client, db, app):
"""The auto-attach path is gone: a shipped monitor is not expected anywhere
until a Site Policy composes it.
Replaces two tests that pinned the old behaviour - that shipped monitors
followed a configurable PC type name, and that blanking the setting turned
them off. Both described a route that should not exist. Keeping them would
have meant keeping the bug they documented: every endpoint typed
'Shopfloor PC' silently expected eDNC, UDC and MTConnect, which is what put
dncMain.exe in Fleet health for all endpoints.
"""
from plugins.endpoints.api.monitors import _resolve_pctype_monitors
_shipped(db)
Setting.set(SHIPPED_PCTYPE_SETTING, 'Shop Floor Workstation')
db.session.commit()
# The plant's own name for the type resolves the bundle...
assert _resolve_pctype_monitors('Shopfloor PC') == []
assert _resolve_pctype_monitors('Shop Floor Workstation') == []
def test_a_shipped_monitor_still_resolves_once_policy_attaches_it(
client, db, app):
"""Removing auto-attach must not make shipped monitors unusable - a site
composes them like any other monitor."""
from plugins.endpoints.api.monitors import _resolve_pctype_monitors
asset, ctype = _computer(db, 'SHIPPEDPC1', 'Shopfloor PC')
monitor = _shipped(db)
_attach_monitor_to_pctype(db, monitor, ctype.computertypeid)
assert [m.displayname for m in
_resolve_pctype_monitors('Shop Floor Workstation')] == ['eDNC']
# ...and the name this shipped with no longer does.
assert _resolve_pctype_monitors('Shopfloor PC') == []
def test_a_blank_shipped_pctype_attaches_the_bundle_to_nothing(client, db, app):
"""The neutral behaviour for a site that does not want the bundle: off,
rather than silently attached to whatever happens to match."""
from plugins.endpoints.api.monitors import (
SHIPPED_PCTYPE_SETTING, _resolve_pctype_monitors)
_shipped(db)
Setting.set(SHIPPED_PCTYPE_SETTING, '')
db.session.commit()
assert _resolve_pctype_monitors('Shopfloor PC') == []
_resolve_pctype_monitors('Shopfloor PC')] == ['eDNC']

View File

@@ -351,3 +351,33 @@ def test_install_script_helper_without_pins_or_key():
assert '[string[]] $ServerPins = @()' in script
assert 'CollectorKey' not in script
assert 'ServerPins left as is' in script
def test_the_package_ships_no_baked_in_monitors(client, db, auth_headers, servercert):
"""The server owns the watch list, so the installer must seed nothing.
Seeding monitors here is what put dncMain.exe on every endpoint in Fleet
health: the entry rode into appsettings.json on every package, and
appsettings.json is never rewritten on the box, so it outlived any policy
that justified it. The agent now replaces its watch list from
/api/endpoints/monitors/resolved, which makes a seeded entry pointless as
well as harmful.
"""
settings = json.loads(_files(_build(
client, auth_headers, collectorkey=SECRET))['appsettings.json'])
assert settings['monitors'] == {'processes': [], 'services': []}
def test_the_package_ships_no_baked_in_launch_paths(client, db, auth_headers, servercert):
"""Launch paths are server authoritative, so the installer seeds none.
A local launchpaths entry wins over the server-provisioned path AND skips
LaunchPathPolicy validation, so a seeded entry both outlives the config that
justified it and makes WebUI edits never reach the endpoint. eDNC, UDC and
MTConnect carry their launchpath on the shipped monitor rows instead.
"""
settings = json.loads(_files(_build(
client, auth_headers, collectorkey=SECRET))['appsettings.json'])
assert settings['launchpaths'] == {}

View File

@@ -0,0 +1,164 @@
"""The Dell routes must not hold a database connection across the vendor call.
A bulk sync asks Dell about 100 service tags per request and a fleet-sized run
is dozens of those, so a route that keeps its transaction open for the duration
pins a pooled connection for minutes. Under gevent workers that exhausts the
pool and takes down every other request, not just warranty.
These pin the boundary itself rather than the result: a fake provider inspects
the session at the moment it is called and records whether a transaction was
still open. Behaviour is covered elsewhere - what would silently regress here is
someone reading an ORM attribute after the commit, which re-checks a connection
straight back out and restores the original defect while every other test
continues to pass.
"""
from plugins.warranty.api import routes as warranty_routes
from plugins.warranty.services import dellsync as warranty_dellsync
from plugins.warranty.models import Warranty, WarrantyAsset
def _asset(db, assetnumber='PC-0001', serialnumber='ABC1234'):
from shopdb.core.models import AssetType, AssetStatus
from shopdb.core.models.asset import Asset
assettype = AssetType(assettype='computer')
status = AssetStatus(status='In Use')
db.session.add_all([assettype, status])
db.session.flush()
asset = Asset(
assetnumber=assetnumber,
name='Bench PC',
assettypeid=assettype.assettypeid,
statusid=status.statusid,
serialnumber=serialnumber,
)
db.session.add(asset)
db.session.commit()
return asset
class _ProbeProvider:
"""Stands in for DellProvider and records the session state when called.
The real provider blocks on HTTP here; what matters is only whether the
caller was still inside a transaction at that instant.
"""
def __init__(self, db, result):
self._db = db
self._result = result
self.in_transaction_at_call = None
self.called = False
def _observe(self):
self.called = True
# db.session is a scoped_session proxy and does not forward
# in_transaction; calling it returns the real Session for this scope.
self.in_transaction_at_call = self._db.session().in_transaction()
def lookup(self, servicetag, vendor=None):
self._observe()
return self._result
def bulk_lookup(self, servicetags):
self._observe()
return {tag: self._result for tag in servicetags}
def test_refresh_releases_the_connection_before_the_vendor_call(
client, db, auth_headers, monkeypatch):
warranty = Warranty(vendor='Dell', servicetag='ABC1234', provider='dell')
db.session.add(warranty)
db.session.commit()
warrantyid = warranty.warrantyid
probe = _ProbeProvider(db, {
'servicelevel': 'ProSupport',
'startdate': '2024-01-01',
'enddate': '2027-01-01',
})
monkeypatch.setattr(warranty_routes, 'get_provider', lambda name: probe)
response = client.post(f'/api/warranty/{warrantyid}/refresh', headers=auth_headers)
assert response.status_code == 200, response.get_json()
assert probe.called
assert probe.in_transaction_at_call is False, (
'refresh_warranty held a database connection across the vendor lookup'
)
# The write half still has to land, or releasing early would be a
# regression dressed up as a fix.
refreshed = db.session.get(Warranty, warrantyid)
assert refreshed.servicelevel == 'ProSupport'
assert refreshed.lastcheckeddate is not None
def test_sync_dell_releases_the_connection_before_the_bulk_call(
client, db, auth_headers, monkeypatch):
asset = _asset(db)
assetid = asset.assetid
probe = _ProbeProvider(db, {
'servicelevel': 'Basic Onsite',
'startdate': '2023-05-02',
'enddate': '2026-05-02',
})
monkeypatch.setattr(warranty_dellsync, 'get_provider', lambda name: probe)
response = client.post('/api/warranty/sync/dell', headers=auth_headers)
assert response.status_code == 200, response.get_json()
assert probe.called
assert probe.in_transaction_at_call is False, (
'sync_dell held a database connection across the Dell bulk lookup'
)
body = response.get_json()['data']
assert body['created'] == 1
assert body['matched'] == 1
links = WarrantyAsset.query.filter_by(assetid=assetid).all()
assert len(links) == 1
created = db.session.get(Warranty, links[0].warrantyid)
assert created.provider == 'dell'
assert created.servicetag == 'ABC1234'
def test_sync_dell_updates_instead_of_duplicating_on_a_recheck(
client, db, auth_headers, monkeypatch):
"""Preloading the existing links must not lose the Dell-by-any-signal match.
The write loop now reads from one preloaded query instead of a query per
asset. A manually-added Dell warranty carries provider 'manual', so if the
preload missed it a re-check would create a second warranty for the asset
rather than updating the one already there.
"""
asset = _asset(db)
assetid = asset.assetid
existing = Warranty(vendor='Dell', servicetag='ABC1234', provider='manual')
existing.links.append(WarrantyAsset(assetid=assetid))
db.session.add(existing)
db.session.commit()
existingid = existing.warrantyid
probe = _ProbeProvider(db, {
'servicelevel': 'ProSupport Plus',
'startdate': '2024-02-02',
'enddate': '2028-02-02',
})
monkeypatch.setattr(warranty_dellsync, 'get_provider', lambda name: probe)
response = client.post('/api/warranty/sync/dell?all=true', headers=auth_headers)
assert response.status_code == 200, response.get_json()
body = response.get_json()['data']
assert body['created'] == 0
assert body['updated'] == 1
links = WarrantyAsset.query.filter_by(assetid=assetid).all()
assert len(links) == 1, 'a re-check duplicated the warranty instead of updating it'
updated = db.session.get(Warranty, existingid)
assert updated.provider == 'dell'
assert updated.servicelevel == 'ProSupport Plus'

View File

@@ -0,0 +1,116 @@
"""The Dell sync must be bounded per run and must make progress across runs.
`bulk_lookup` asks Dell about 100 tags per HTTP call at HTTP_TIMEOUT seconds
each, so an unbounded fleet-sized sync is dozens of sequential calls inside one
web request - long enough for a gunicorn worker timeout or a proxy to kill it
first. The fix is a per-run cap, with the operator re-running until nothing is
left pending.
What would silently regress: the cap being dropped (the route goes back to
unbounded), or the chunk being taken off a stable ordering. The second is
subtle - a serial Dell does not recognize never gains a warranty, so it stays
pending forever, and under a stable order those dead tags would fill every
chunk and the sync would never reach the rest of the fleet.
"""
from plugins.warranty.services import dellsync as warranty_dellsync
from plugins.warranty.services.dellsync import run_dell_sync
from plugins.warranty.models import Warranty, WarrantyAsset
def _assets(db, count, prefix='PC'):
from shopdb.core.models import AssetType, AssetStatus
from shopdb.core.models.asset import Asset
assettype = AssetType(assettype='computer')
status = AssetStatus(status='In Use')
db.session.add_all([assettype, status])
db.session.flush()
for index in range(count):
db.session.add(Asset(
assetnumber=f'{prefix}-{index:04d}',
name=f'Bench PC {index}',
assettypeid=assettype.assettypeid,
statusid=status.statusid,
serialnumber=f'TAG{index:04d}',
))
db.session.commit()
class _CountingProvider:
"""Records every tag it was asked about, and answers for none of them."""
def __init__(self, known=None):
self.seen_batches = []
self._known = known or {}
def bulk_lookup(self, servicetags):
self.seen_batches.append(list(servicetags))
return {tag: self._known[tag] for tag in servicetags if tag in self._known}
def test_sync_is_capped_at_the_limit_and_reports_the_remainder(app, db, monkeypatch):
with app.app_context():
_assets(db, 25)
provider = _CountingProvider()
monkeypatch.setattr(warranty_dellsync, 'get_provider', lambda name: provider)
summary = run_dell_sync(limit=10)
assert len(provider.seen_batches) == 1
assert len(provider.seen_batches[0]) == 10, 'more tags went to Dell than the limit'
assert summary['tags'] == 10
assert summary['remaining'] == 15
def test_no_limit_checks_everything(app, db, monkeypatch):
with app.app_context():
_assets(db, 12)
provider = _CountingProvider()
monkeypatch.setattr(warranty_dellsync, 'get_provider', lambda name: provider)
summary = run_dell_sync(limit=None)
assert summary['tags'] == 12
assert summary['remaining'] == 0
def test_repeated_runs_reach_tags_dell_does_not_recognize(app, db, monkeypatch):
"""Unmatched tags stay pending, so chunks must not keep re-picking the same ones.
Every asset here is unknown to Dell, which is the worst case: nothing ever
becomes covered, so the pending set never shrinks. A stable ordering would
send the same first chunk every run and the later tags would never be
looked up at all.
"""
with app.app_context():
_assets(db, 30)
provider = _CountingProvider()
monkeypatch.setattr(warranty_dellsync, 'get_provider', lambda name: provider)
for _ in range(12):
run_dell_sync(limit=5)
reached = {tag for batch in provider.seen_batches for tag in batch}
assert len(reached) > 5, (
'repeated runs never got past the first chunk - the sync cannot drain'
)
def test_covered_assets_are_skipped_so_runs_resume(app, db, monkeypatch):
with app.app_context():
_assets(db, 5)
known = {
'TAG0000': {'servicelevel': 'Basic Onsite',
'startdate': '2023-05-02', 'enddate': '2026-05-02'},
}
provider = _CountingProvider(known=known)
monkeypatch.setattr(warranty_dellsync, 'get_provider', lambda name: provider)
first = run_dell_sync(limit=None)
assert first['created'] == 1
second = run_dell_sync(limit=None)
assert second['tags'] == 4, 'a covered asset was looked up again'
assert 'TAG0000' not in provider.seen_batches[-1]

View File

@@ -1,86 +0,0 @@
# ShopDB MCP server
Exposes a ShopDB instance to an LLM client (Claude Code, Claude Desktop, or any
other MCP client) so it can discover and call the API directly, instead of
someone generating a client by hand from the spec.
## The token is the security boundary
**Read this before pointing it at production.**
Every operation the credential is allowed to perform is reachable through this
server, including writes and deletes. There is no separate allow-list here: the
scope of the personal access token decides what can happen, and nothing else
does.
That is a deliberate choice, and it puts the decision where it can be enforced
server-side rather than in a client that a model is talking to. It also means:
- For anything exploratory, create a **read-scoped token**. A read-scoped token
cannot be talked into a write, whatever the model decides to call.
- Give it the narrowest scopes that cover the task, not your admin session.
- `DELETE` against an asset database is irreversible. The tool descriptions say
so, which makes a model markedly more careful, but a token that permits it is
the only thing standing in the way.
Create a token under Settings > API Tokens. `shopdb_whoami` reports what the
token can do; call it first.
## Install
pip install -e tools/mcp
Then register it with your client. For Claude Code:
claude mcp add shopdb \
--env SHOPDB_URL=https://shopdb.example.net/shopdb \
--env SHOPDB_TOKEN=<token> \
-- shopdb-mcp
`SHOPDB_URL` must include the mount path if the site is served under one
(`/shopdb`), because that is what the instance's own spec declares.
| Variable | Required | Meaning |
|---|---|---|
| `SHOPDB_URL` | yes | Base URL including any mount path |
| `SHOPDB_TOKEN` | yes | Personal access token |
| `SHOPDB_VERIFY_TLS` | no | `0` to accept a self-signed certificate. Defaults to verifying |
## Tools
| Tool | Purpose |
|---|---|
| `shopdb_whoami` | Instance, identity, scopes, operation count. Call first |
| `shopdb_search_operations` | Find operations by keyword or tag. Start here |
| `shopdb_describe_operation` | Parameters, request body and responses for one operation |
| `shopdb_call` | Call any operation |
| `shopdb_llms_guide` | The instance's own orientation guide |
## Why five tools and not 483
ShopDB publishes 483 operations. One tool each would be unusable: MCP clients
degrade badly past a few dozen tools and models start choosing wrong ones. So
the surface is small and generic, and the instance's own published index is what
makes it navigable - `search` reads `/api/docs/operations.txt` (about 70 KB for
every operation) and `describe` fetches only the tag group it needs rather than
the 700 KB spec.
Nothing is generated ahead of time. Enabling a plugin adds operations this
server can already reach, with no change here and no reinstall.
## Staying current
The server reads everything from the live instance, so it describes whatever
that instance actually serves. The staleness risk is upstream: `docs/openapi.json`
is generated and committed, and if it drifts, the instance serves a stale spec
and this server faithfully reports stale information.
That is guarded by the `api-contract` CI job and by
`python scripts/gen_openapi.py --refresh`. See CONTRIBUTING.md.
## Limitations
- Air-gapped sites need `mcp` vendored; it is not bundled here.
- No streaming or subscriptions - every tool is a request and a response.
- Responses over 100 KB are truncated with a note. Paginate instead
(`perpage` is silently capped at 100 by the API).

View File

@@ -1,16 +0,0 @@
[project]
name = "shopdb-mcp"
version = "0.1.0"
description = "MCP server for a ShopDB instance"
requires-python = ">=3.10"
dependencies = ["mcp>=1.2"]
[project.scripts]
shopdb-mcp = "shopdb_mcp:main"
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
include = ["shopdb_mcp*"]

View File

@@ -1,8 +0,0 @@
"""MCP server exposing a ShopDB instance to an LLM client."""
__all__ = ['main']
def main():
from .server import main as _main
_main()

View File

@@ -1,206 +0,0 @@
"""HTTP access to a ShopDB instance, and the operation index it publishes.
Everything the server knows about the API is fetched from the instance at
runtime: the operation index, the schemas, the mount path. Nothing is generated
ahead of time and nothing is pinned, so a site running an older or newer build
is described accurately rather than according to whatever was true when this
package was written.
"""
from __future__ import annotations
import json
import os
import re
import threading
import time
from dataclasses import dataclass
from typing import Any
import http.client as http_client
from urllib import error, parse, request
# How long a fetched index or spec stays usable before it is re-read. Long
# enough that a burst of tool calls does not re-download a 700 KB document,
# short enough that enabling a plugin shows up without restarting the server.
CACHE_SECONDS = 300
DEFAULT_TIMEOUT = 30
# Verbs that change or destroy data. Named so the tool descriptions can say so:
# a model told that DELETE is irreversible behaves better than one left to infer
# it from the verb.
WRITE_METHODS = {'POST', 'PUT', 'PATCH', 'DELETE'}
DESTRUCTIVE_METHODS = {'DELETE'}
class ShopdbError(RuntimeError):
"""An error worth showing the caller verbatim."""
@dataclass
class Operation:
method: str
path: str
tag: str
auth: str
summary: str
def matches(self, needle: str) -> bool:
blob = f'{self.method} {self.path} {self.tag} {self.summary}'.lower()
return all(word in blob for word in needle.lower().split())
class ShopdbClient:
"""Talks to one ShopDB instance with one credential."""
def __init__(self, base_url: str | None = None, token: str | None = None,
verify_tls: bool | None = None, timeout: int = DEFAULT_TIMEOUT):
self.base_url = (base_url or os.environ.get('SHOPDB_URL') or '').rstrip('/')
self.token = token or os.environ.get('SHOPDB_TOKEN') or ''
if verify_tls is None:
# Sites commonly run a self-signed certificate on an internal name,
# so this has to be settable - but it defaults to verifying, and the
# caller has to say otherwise deliberately.
verify_tls = (os.environ.get('SHOPDB_VERIFY_TLS', '1').strip().lower()
not in ('0', 'false', 'no'))
self.verify_tls = verify_tls
self.timeout = timeout
self._lock = threading.Lock()
self._cache: dict[str, tuple[float, Any]] = {}
if not self.base_url:
raise ShopdbError(
'SHOPDB_URL is not set. Point it at the instance, including any '
'mount path, for example https://shopdb.example.net/shopdb')
# ---- plumbing --------------------------------------------------------
def _opener(self):
if self.verify_tls:
return request.build_opener()
import ssl
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
return request.build_opener(request.HTTPSHandler(context=ctx))
def request(self, method: str, path: str, params: dict | None = None,
body: Any = None) -> tuple[int, Any]:
"""One HTTP call. Returns (status, parsed-body-or-text)."""
# Encode what a caller typed. A model filling an id into a templated
# path can easily produce a space or an accent; leaving those raw turns
# a legitimate request into an exception rather than a 404. '/' and the
# query delimiters stay literal so an already-formed path still works.
url = self.base_url + '/' + parse.quote(path.lstrip('/'), safe='/?=&%+,')
if params:
# Drop empty values rather than sending key= , which several list
# endpoints treat as a filter on the empty string.
clean = {k: v for k, v in params.items() if v not in (None, '')}
if clean:
url += ('&' if '?' in url else '?') + parse.urlencode(clean, doseq=True)
data = None
headers = {'Accept': 'application/json'}
if self.token:
headers['Authorization'] = f'Bearer {self.token}'
if body is not None:
data = json.dumps(body).encode('utf-8')
headers['Content-Type'] = 'application/json'
try:
req = request.Request(url, data=data, headers=headers,
method=method.upper())
with self._opener().open(req, timeout=self.timeout) as response:
return response.status, _decode(response.read(),
response.headers.get('Content-Type'))
except error.HTTPError as exc:
# An error body from this API is the useful part (it carries the
# error code and message), so return it rather than raising.
return exc.code, _decode(exc.read(), exc.headers.get('Content-Type'))
except error.URLError as exc:
raise ShopdbError(f'could not reach {self.base_url}: {exc.reason}') from exc
except (OSError, ValueError, http_client.HTTPException) as exc:
# Catching only the two urllib errors let ordinary mistakes escape
# as raw exceptions: a space in a path raises InvalidURL, a non
# ASCII path raises UnicodeEncodeError, a read that outruns the
# timeout raises TimeoutError. The caller then saw a bare "error
# executing tool" with no cause and nothing to try differently.
raise ShopdbError(
f'{method.upper()} {path} could not be sent: '
f'{type(exc).__name__}: {exc}') from exc
def _cached(self, key: str, fetch):
with self._lock:
hit = self._cache.get(key)
if hit and (time.time() - hit[0]) < CACHE_SECONDS:
return hit[1]
value = fetch()
with self._lock:
self._cache[key] = (time.time(), value)
return value
# ---- what the instance publishes about itself ------------------------
def operations(self) -> list[Operation]:
"""The published operation index, parsed.
Read from /api/docs/operations.txt rather than the full spec: the index
is around 70 KB against 700 KB, and listing what exists does not need
the schemas.
"""
def fetch():
status, text = self.request('GET', '/api/docs/operations.txt')
if status != 200:
raise ShopdbError(
f'operation index unavailable (HTTP {status}). This instance '
'may predate /api/docs/operations.txt.')
return _parse_index(text if isinstance(text, str) else str(text))
return self._cached('operations', fetch)
def spec(self, tag: str | None = None) -> dict:
"""The OpenAPI spec, optionally narrowed to one tag group."""
def fetch():
path = '/api/docs/openapi.json'
status, payload = self.request('GET', path, params={'tag': tag} if tag else None)
if status != 200:
raise ShopdbError(f'spec unavailable (HTTP {status}): {payload}')
return payload
return self._cached(f'spec:{tag or "*"}', fetch)
def find_operation(self, method: str, path: str) -> dict | None:
"""The spec entry for one operation, fetched by its tag to stay small."""
method = method.lower()
for op in self.operations():
if op.method.lower() == method and op.path == path:
spec = self.spec(op.tag)
return (spec.get('paths', {}).get(path, {}) or {}).get(method)
return None
_INDEX_LINE = re.compile(
r'^(?P<method>[A-Z]+)\s+(?P<path>\S+)\s+\[(?P<tag>[^\]]*)\]\s+'
r'\((?P<auth>[^)]*)\)\s*(?P<summary>.*)$')
def _parse_index(text: str) -> list[Operation]:
out = []
for line in text.splitlines():
line = line.strip()
if not line or line.startswith('#'):
continue
m = _INDEX_LINE.match(line)
if m:
out.append(Operation(m.group('method'), m.group('path'),
m.group('tag'), m.group('auth'),
m.group('summary').strip()))
return out
def _decode(raw: bytes, content_type: str | None):
text = raw.decode('utf-8', errors='replace')
if content_type and 'json' in content_type.lower():
try:
return json.loads(text)
except json.JSONDecodeError:
return text
return text

View File

@@ -1,238 +0,0 @@
"""MCP server for a ShopDB instance.
WHY IT IS SHAPED LIKE THIS. ShopDB publishes 483 operations. Exposing one tool
per operation would be useless: MCP clients degrade badly past a few dozen tools
and a model starts picking the wrong one. So the surface here is small and
generic - find an operation, read its contract, call it - and the instance's own
published index is what makes that navigable. Adding a plugin adds operations
that this server can already reach, with no change here.
SECURITY MODEL: the token IS the boundary. Every operation the credential is
allowed to perform is reachable, including writes and deletes. That is a
deliberate choice, and it means the scope of the personal access token decides
the blast radius, not this code. Create a read-scoped token for anything
exploratory. See README.md.
"""
from __future__ import annotations
import json
import os
from typing import Any
# The SDK renamed FastMCP to MCPServer in 2.0. The two have the same shape for
# what this server uses (a .tool() decorator and .run()), so support both rather
# than forcing a pin: a site may already have either installed.
try:
from mcp.server.mcpserver import MCPServer as _Server # mcp >= 2
except ImportError: # pragma: no cover
from mcp.server.fastmcp import FastMCP as _Server # mcp 1.x
from .client import (DESTRUCTIVE_METHODS, WRITE_METHODS, ShopdbClient,
ShopdbError)
mcp = _Server('shopdb')
_client: ShopdbClient | None = None
def client() -> ShopdbClient:
global _client
if _client is None:
_client = ShopdbClient()
return _client
def _fail(message: str) -> str:
"""Errors come back as text the model can read and act on.
Raising would surface a stack trace the model cannot do anything with; a
sentence telling it what went wrong and what to try next is more useful.
"""
return f'ERROR: {message}'
@mcp.tool()
def shopdb_whoami() -> str:
"""Check which ShopDB instance this server targets and who the token is.
Call this first: it confirms the instance is reachable and the credential is
valid.
IMPORTANT: the permissions reported are the ACCOUNT's, not the token's. A
scoped personal access token carries its own narrower scope list, which the
server enforces but does not publish, so a call can still be refused with
403 even though the account shown here holds the permission. Treat this as
"who am I", not "what may I do".
"""
try:
c = client()
except ShopdbError as exc:
return _fail(str(exc))
try:
status, body = c.request('GET', '/api/auth/me')
except ShopdbError as exc:
return _fail(str(exc))
if status == 401:
return _fail('token rejected (HTTP 401). Check SHOPDB_TOKEN.')
if status != 200:
return _fail(f'HTTP {status} from /api/auth/me: {body}')
user = (body or {}).get('data') or {}
try:
count = len(c.operations())
except ShopdbError as exc:
count = f'unknown ({exc})'
return json.dumps({
'instance': c.base_url,
'username': user.get('username'),
'roles': user.get('roles'),
# Named account_permissions, not permissions, because that is what it
# is. Calling it "permissions" next to a token invited exactly the
# wrong reading: that an admin account means every call will succeed,
# when a read-scoped token on that same account is refused a write.
'account_permissions': len(user.get('permissions') or []),
'note': ('permissions shown are the account\'s; a scoped token may '
'allow fewer, and a 403 from shopdb_call means the TOKEN '
'lacks the scope even if the account holds the permission'),
'operations_published': count,
'tls_verified': c.verify_tls,
}, indent=2)
@mcp.tool()
def shopdb_search_operations(query: str = '', tag: str = '',
limit: int = 40) -> str:
"""Find API operations by keyword. START HERE to discover what exists.
Searches method, path, tag and summary. `query` words are ANDed, so
"printer supplies" matches an operation mentioning both. `tag` narrows to
one group (for example plugin-printers, core-assets).
Returns one line per operation. Use shopdb_describe_operation for the
parameters and body schema of anything you intend to call.
"""
try:
ops = client().operations()
except ShopdbError as exc:
return _fail(str(exc))
if tag:
ops = [o for o in ops if o.tag == tag]
if query:
ops = [o for o in ops if o.matches(query)]
if not ops:
tags = sorted({o.tag for o in client().operations()})
return (f'No operations matched query={query!r} tag={tag!r}.\n'
f'Available tags: {", ".join(tags)}')
lines = [f'{o.method} {o.path} [{o.tag}] ({o.auth}) {o.summary}'
for o in ops[:limit]]
more = len(ops) - len(lines)
if more > 0:
lines.append(f'... and {more} more; narrow the query or raise limit')
return '\n'.join(lines)
@mcp.tool()
def shopdb_describe_operation(method: str, path: str) -> str:
"""Full contract for one operation: parameters, request body, responses.
`path` must be the templated form exactly as shopdb_search_operations
printed it, for example /api/machines/{machineid} - not a filled-in URL.
Read this before calling anything you have not called before. It also warns
when the operation writes or deletes.
"""
try:
op = client().find_operation(method, path)
except ShopdbError as exc:
return _fail(str(exc))
if op is None:
return _fail(f'no operation {method.upper()} {path}. '
'Use shopdb_search_operations to find the exact path.')
out = {
'method': method.upper(),
'path': path,
'summary': op.get('summary'),
'description': op.get('description'),
'parameters': op.get('parameters'),
'requestBody': op.get('requestBody'),
'responses': {k: {'description': v.get('description')}
for k, v in (op.get('responses') or {}).items()},
}
warning = ''
if method.upper() in DESTRUCTIVE_METHODS:
warning = ('\n\nWARNING: this DELETES data and cannot be undone. '
'Confirm with the user before calling it.')
elif method.upper() in WRITE_METHODS:
warning = '\n\nNote: this MODIFIES data.'
return json.dumps(out, indent=2, default=str) + warning
@mcp.tool()
def shopdb_call(method: str, path: str, params: dict[str, Any] | None = None,
body: dict[str, Any] | None = None) -> str:
"""Call any ShopDB operation. Use shopdb_describe_operation first.
`path` may contain filled-in ids (/api/machines/42). `params` become the
query string; `body` is sent as JSON.
Every response is the envelope {status, data, meta} - the payload you want
is under "data", not at the top level. List endpoints paginate with page and
perpage, and perpage is silently capped at 100, so asking for 500 returns
100 without an error.
This can write and delete if the token allows it. For DELETE, confirm with
the user first.
"""
try:
c = client()
except ShopdbError as exc:
return _fail(str(exc))
verb = method.upper()
try:
status, payload = c.request(verb, path, params=params, body=body)
except ShopdbError as exc:
return _fail(str(exc))
if status == 401:
return _fail('HTTP 401. The token is missing, expired or rejected.')
if status == 403:
return _fail('HTTP 403. The token lacks the scope or role for this '
'operation - check shopdb_whoami.')
if status == 404 and isinstance(payload, str):
return _fail(f'HTTP 404 and a non-JSON body, which usually means the '
f'path is wrong rather than the record missing: {path}')
text = payload if isinstance(payload, str) else json.dumps(payload, indent=2,
default=str)
if len(text) > 100_000:
text = text[:100_000] + '\n... truncated; narrow with params or paginate'
return f'HTTP {status}\n{text}'
@mcp.tool()
def shopdb_llms_guide() -> str:
"""The instance's own orientation guide for agents.
Short. Covers the response envelope, authentication and how the rest of the
documentation is laid out. Worth reading once at the start of a session.
"""
try:
status, text = client().request('GET', '/api/docs/llms.txt')
except ShopdbError as exc:
return _fail(str(exc))
if status != 200:
return _fail(f'guide unavailable (HTTP {status})')
return text if isinstance(text, str) else str(text)
def main() -> None:
if not os.environ.get('SHOPDB_URL'):
raise SystemExit(
'SHOPDB_URL is not set.\n'
' SHOPDB_URL=https://shopdb.example.net/shopdb\n'
' SHOPDB_TOKEN=<personal access token>\n'
'See tools/mcp/README.md.')
mcp.run()
if __name__ == '__main__':
main()