Commit Graph

33 Commits

Author SHA1 Message Date
cproudlock
fa6b99bf91 Finish a restore without a browser, from the set that was asked for
Some checks failed
CI / naming (push) Successful in 4s
CI / frontend (push) Successful in 14s
CI / migrations-mysql (push) Failing after 14s
CI / backend (push) Failing after 4m57s
Closes the three defects in the imported restore path, at Ben's request. Each
was proven against a real stack before and after: two backup sets with different
data and different instance files, then a restore of the OLDER one, which is the
case that was broken.

A restore no longer depends on a tab staying open. The backup sidecar has no
Flask, so it loads the dump and leaves a sentinel for the api container to run
the migrations. The only thing that ever acted on that sentinel was the poll
loop of the modal that queued the restore, so closing the tab - or letting a
token expire - during the minutes a load takes left the data at the old revision
under new code, every query on a newer column failing, and nothing retrying. A
watcher in the api container now claims and runs it. The endpoint no longer runs
the upgrade at all; it only makes sure something is pending, which is also how a
failed upgrade is retried. That takes Alembic out of the request path, where it
could outrun the gunicorn worker timeout and be killed mid-chain, leaving a
half-applied revision the next attempt trips over.

The claim is a rename, because rename(2) is atomic: several workers and a
browser poll can race the same sentinel and exactly one wins. A failed upgrade
parks itself as .upgrade-failed rather than re-arming, since putting it back
would rerun the same broken chain on every poll forever.

Instance files come from the set being restored. The restic call asked for
'latest' regardless of which set was named, so restoring anything but the newest
paired that set's DATABASE with the NEWEST snapshot's uploads and plugins.json -
a mismatch nothing downstream could detect. Now scoped by the set's own tag,
which backup.sh has always written. The tree is replaced rather than restored
over, so files created after the snapshot are gone afterwards: a restore has to
put the site back to the snapshot, not merge into it. Done as a staged swap
rather than 'restic restore --delete --target /', because that aims a delete
pass at the container root and the cost of getting its filter wrong is the whole
filesystem.

Nothing destructive happens before a copy is taken. The dump carries DROP TABLE
per table with no transaction around it, and the instance tree is replaced
wholesale, so a wrong set or a failure part-way through left nothing to go back
to. restore.sh now writes a pre-restore-<stamp>/ set - dump, instance tarball
and a README saying what it is and how to load it - and refuses to touch
anything if that copy cannot be written.

Two more found while proving the above, both of which stopped the overlay
working at all:

* Every script under scripts/backup/ was committed non-executable, while every
  other .sh in the repo is 755. The compose entrypoints are
  /opt/backup/watch.sh and backup.sh from a read-only bind mount, so the backup
  and watcher containers could not start from a clean checkout on any machine,
  and nothing could fix it at runtime.

* Dockerfile.backup could not build without the corporate CA staged. A plain
  COPY of the gitignored cert fails outright; the trailing-glob form does too,
  because BuildKit fails a COPY whose pattern matches nothing - which is exactly
  the case being handled. Copying the directory and testing for the file inside
  it is the form that works both ways. Verified by building the image with no
  cert present.

Suite 2600 passed / 9 skipped. The finalizer has unit tests covering the atomic
claim, the parked failure and the endpoint no longer running the upgrade.

Scope: product
2026-09-06 17:38:10 -04:00
cproudlock
6e36df841e Import the wilm endpoint telemetry, PC Detail rework and backup overlay
Ports 37 of Benjamin Baker's commits from the wilm branch, mirrored from the
corporate GitHub at 7905917c. wilm rides the publication lineage and shares no
ancestor with wj, so this is a filtered tree patch rather than a merge:
94 files, and every path the publication export strips (docs/, tools/, mcp/,
CLAUDE.md, start-*.sh, .gitea/, the three excluded tests) is held back, because
a raw diff would delete each of them from this tree.

What comes across:

* Endpoint telemetry, phase 2. Physical disk health from two sources, NIC link
  state, boot type, Defender health, processor/memory/chassis inventory, the
  listening TCP set, mapped network drives and Windows Update history. Five
  additive migrations, 0012 through 0016, off the 0011 head.
* Mapped drives are collected by a probe running in the interactive user's
  session, because they do not exist in the agent's session 0. Two Windows
  sources can disagree about one drive and the worse status wins, so a
  half-broken share is never rendered as healthy.
* Windows events are reported once each, keyed on the channel's own record id,
  instead of being re-sent whenever a report overlapped the last one.
* PC Detail rework: Details cards flow, the hand-written Network card gives way
  to a full address history, Storage and disk health merge into one card with
  two sections, Operator Reports and Corporate Tools get their own tabs.
* An opt-in restic backup overlay with a third container, restore gated on the
  dump's own data revision rather than the code head, and backup/restore history.
* The agent's version is read from the binary's PE version rather than trusted
  from the csproj, and the Fleet view gains an Agent Version column.
* The enqueue route now asks caller_holds instead of the owner's permissions,
  which closes the scoped-token elevation on run_script, update_app and
  system_restart.

Held back deliberately:

* agent/installer/install-shopdbagent.ps1 stays deleted, as upstream deleted it.
  The installer that reaches a bay is generated by _install_script() in
  plugins/endpoints/api/siteagent.py; the standalone script had stopped being
  shipped and was collecting fixes no endpoint ever ran.
* The .gitignore hunk adding docs/*.md. Correct on the publication lineage,
  where docs/ becomes the wiki and its absence is how the gates detect a
  published checkout. Here it would silently ignore every new doc.
* Their deletions of docs/DEV-BUILD-TEST.md and docs/PLUGIN-HOOKS.md. Upstream
  those were stubs; here PLUGIN-HOOKS.md is the 31KB document CLAUDE.md points
  readers at.
* Five zero-byte files committed by an accidental `git add -A`, a run log from
  the author's own machine, and docs/notes/, which is working scratch that says
  so in its own header and belongs in devnotes/ on this side.

Carried with the import so it lands green: the 17 new routes are described in
docs/api-inventory.json with the spec and project map regenerated, and two new
frontend specs read fixtures through fileURLToPath(URL) which node rejects under
jsdom's own URL global.

Suite 2591 passed / 9 skipped; frontend 391 passed; naming, style and doc gates
clean.

Scope: product
2026-09-06 16:47:24 -04:00
cproudlock
13be0ef90f Import the wilm agent store and WINC adoption work, with four fixes
Eleven commits from the sister site: a reversible Maximo/WINC machine-adoption
import, a versioned agent binary store with upload and rollback, remote-BuildKit
CI, tray version reporting, and UI fixes. Applied as a patch again - the
published mirror rewrites hashes, so the branches have unrelated histories and
git refuses to merge them.

Reviewed by three agents before merging. Four things fixed here.

THE PUBLISH WORKFLOW WAS DEAD. The new remote-BuildKit step in
publish-images.yml sits at column 0 instead of indented under steps:, so the
whole file is invalid YAML and GitHub would refuse it. Their entire GHCR
distribution channel - which Start-ShopDBDev.ps1 and the compose files consume -
was broken at the source and clearly never run. Indented; all five workflows now
parse.

THE IMPORT'S APPROVAL CHECK COULD NOT FAIL. commit() built the plan and the
approval packet from the same upload and then asked the writer whether they
matched, so packet_matches_plan compared a hash against itself. The documented
contract - "a human approves a specific, frozen plan, and the writer can prove
nothing drifted" - did not exist in the web path: an admin could preview file A
and commit file B, gated only by the needs-create group NAMES they ticked. The
preview's planhash is now required at commit and checked against a freshly
rebuilt plan. Their tests committed without previewing, which is what let this
through; they go through the real flow now, and three new tests pin it - all
confirmed to fail with the check stubbed out.

endpoints.agentpackage QUIETLY GREW. It used to mean "wrap the staged exe in a
ZIP"; with the upload routes it also means "choose the executable every future
package installs and runs as SYSTEM". Before this, staging needed filesystem
access to the server. The permission description still said only "generate and
download a package", so an admin granting it could not see what they were
granting. Reworded.

NOTHING RECORDED WHAT WAS STAGED, OR BY WHOM. The manifest held version, size
and timestamp; re-uploading a version replaces the bytes in place under the same
label. A sha256 of the stored file is now recorded, and the upload is
audit-logged with it - after an incident there was previously no answer to who
put that binary on the fleet.

Also documented the nine new routes in docs/api-inventory.json (that gate cannot
run on the mirror, so it falls to us at promotion time again), regenerated the
spec and the project map, and kept our own docs where they conflicted: docs/ is
scrubbed from the mirror, so their DEV-BUILD-TEST.md still had the dev-be
channel mapping we corrected, and their PLUGIN-HOOKS.md was a new 45-line file
rather than our 737-line canonical one. Their journal section is appended to
ours.

Contract version comes across as 0.30.0 for a genuinely additive surface.

Suite 2369, frontend 324, gate green.

Scope: product
2026-09-03 07:54:14 -04:00
cproudlock
56ed89bb7b Document the ten new endpoint routes and undo the stray re-indent
Two of the three things holding the wilm import out of main.

The monitor CRUD, the two resolver reads and the three operator-report routes
had no entry in docs/api-inventory.json, so test_api_inventory_parity failed.
The gate cannot run on the published mirror - the test ships but its data file
is scrubbed, and it skips on the file's absence - which is why the branch that
added the routes never saw it. Cost lands here at promotion time instead.

/api/endpoints is registered both with and without a trailing slash and both
forms are served, so both are described rather than guessing which one counts.
Spec regenerated: 353 paths, 504 operations.

plugins/endpoints/plugin.py had a contiguous run of continuation lines shifted
by one space. Python accepted it and git diff -w showed nothing, but it put
deletions in a diff that is otherwise purely additive. Restored; the file now
differs from the published main by additions only.

Scope: product
2026-09-02 13:49:54 -04:00
cproudlock
df083776b3 Offer the search box to whoever can actually search
All checks were successful
CI / naming (push) Successful in 3s
CI / frontend (push) Successful in 12s
CI / migrations-mysql (push) Successful in 42s
CI / backend (push) Successful in 4m24s
The sidebar box was drawn for anyone signed in, on the reasoning that
offering it to a signed-out reader gives them a control that can only
answer 401. That stopped being the right test when the guest role became
able to hold assets.view: the box was hidden from exactly the visitor the
grant was reopened for, and a guest could reach search only by typing
/search?q= by hand.

The client cannot decide this. Its hasPermission getter reads the signed-in
user's permission list and a guest has no user at all, so asking it about a
guest answers no for everyone. get_navigation already had the problem and
already solved it by letting the server decide, so capabilities does the
same thing and reports what the CALLER holds.

It reports booleans about the caller and never what the guest role holds in
general, so it tells an anonymous reader nothing the rendered page does not.
Public because it is the pre-login call that decides whether the box is
drawn at all.

Members who lack assets.view now lose a box that only ever 401'd for them.
On a failed call the box stays hidden rather than falling back to the old
signed-in test, which would put it back in front of those same members.

Scope: product
2026-08-31 09:56:38 -04:00
cproudlock
5f27f81e88 Clear the review blockers on the merged wilm work
All checks were successful
CI / naming (push) Successful in 3s
CI / frontend (push) Successful in 13s
CI / migrations-mysql (push) Successful in 42s
CI / backend (push) Successful in 4m18s
A multi-agent review of the merged branch produced 29 findings that survived
adversarial verification. This clears the blockers and the four worth fixing
before it reaches main. The design held up well: path-traversal defence on all
four serving routes is clean, the agent's SHA-256 check fails closed, enqueue
authority is proportionate and audited, and no site literals or credential leaks
in the new C#.

THE TESTS NEVER RAN

pytest.ini sets testpaths = tests, so plugins/endpoints/tests/ was collected ZERO
times. Fifty-five lines of delivery assertions were dead code, and run directly
five of them failed. plugins/printedparts/tests/ had been invisible for longer.

That is the worst shape a gap can take: the suite is green, the coverage is
imaginary, and the collector auth on a download route could be deleted without
anything going red.

Both files moved under tests/. tests/test_no_hidden_tests.py now fails if a test
file appears anywhere pytest will not look; it caught a staged copy inside the
installer bundle on its first run, and was verified by planting a violation.

The five failures were stale expectations, and they split by cause. His: the
applications panel made six panels where his test said five, and 'applications'
belongs in KNOWN_PAYLOAD_KEYS now that it is persisted - his own test contradicted
his own feature. Ours: we renamed the events card to activity, moved the
fleet-wide cards to endpoints.fleet, and made the list route a bare route('')
which Flask rejects unless the blueprint is mounted under a prefix.

FOUR FIXES

The snapshot table's key was a 32-bit autoincrement on a table rewritten
wholesale every check-in, so it climbs at (endpoints x apps) per report and
exhausts on MySQL. Widened while the table is unreleased. Widening it broke every
insert on SQLite, which only autoincrements a column typed exactly INTEGER
PRIMARY KEY, so it carries a dialect variant - and nothing caught either fault,
because no test inserted into that table at all. One does now.

Ingest truncated displayname but not displayversion or publisher. On strict MySQL
an over-long registry value fails the whole check-in transaction, so one machine
with a malformed Uninstall entry goes silent entirely. SQLite does not enforce
lengths, so the suite would never have shown it.

update_app's version: branch pinned AppVersion.sha256, but that route resolves the
version to its APPLICATION and serves application.installpath - so the digest
described an artifact the route never sends, and where it was null the agent had
nothing to verify and failed open. It now pins the file actually served.

The same branch ended the delivery URL at the id, so an MSI arrived with no
extension and could not be routed through msiexec. It carries the filename now.

Agent packaging moved off require_role('admin') to a new endpoints.agentpackage
permission. The generated ZIP carries a live collector key, so this is a heavier
grant than deploying to one endpoint - and a role cannot be given to one person
for one job the way a permission can.

Full suite 2231 passed (2198 before, plus the 22 that never ran and 11 new),
frontend 314.
2026-08-30 17:17:13 -04:00
benjamin.baker
1fa005eb8a Green up
Tracking docs/openapi.json (so the app can serve the spec at runtime) makes docs/ exist on the published GitHub mirror, which defeated the skipif(not DOCS.is_dir()) guard in the docs publish/version tests: they then ran against a prose-less checkout and failed on CI. Key the skip off docs/*.md prose instead, matching the canonical published-checkout definition in scripts/check-gates-ran.sh.

Also: the OpenAPI generator emitted a trailing-slash path (/api/docs/), which a strict proxy like IIS treats as a redirecting resource a subpath-mounted client cannot follow; normalise trailing slashes in _norm_path and correct the committed spec. And allow docx (python-docx) as an undeclared import: it is used only by scripts/docs/generate_docx.py, a developer tool that never runs on a deployed site.
2026-08-30 15:19:35 -04:00
cproudlock
0baed4cc80 Close the gaps against the design: critical tile, filters, policy card, action placement
All checks were successful
CI / naming (push) Successful in 3s
CI / frontend (push) Successful in 13s
CI / migrations-mysql (push) Successful in 42s
CI / backend (push) Successful in 4m20s
Four things the mockup showed and the first cut did not have.

FLEET
  - A 'with critical events' tile, from the stability route that existed and
    nothing called. Clicking it narrows to rows whose health is not ok - client
    side, because the health rollup is assembled per row after pagination, so
    it narrows the PAGE rather than the fleet. Worth knowing before the count
    is read as fleet-wide.
  - PC type and Location filters. The list route already accepted both; there
    was simply no control. Both use listAll, not list: perpage is clamped to
    100 server-side, so a long location list would lose its tail in a dropdown
    while still looking complete.
  - The machine cell shows number and name together where they differ, so
    '3105 Okuma' rather than a bare number.

PC DETAIL
  - A Policy card, contributed by the computers plugin. It matters because the
    effective policy already narrows the command allowlist, so without it the
    reason a control is unavailable on a given PC is written down nowhere.
    Provenance, so it sits in the reference column, not the triage band.

    Its route returns an EMPTY field list rather than the feature-gate 404 when
    policies are disabled. The panel contract hides a card with no fields, so
    it disappears on sites that have not enabled the feature - a 404 would have
    rendered a broken card on most of them. That was flagged in the design
    review as a snag; this is the fix.

  - Refresh Status and Restart Computer moved out of the page header into the
    triage band. They act on the MACHINE; Print Label, Edit, Retire and Back to
    List act on the RECORD, and mixing them put the most consequential button
    on the page - it halts whatever machine the PC serves - between printing a
    sticker and editing a form. In the band it sits beside the liveness chip
    and the machine it would take down.

    The band takes them through an 'actions' slot rather than knowing about PC
    verbs, so a measuring tool can put 'Calibrate' there without the core
    component learning anything about computers.

Full suite 2124 passed, vitest 260 passed, gates green, spec at 333 paths.
2026-08-28 23:00:25 -04:00
cproudlock
414a7243d7 Merge events and commands into one Activity feed, and lead with live state
All checks were successful
CI / naming (push) Successful in 3s
CI / frontend (push) Successful in 13s
CI / migrations-mysql (push) Successful in 42s
CI / backend (push) Successful in 4m9s
Two changes to the PC page, both about what a responder meets first.

LIVE STATE FIRST. The contributed panels moved to the head of the card flow.
The grid is a two-column FLOW (column-count, with display:contents on the
column wrappers), so a card cannot be pinned to a side and only document order
decides where it lands - putting the panels first fills the left column with
endpoint status, services, processes, interfaces and activity, and leaves
identity, hardware, network and location to the right. Panels still self-order
among themselves by declared position and still self-hide when no plugin
answers for the asset type.

ONE ACTIVITY FEED replaces the Recent events card. Events and commands answer
the same question - what happened to this box - and reading them as two lists
meant interleaving by eye. GET /api/endpoints/asset/<id>/activity merges them
newest-first, capped at 15. Both are keyed by assetid, so it is two indexed
reads and a merge sort rather than a join, and it renders through the existing
ADR-010 list renderer with no bespoke frontend work. A failed command is marked
critical, because that is the row somebody needs to notice. Not collapsed by
default, unlike the card it replaces.

The core AuditLog is deliberately NOT a third source: its rows carry the action
as the entity name and the asset only inside a details string, so including it
would mean parsing prose to find out which machine a row belongs to.

Requester names are withheld from callers who cannot themselves issue commands,
via caller_holds. A read-only viewer - a guest, on a site that grants this
panel - gets the history without a roster of who restarted what, and the lookup
is skipped entirely rather than fetched and discarded. requestedby holds a user
ID, so names resolve through one batched query, not one per row.

Verified on dev telemetry: a queued command shows as 'Collect now / Pending by
admin' above the Windows events, ordered by time.

Full suite 2124 passed, vitest 260 passed, naming/contract/parity gates green,
spec regenerated at 332 paths.
2026-08-28 22:34:59 -04:00
cproudlock
af42ba3589 Register the endpoints list without a trailing slash, and stop hiding its errors
All checks were successful
CI / naming (push) Successful in 3s
CI / frontend (push) Successful in 13s
CI / migrations-mysql (push) Successful in 40s
CI / backend (push) Successful in 4m7s
An endpoint reported telemetry correctly for hours while the Fleet page showed
"No endpoints found". The PC detail page displayed its panels the whole time, so
ingest was plainly working; only the list was empty.

The list route was the one route in 331 paths registered with a trailing slash:

    @endpoints_bp.route('/', methods=['GET'])      ->  /api/endpoints/

The browser client calls it without one, so Flask answered a 308 redirect to an
ABSOLUTE url. On dev that resolves and nobody notices. Behind the IIS subpath it
cannot: with no ProxyFix, Werkzeug builds that Location from the scheme and host
it believes it has, so it emitted http:// and dropped the /shopdb prefix. A
browser will not follow an https page to http, the request failed, and the
page's catch block rendered the empty state.

Every other route in the plugin worked throughout, because none of them carries
a trailing slash - the detail panels call /endpoints/asset/<id>/status and were
never redirected. Detail works, list empty is the signature of this bug.

Registered as '' now, so the rule is /api/endpoints. That is what
CONTRIBUTING.md's API section spells and what every other list route in the
product already did. The inventory and the generated spec are updated to match.

Also here:

  - tests/test_core/test_no_redirecting_routes.py fails the build if any /api
    rule is registered with a trailing slash and strict slashes. Verified by
    planting the violation: it named the offending route, then passed once
    reverted. It takes the shared app fixture rather than building its own
    application - doing that rebinds SQLAlchemy and leaves the NEXT module
    without tables, which showed up as three unrelated "no such table: settings"
    failures visible only in a full run.

  - The Fleet page now renders "Could not load endpoints (HTTP nnn). This is a
    failed request, not an empty fleet." A list that reports a 403 and an empty
    fleet identically is what made this take as long as it did.

  - TRUSTED_PROXY_COUNT (default 0) applies ProxyFix when set. Off by default on
    purpose: it makes X-Forwarded-* trusted, and a client can send those itself,
    while the login rate limit and account lockout are keyed on the client
    address. Set it to 1 on an instance behind IIS.

Full suite 2124 passed, vitest 260 passed, naming and docs gates green.
2026-08-28 17:48:06 -04:00
cproudlock
784964c907 Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.

What arrives:

  endpoints plugin  - a ShopDB agent reporting endpoint state, telemetry,
                      services and interfaces, with a command channel the agent
                      long-polls. Ingest stays on the generic collector route
                      (ADR-006); the agent key is a UI-managed single-scope
                      token whose revocation is an instant kill switch.
  computers policies - Configuration and Business Policies (RMM-style), scope
                      trees validated at write time, actions checked against
                      the endpoints command allowlist. Feature-gated: every
                      route 404s while the setting is off, so enabling it can
                      never half-expose the surface.
  asset CSV import  - templates, a collector script carrying the site path, and
                      a validate-then-commit import. A plugin can own a virtual
                      column plus a post-write row hook, which is how a PC's
                      machine number becomes a controls link without a
                      hand-built relationships sheet.
  agent/            - the Windows agent and its installer.

Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.

MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).

Review fixes carried on top:

  - six policy GETs were jwt_required(optional=True), which authenticates a
    caller without checking anything; they require computers.view now, and six
    redundant jwt_required decorators are gone
  - the agent installer took a default BaseUrl and CollectorKey, so a mistyped
    run silently pointed an endpoint at the wrong instance; both are mandatory
    and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
  - a committing CSV import wrote no audit row

Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.

Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
2026-08-28 16:17:17 -04:00
cproudlock
f59aca5a1b USB check-in / check-out audit, as a report rather than a paged list
CMMC removable-media tracking is an audit obligation, and answering "who had
this drive, and when" meant paging through the check-in/out list by hand.
GET /api/usb/reports/audit returns the events - a take and a return are separate
rows, because a count cannot answer the question an audit actually asks - with
summary figures riding along for the people who only want the shape of it, and
a CSV for the people who want it in a spreadsheet.

The window is named rather than a day count: today, last month, six months, a
year, all time. Nobody should have to work out that a year is 365.

Both directory modes answer in one voice. The external path reads the
cmmc_usb checkinoutlog; self-hosted expands one usbcheckouts row into its two
ends, so a returned drive is distinguishable from one still out, and the window
filter runs in SQL against either end of the loan - a drive taken before the
window and returned inside it is a check-in the window has to show. Rows come
back from the mode-specific half; the CSV, the summary and the response shape
are built once in the route.

Ordering is total in both modes. Timestamp alone ties for two events in the same
second, and the same query then returns them in a different order each run,
which reads as the log being unstable.
2026-08-27 10:20:21 -04:00
cproudlock
95ad4f1ef3 Docs: the USB kiosk, its endpoints and why the list costs a badge
Some checks failed
CI / naming (push) Successful in 1m7s
CI / frontend (push) Successful in 1m7s
CI / migrations-mysql (push) Failing after 3m22s
CI / backend (push) Has been cancelled
Changelog, the plugin table, the three kiosk endpoints in the API inventory,
and regenerated openapi.json / PROJECT-MAP.
2026-08-27 08:14:02 -04:00
cproudlock
90c3c3102b Docs: guides in the indexes, the stalest-kind rule, and the changelog
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / migrations-mysql (push) Has been cancelled
Guides was shipped without an entry in PLUGINS.md or START-HERE, so a site
adopting the product had no way to find out it exists.

BACKUP-KINDS.md described a host's badge as coming from the most recently
confirmed kind. It has not worked that way since the fleet report was reworked:
the freshest revision per (host, kind) is taken, and then the STALEST of those
kinds decides, because reporting the freshest would let a healthy NTLARS backup
mask a part-marker capture that stopped a month ago. Stalest among kinds that
can be JUDGED - a kind never confirmed is unmeasurable, not stale, and treating
it as the worst value made a host with a working backup show no badge at all.

The backups row also still advertised UDC and CMM kinds that do not exist.

api-inventory, openapi.json and PROJECT-MAP regenerated.
2026-08-26 15:06:20 -04:00
cproudlock
d4b49b91b4 Move secure send into Tech Tools, where somebody would look for it
Some checks failed
CI / backend (push) Failing after 7m10s
CI / naming (push) Failing after 7m14s
CI / frontend (push) Failing after 7m10s
CI / migrations-mysql (push) Failing after 7m10s
9845867 gave it its own plugin because Tech Tools is browser-only and this needs
storage. That was the right reading of the code and the wrong answer for the
person using it: Tech Tools is where you go for a utility, and a tool nobody can
find is a tool nobody uses.

So Tech Tools now has a blueprint, a table and a migration chain where it had
none. Its docstring and manifest say why, because "every tool here runs in the
browser" was load-bearing documentation and is no longer true.

WHAT MOVED
  /api/securesend       -> /api/tools/secrets
  securesend.create     -> tools.secrets
  securesend0001baseline -> tools0001securesend
  PLUGIN_TABLE_OWNERS, EXPECTED_HEAD_REVISION, EXEMPT_ENDPOINTS, the universal
  profile and the api-inventory surface all follow it.

THE OLD "NO BACKEND" TEST IS NARROWED, NOT DELETED. It asserted
get_blueprint() is None, which is the sort of guard that gets removed when it
becomes inconvenient. It now asserts the blueprint serves EXACTLY the three
secret routes, with their methods - so a future tool that quietly grows an
endpoint or a table has to edit this test, and therefore make the decision on
purpose rather than by drift.

Asserting methods rather than paths also pins the detail this design turns on:
redeeming is POST-ONLY. Chat and mail clients prefetch GET urls to build
previews, and a preview that could redeem would consume a one-view secret before
its recipient opened the link. Previously that was a comment; now it fails a
test.

UNINSTALLING THE OLD PLUGIN COULD NOT BE DONE WITH THE PLUGIN TOOLING. `flask
plugin uninstall securesend` loads a plugin before uninstalling it, so a plugin
whose directory has been deleted cannot be removed by the command that manages
plugins - the registry entry had to be edited by hand. Worth fixing separately;
it is the same registry-versus-code divergence the device work hit from the
other side.

MIGRATING A CHAIN THAT HAS ALREADY RUN needs the old table and its version row
dropped first, because the relocated chain has no stamp and tries to create a
table that exists. Only relevant to anyone who installed securesend during the
hour it existed - it did reach the publication repo in 4242028 - and the fix is
DROP TABLE securesendsecrets plus DROP TABLE alembic_version_securesend before
running plugin upgrade-all.

1829 tests green. Still backend only: there are no pages yet, so nothing here is
reachable by a person.
2026-08-21 14:17:57 -04:00
cproudlock
98458672eb Hand somebody a password without leaving it in a chat window
Some checks failed
CI / backend (push) Failing after 7m16s
CI / naming (push) Failing after 7m5s
CI / frontend (push) Failing after 7m18s
CI / migrations-mysql (push) Failing after 7m10s
The habit this replaces is pasting a credential into Teams, where it stays until
someone deletes the conversation. Tabled on 2026-08-21 pending a decision about
scope; the decision was text only, so this is the half that survives it.

BACKEND ONLY. There is no page yet, so nothing here is reachable by a person.

TEXT, NOT DOCUMENTS, and that shapes everything. Dropping files removed the
export-control question entirely - a password is not technical data - along with
blob storage, size and MIME handling, and the need to authenticate the recipient
so a document fetch could be pinned to a name.

TWO DOORS. Creating needs a login AND securesend.create: anonymous creation
would be an untraceable way to move data out of a system that otherwise records
who did what, and the creator is the only person these rows can ever name.
Reading needs neither, because the recipient usually has no account here and the
link is handed over on a call - the unguessable token IS the authorisation,
which is why it is 128 bits of randomness and why the row is spent on first read.

REDEEMING IS A POST, and that is the detail that would have broken this in
practice. Teams, Outlook and Safe Links prefetch GET urls to build previews and
scan for malware, so a GET that redeemed would let a preview window consume a
secret before its recipient ever saw it. The GET route that remains reports only
whether a link is still live, so the read page can say "opening this will
destroy it" without spending the only view to find out.

THE SIZE LIMIT IS THE WHOLE ENFORCEMENT. The key lives in the URL fragment,
which browsers never transmit, so the server holds ciphertext it cannot read and
cannot tell a password from a page of a drawing. MAX_CIPHERTEXT is therefore not
a tuning knob: raising it quietly turns this into the document transfer it was
scoped not to be. The refusal says what the limit is FOR, because somebody
hitting it is usually trying to send a file and the answer is a different tool.
Line counts, file signatures and refusing a pasted file belong in the browser,
guide an honest person, and are bypassable by anyone willing to POST directly.

NOT ENUMERABLE. Missing, expired and spent all answer identically - telling them
apart would confirm to anyone holding a guessed token whether it ever named
anything. A spent or expired row is deleted on the way past rather than waiting
for a sweeper, because it has no reason to keep its ciphertext.

Its own plugin rather than a Tech Tool: that plugin is browser-only by design,
and this needs storage, expiry and a view counter. Not an asset extension
either, so no assetid and no asset type - the scaffold assumes one and it was
removed.

FOUND BY ITS OWN TESTS: expiryhours=0 returned 201. `int(x or DEFAULT)` treats
an explicit zero as absent, so "expire immediately" silently became "expire in a
day". A caller who sends a number is now told when it is not allowed rather than
quietly given a different one.

BEFORE ANYONE RELIES ON THIS, test that your link rewriter preserves URL
fragments: create a secret, send the link to yourself in Teams, click it from the
message, and check the key is still after the #. If Safe Links strips it the
secret is unrecoverable and the server cannot help, which is the point of the
design. The fallback is passphrase mode, worth building only if that test says so.

Three product gates caught what a new plugin has to declare, and each was right
to: the route-parity gate wanted the endpoints described in api-inventory.json,
the lean-build guard wanted the plugin listed in the universal profile the
released installer builds from, and the authz sweep wanted the public redeem
route named in EXEMPT_ENDPOINTS with a reason. That last one matters most - a
mutating route that answers an unauthenticated caller is a decision somebody has
to write down, not a default, and reading a secret is a mutation because it
spends it.
2026-08-21 13:44:13 -04:00
cproudlock
5001aedcf9 Offer a spelling on every list, not just the global search
Some checks failed
CI / backend (push) Failing after 7m15s
CI / frontend (push) Has been cancelled
CI / migrations-mysql (push) Has been cancelled
CI / naming (push) Has been cancelled
Correction only helped on the global search page, and people search from the
list they are already on. Extending it turned out to be a shape question rather
than a volume one: thirty routes take a search parameter across a dozen files,
so carrying a suggestion in each of their responses is a large change today and
one more thing every future plugin author has to remember.

So the suggestion moved to ITS OWN ENDPOINT, /api/search/suggest, which any page
can call after rendering no rows. A page that has not adopted it shows nothing,
which is exactly what it showed before - nothing breaks by omission.

The plumbing lives in useListQuery, which already owned the search term, so a
list needs three lines: take `suggestion` and `reportCount` from the composable,
call reportCount(rows.length) after a fetch, and drop SearchSuggestion into the
empty state it already has. A list that never calls reportCount never offers a
suggestion.

Wired: global search, machines, printers, PCs, network devices, measuring tools,
knowledge base, vendors. NOT wired, deliberately: the type and reference lists
(machine types, PC types, VLANs, subnets, operating systems and the rest), which
are small controlled vocabularies nobody typo-searches, and USB, whose empty
state has a different shape and wants doing by hand rather than by pattern.

TRAP FOUND WHILE WIRING IT, and left commented in every page: applying a
suggestion by calling setSearch alone updates the box and the URL and does NOT
reload the list. setSearch only syncs the URL, and the watcher that would reload
is suppressed because search.value already holds the new term - the same trap
the global search page documents in performSearch. Each page calls its own load
function directly.

The composable guards a stale answer arriving after a newer search was typed,
never offers back the word that was typed, and swallows its own errors: a search
that found nothing is already the answer, and failing to improve on it is not
worth an error in front of anyone.

The route-parity gate caught the new endpoint being served without an entry in
docs/api-inventory.json, which is hand-written on purpose; added, and the spec
regenerated from it (283 paths, 418 operations). That regeneration also carries
the openapi version to 0.12.0, left over from the release.
2026-08-21 11:14:56 -04:00
cproudlock
2d09fa3201 Collect what bays actually have, separately from what they are told to have
Some checks failed
CI / backend (push) Failing after 7m15s
CI / naming (push) Failing after 7m22s
CI / frontend (push) Failing after 7m14s
CI / migrations-mysql (push) Failing after 7m14s
ShopDB knew what a bay SHOULD have and nothing about what it DOES. Adding the
observed half makes a rollout a review instead of a typing exercise: the floor
reports itself in, you look, and you adopt.

The collection uses the mechanism that already exists rather than a new one.
POST /api/collector/printers dispatches to the printers plugin's
apply_collector_payload, the same ADR-006 hook the computers and backups plugins
implement. New client script, new plugin-owned table, no new transport and no new
credential.

OBSERVED AND ASSIGNED STAY APART, and that is the point rather than a detail. A
collector report can never write an assignment row: _reconcile_edges is the only
function that writes usesprinter/defaultprinter, it has two call sites, and both
are authenticated routes a human calls. If a drifted bay's own state were allowed
to become what it is told to install, every configuration error would become
permanent the next time that PC checked in.

Seeding an assignment from observed state is explicit -
POST /assignments/seed-from-observed - because a rollout adopts many machines at
once. It routes through the same _reconcile_edges as the editor, so there is one
write path with two doors, and a queue matching no known printer is REFUSED
rather than guessed into an assignment. That last rule is the lesson from the
measuring tools: adopting on a weak key produced 43 duplicate instruments.

Two fixes on top of what the agents built. The replace deleted a host's previous
rows by exact case-folded name while the read path treats a short name and its
FQDN as one machine, so a PC that changed spelling appeared to hold every queue
twice - which reads as drift that is not there. And the client sent 'reportedat'
where the declared schema said 'observedat'.

Also here: the legacy loader now imports machines.printerid, the classic system's
record of each machine's default printer, which it silently dropped - the
production import would have lost every one. And Set-ShopdbPrinters.ps1 finally
registers the per-user logon task, staging Apply-ShopdbDefaultPrinter.ps1 to
C:\ProgramData first because the share it lives on is mounted only during the
enforcement cycle and the task runs at logon when it is gone.

VALIDATED ON WINDOWS 11 (build 26200), not just on Linux pwsh, which parses these
scripts happily and executes none of the spooler branches.

The reporter: posts a correct payload with the X-API-Key header; resolves BaseUrl
and CollectorKey from HKLM when given no arguments; suppresses the virtual queues
by port; resolves port addresses; and reads the CONSOLE USER's default out of
HKU rather than SYSTEM's own, which is a different and usually wrong answer.

Two results matter more than the rest. With the spooler stopped, both the cmdlet
and the CIM path fail and the script posts NOTHING - verified against a capture
server that recorded zero requests, where an empty list would instead have
erased that host's observed rows and read as a bay that lost its printers. A
genuinely empty host still posts [], because that is a real and different fact.

The logon task registers as the Users group at Limited, and falls back to the
well-known SID S-1-5-32-545 when the group name will not resolve, as it will not
on localised Windows. It was then run with the source directory RENAMED AWAY, to
stand in for the share being unmounted, and it still moved the user's default -
which is the whole reason the script is staged to C:\ProgramData rather than run
from where it lives.

The guarantees against damage were re-checked rather than assumed: an empty
assignment changes nothing, an unreachable server changes nothing, -WhatIfOnly
leaves no queue, no task, no staged file and no registry value behind, and a
drifted queue is repointed IN PLACE with Set-Printer so whoever has it as their
default keeps it.

Not covered by any of this: the driver-staging path, which needs a real vendor
package rather than the class drivers a VM ships with.
2026-08-19 15:32:18 -04:00
cproudlock
e2c45d33bc One printer picker for machines and PCs, and one default per asset
The assignment belongs to the MACHINE, and until now there was no way to set it
except the generic relationships card or the API - the form for the thing the
feature is about did not exist. MachineForm now carries the picker, and PCForm
uses the SAME component rather than its own copy: the PC's set overrides the
machine's, and two implementations of that would drift, with the two ends of an
override disagreeing being exactly the bug nobody would spot.

The shared picker also fixes what PCForm did on save. It wrote row at a time
through the generic relationship endpoints, which is a non-atomic reconcile: an
HTTP failure part way left a PC half-assigned with nothing recording what was
meant. It now calls the reconcile endpoint, which validates the default before
writing anything.

A relationship type can now say it allows one active row per asset
(relationshiptypes.issingular, migration 7d34), and defaultprinter says it.
Cardinality belongs to the type rather than the printers plugin: core's create
path is where every hand-made link passes, and the next type meaning "exactly
one" gets the rule for free. Setting a second default REPLACES the first instead
of refusing, because "make this the default" means that - and a card answering
409 would leave the user hunting for the old row.

Without it the schema was happy to hold two defaults: the unique constraint is
(source, target, type), so two different targets are two valid rows, and the
resolver takes the OLDEST - the new default silently lost. Proven by disabling
the new rule and watching the tests fail.

FOUND WHILE TESTING IN A BROWSER, and it was not mine: MachineForm read
.data.data off computersApi.listAll(), which resolves to the ARRAY - fetchAllPages
has already unwrapped every page. The whole parallel load threw into the catch,
so every dropdown on the machine edit form came up empty and the machine's own
values never loaded. A build cannot see this; only opening the page can.

GET /api/printers/assignments/for-asset/<id> returns an asset's OWN assignment,
without inheritance, because the editor must show what this asset's rows say -
otherwise a machine's printers appear ticked on the PC that inherits them and
unticking one silently creates an override.
2026-08-19 11:22:48 -04:00
cproudlock
0dc0ac13c8 Assign printers to a machine, and let the PC that drives it inherit them
Printers belong to the bay, not to the box currently driving it. The assignment
goes on the MACHINE asset and reaches whichever PC controls it, so a reimaged or
swapped PC comes back with the right printers and nothing had to be saved off the
old one. The asset register is the backup.

New relationship type usesprinter ("this printer is installed here"), beside the
existing defaultprinter ("which of them is the default"), both seeded and both
given a propagation rail through controls. The rails are consumed at READ time
only: the create-time fan-out skips directional through-types, and controls is
directional, so assigning a printer to a machine does not copy rows onto its PC.
That is what keeps own-beats-inherited possible.

Resolution for a PC is its OWN rows if it has any, otherwise one hop out along
controls to the machines it drives. Whole set at a time, not merged: a PC with
its own assignment is overriding the bay deliberately, and the UI has to say so
or a tech "fixing" a bay by editing the PC will shadow the machine's record and
wonder why they keep disagreeing.

GET /api/printers/for-host/<hostname> is what the convergence client asks every
cycle. Resolved by hostname because the collector upserts PCs by hostname and an
office PC has no machine number. An unknown host, a site without the computers
plugin, and nothing assigned all return an empty set - that is the client's
designed no-op and it must stay indistinguishable from "assigned nothing".

PUT /api/printers/assignments/for-asset/<id> reconciles the whole set in one
call. The endpoint was specified, documented and asserted by three tests, and
never written - the verification pass caught that, with four failures. It
validates the default BEFORE any write, so a rejected request changes nothing;
soft-deletes rows that went away; and REACTIVATES soft-deleted rows rather than
inserting, because the unique constraint spans inactive rows and a blind insert
after an unassign raises IntegrityError on MySQL while passing on SQLite.

One default per asset, enforced here because the schema cannot: the constraint is
(source, target, type), which accepts two different defaults quite happily. Two
active defaults are still reachable through the generic relationships endpoint,
where the oldest silently wins - recorded in the proposal as the next thing to
close.

printerdrivers gains drivername: the exact string the INF declares, which
Add-PrinterDriver matches on and nothing else. Deriving it by parsing INFs on
hundreds of bays is fragile; a human confirming it once is not.
2026-08-19 09:33:22 -04:00
cproudlock
2083029ff2 Generate the collector script per site, and bring EventSaver into the repo
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 3s
CI / frontend (push) Successful in 10s
CI / migrations-mysql (push) Failing after 7s
A site adopting ShopDB had to be handed two files and told what to edit in them.
Both are now the product's, and one of them the server writes for you.

GET /api/computers/client-script (admin) returns Report-AssetToShopDB.ps1 with
this site's values already in it: site_base_url becomes the -ApiUrl default and
the new computers_routableranges setting becomes -AllowedRanges. Only the
PARAMETER DEFAULTS are substituted - the copy in plugins/computers/client/ stays
runnable, so there is no second version to drift from the first - and everything
stamped stays overridable by argument or registry, because a bay may need to
differ from its site. Settings > Computers > Asset reporter edits the ranges,
downloads the script and shows its SHA-256.

The collector key is deliberately not stamped in, and a test fails if it ever
is. That file lands on every shop-floor PC, and a token spread across hundreds
of bays cannot be rotated quietly; it stays in the registry, provisioned per
ADOPTING-AT-ANOTHER-SITE.md.

The routable ranges are the last thing that was hardcoded in that script. They
are now a setting, so West Jefferson's two CIDRs move out of source code and
into that site's own configuration - which is what ADR-015 asks for - and a site
that sets nothing still works, because the script falls back to the NIC carrying
the default route.

EventSaver joins it in plugins/slides/client/, source only: EventSaver.cs and
EventSaver.ini, no compiled .scr - a binary is a release asset, like the
installer exe. The share path that was compiled into Config.Folder is gone. It
used to be the fallback when the ini was missing, which silently pointed a new
site at the reference site's file server; it is now empty, and failing visibly
beats displaying another site's slides. Verified by compiling the edited source
in the Windows VM with the in-box csc.exe: 15,872 bytes, exit 0.

Also: the DSC example in the adoption guide gains a CollectorRanges resource and
stops passing -ApiUrl to a script that already reads BaseUrl from the registry
the same example writes, and the guide points at the generated download instead
of hand-editing a URL.

The contract test caught the endpoint importing shopdb directly for the version
string, which ADR-002 forbids a plugin from doing. The product and contract
versions are in app.config now, which a plugin reads through current_app.

Adds docs/proposals/printer-assignment.md: assign printers to a PC in ShopDB and
let the bay install them, with what the fleet data says about drivers - HP and
Xerox cover 41 of 44 printers with universal drivers, there are no Brother
printers at all despite 208 files of Brother inkjet drivers in the installer,
and printerdrivers holds one row pointing at a per-model folder instead of a
universal driver.
2026-08-18 15:51:14 -04:00
cproudlock
aede5ac621 Match the optional identifiers in every list's search box
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
gaugelabreference and maintenancereference are collected on the form and shown
on the detail page for all five asset types, and no list's ?search= matched
either. Someone holding a physical tag could read it off the machine, type it
into the list they were already looking at, and get nothing back - while global
search, fixed in the previous commit, found it.

The clause is copy-pasted once per plugin, which is how all five came to omit
fields their own forms collect, so the test is one parametrized pass over all
five lists rather than five per-plugin tests that would drift the same way. It
also pins that widening the clause did not turn the box into a pass-through.

The api-inventory entries enumerate the fields each search matches, so all five
were stale the moment the queries changed; updated with the OpenAPI spec.

Not touched: the collector does not send either identifier, correctly - a
lab-assigned tag is not something an agent on a PC can discover. The legacy
import loader does not map them either, and the classic schema has no
gauge-named column to map from.
2026-08-17 13:16:22 -04:00
cproudlock
3324dbd91e Buildings and levels for the floor map, and make every identifier searchable
Some checks failed
CI / backend (push) Failing after 9s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 10s
CI / migrations-mysql (push) Failing after 7s
The map was one picture of one floor. A second floor was added, the blueprint
changed size, and machines moved, so a position now records WHICH DRAWING its
coordinates belong to.

Buildings and levels (ADR-017). Each level owns its blueprint per theme and its
own native pixel size; assets.mapx/mapy are pixels of assets.levelid, not of the
site. A position whose level is unknown renders "level unknown" and is never
drawn on the default level, because a marker on the wrong floor plan looks
entirely correct while pointing at the wrong place.

Repositioning in bulk: filter by unplaced, needs-review or level, search, place,
confirm. Landmark recalibration solves the transform PER AXIS from landmark
pairs and never from image dimensions - the canvas grew taller without
rescaling, so a dimension-derived scale would stretch Y by 1.57 and be wrong
everywhere. It defaults to a dry run, reports what would land off the drawing,
snapshots before applying, and clears mapverifiedat because a transform is a
guess awaiting review. Snapshots restore, including the level and the review
state, and a restore snapshots first so an undo is undoable.

Search: gaugelabreference was matched only for measuring tools and
maintenancereference was matched nowhere at all, for any asset type, while
Settings happily offers both identifiers on machines and PCs. A tag an operator
is told to record has to be findable or it is a write-only field. USB devices
and printed items were unreachable from search entirely - neither is an asset,
so the generic asset search could not see them and no searcher existed; they
now match on serial, asset tag, label, bin code and gage-lab tag, honouring
isactive, with Settings toggles and result labels to match.

The retired-application rule was half a rule: GET /api/knowledgebase hid
articles whose topic application is retired while global search still returned
them and printed the retired application as the subject. A filter is only real
if every path that reaches the row applies it.

Contract to 0.20.0 (additive): Asset gained levelid and mapverifiedat, Location
gained levelid, and resolve_asset_position returns the levelid belonging to
whichever source supplied the coordinates. The five plugins that write a map
position are re-pinned. The install-list text format gained levelid as a NINTH
field, appended, because the shipped Pascal installer reads fields 0-7 by index.

That installer still compiles in one drawing's dimensions and bundles one
blueprint, so its map is accurate for the default level only; /api/maplevels is
deliberately unauthenticated so it can read both at runtime once rebuilt.
Recorded in PRINTER-INSTALLER.md section 6 along with the other known gaps.

Migration 7d33 converts an existing single-map site into one building and one
default level carrying the old map_* settings, then assigns every placed asset
and location to it. Nothing moves on screen. Old settings rows are kept so a
rollback still finds them. Verified end to end on MySQL 5.6 from a
production-shaped database.
2026-08-17 12:55:51 -04:00
cproudlock
ad335cfc9e api: document the twenty-two routes that were serving traffic in silence
The inventory is hand-maintained, and should stay that way - its value is the
prose, and what an endpoint is FOR cannot be derived from the code. An audit of
all 372 documented operations found zero phantom routes and zero wrong parameter
names, so the maintenance is in good order.

What hand-maintenance cannot do is notice a route somebody added. Twenty-two
were undocumented: the entire backups plugin surface, every one of the dashboard
card endpoints added with contract 0.19.0, the GE-Enforce publish preflight, the
employee SSO resolver, the protocol update verbs, and the four /api/docs routes -
so the spec did not describe how to fetch the spec.

Coverage is now a test. It walks the live url_map and fails when a served route
has no entry, which means adding an endpoint includes describing it, in the same
commit, while the author still knows what it is for. The reverse direction is
checked too: a documented route that no longer exists sends a reader to a 404.

Writing that test found one more thing. The inventory writes multi-verb routes
as "PUT|PATCH", and neither the parity check nor the generator split on the pipe
- so those operations were absent from the published spec entirely, with nothing
reporting it. The spec now carries all 394 operations the code serves, which is
the first time the two numbers have matched.

The generator's own docstring claimed the inventory could be regenerated. It
cannot; nothing generates it. That sentence is why nobody noticed it was falling
behind.
2026-08-14 16:09:47 -04:00
cproudlock
36b0265668 openapi: emit a spec a machine can actually use
The generated spec carried no `parameters` and no `requestBody` on any of its
372 operations. That is invalid OpenAPI 3.1, and the practical cost was worse
than the formal one: the MCP server builds its tools from this file, so every
tool had an empty input schema and silently dropped whatever the caller passed.
A request for one asset returned the list, and nothing anywhere reported an
error. All 118 templated paths now declare their path parameters, typed from the
Flask converter that named them, and write verbs declare a JSON body.

The body is an open object carrying the prose description rather than an
invented schema. The inventory describes bodies in sentences, and a field list
this generator guessed at would be worse than none - but "an object, described
here" is the difference between a client that can send a body and one that
cannot send anything.

Security was wrong on 123 operations. `jwt-optional` means "works logged out,
returns more logged in", which OpenAPI expresses as the empty requirement
alongside the scheme; publishing them as bearer-required told every reader that
a public endpoint needs a token.

Responses were one hardcoded 200, so a generated client had no idea a call could
fail. Every operation now documents the error envelope - and the envelope itself
is a defined schema, because its error nests under `data.error` rather than at
the top level, which is the single thing people get wrong when writing against
this API.

95 summaries were cut at 120 characters mid-word, which is what a tool picker
shows a user as the whole description of a call. They now end on a word.

Tests pin the shape rather than the prose. One of them contradicted an older
test that REQUIRED the contract version as a literal in PLUGIN-HOOKS.md - the
same copying that left nine documents stale - so that test now asserts the doc
points at the generated map instead.
2026-08-14 15:56:27 -04:00
cproudlock
05be4c4489 docs: fix the examples that fail if you paste them
Six procedures that could not be followed as written.

Eighty-nine curl examples single-quoted `Authorization: Bearer $TOKEN`, so the
shell never expanded it and the server answered 422 "Not enough segments". Nine
more did the same with X-API-Key. The other 129 examples in the same file
already used double quotes, so this was drift rather than a convention, and the
spec regenerated from it carried the fault onward.

The GE-Enforce report example put a `//` comment inside a JSON body. The server
parses with silent=True, so it saw `{}` and answered "hostname is required"
about a body that plainly has one - the worst kind of error message, one that
sends the reader to the wrong field entirely.

The IIS install ran `flask db upgrade` and a per-plugin install loop but never
`flask plugin upgrade-all`, leaving every plugin's own chain unapplied. That is
precisely the 1054 "Unknown column" a deploy then hits somewhere else, days
later, on the page that uses the new column.

The pilot runbook looped `flask plugin enable` over plugins that were not yet
installed; enable refuses those, so on a fresh database it exited 1 on every
iteration and enabled nothing. ADR-013 had already recorded that defect.
`apply-profile` installs and enables in dependency order, which is what the step
was reaching for.

DEPLOY-WINDOWS-IIS named a profile file that does not exist; the shipped ones do.

And PLUGIN-EXTERNAL-REPO never mentioned PLUGIN_TABLE_OWNERS, while the
migration engine raises for any plugin missing from it - so the guide's own
step 5 fails for any external plugin that owns a table. That the registry lives
in the framework repo is deliberate, so the guide now says so, and says what it
costs: a table-owning external plugin is a two-repository change, and a plugin
that owns no tables avoids it entirely.
2026-08-14 15:43:27 -04:00
cproudlock
4995456136 docs: take one site's name, hosts and paths off the public wiki
The publishability gate caught internal tooling names and developer paths but
nothing site-specific, so roughly sixty leaks reached the wiki: the site name in
ten documents, real fleet hostnames in the collector and GE-Enforce examples, an
internal database name through the whole import guide, imaging-share paths, and
a maintainer's username as the Deciders line of every ADR and inside a generated
curl example.

None of it is a security matter on an air-gapped fleet. It matters because these
pages are read by engineers at other plants, and a document that names one site
throughout reads as that site's notes rather than a product's documentation -
which is exactly what it then gets treated as.

Examples now use neutral hostnames, the site is "the reference site" where the
distinction carries meaning, and ADRs are decided by "ShopDB maintainers". The
gate carries all of these patterns, so the next one fails a build.

Two documents leave docs/ because they were never written for an outside reader.
PROJECT-REVIEW.md is an internal health memo pinned to a commit from July, whose
headline finding (an untracked playbook) has since been fixed - it is history,
and git holds it. PILOT-DEPLOY.md is one site's own cutover runbook, complete
with a "re-measure before publishing" placeholder; it moves next to the loader
it belongs to, in scripts/site_imports/wjf/.

ADR-015 is AMENDED rather than rewritten. Its enforcement section still said
report-only and its backlog still listed hardcodes that are now cleared, which
left the record contradicting itself. The amendment says what changed and why
the report-only period ended; the original text stays, because what the decision
looked like when it was taken is the part worth keeping.

Also corrects llms.txt's response envelope, which had errors at the top level
and pagination at meta.total. Both are nested one deeper, so anything written
against that description read undefined on every error it tried to handle.
2026-08-14 15:38:27 -04:00
cproudlock
035419fa51 ADR-015: stop shipping one site's values, and make the rule a gate
The scanner has been reporting the same count for weeks, which is what a rule
that only prints becomes. It now FAILS the build, and it looks where the leaks
actually were: PowerShell, the installer, the seeds, generated JSON, the
frontend - case-insensitively, across plugins, shopdb, scripts, deploy, tools.
A line that is deliberate declares itself with an ADR-015-OK marker and a
reason, so the claim is visible in review instead of tolerated in silence.

What it found, fixed here:

- The shadow client wrote one site's ShopDB URL into HKLM whenever the registry
  disagreed. At the site it was written for that reads as healing drift;
  anywhere else it overwrites the site's own address on every enforce cycle,
  and the site cannot win because the cycle repeats. The bay's value now wins,
  an explicit -BaseUrl seeds it, and with neither there is nothing honest to
  write, so it says so and skips.
- The kiosk dispatcher fell back to one plant's host when HKLM was unset, so a
  kiosk elsewhere quietly opened a server it has no business reaching. The
  fallback is now this site's site_base_url, baked in at seed time, and the
  dispatcher refuses rather than guessing when neither is set. Its legacy
  shortcut matcher derives the host from that URL instead of naming one.
- The OpenAPI generator hardcoded a production hostname into every spec it
  generated, which then published to a public wiki. The relative mount is the
  only server it can honestly name; a site passes its own by environment.
- Placeholders and examples in the UI and the client help offered real internal
  subnets and a real production URL. They now use documentation ranges.

Both publication gates - the export scrub and the docs publishability test -
carry the site patterns, which neither did. One plant's hostname, FQDN and
internal networks are out of the documentation and the generated specs.

Comments naming the reference site are reworded rather than deleted: the
reasoning is worth keeping, the plant name is not what makes it true.
2026-08-14 13:47:39 -04:00
cproudlock
3d83806135 Make the toner forecast an order, not a table
The report answers a purchasing question, and it was answering it in seven
columns, two tables and a rowspan. What someone actually needs from it is a
short list of what to buy.

So it opens with that list, grouped by part number with a quantity. Two
cartridges of the same part in different printers is a quantity of two, which
is the number an order needs and the one a per-printer table made the reader
count by hand. It covers what is empty plus what goes within a fortnight -
ordering only what is already empty means running empty. There is a copy
button, because it ends up pasted into a mail.

Below it the cartridges sit in urgency bands rather than in one long list
sorted by a number. The question is which pile a thing is in, and a pile that
is empty is worth seeing as empty. Everything past "empty" starts collapsed;
the order list above already covers the same ground in a tenth of the height.

The row is a cartridge now, not a printer, so it can carry its own part number,
its own level bar and its own countdown. Nesting supplies under a printer meant
opening a printer to find out whether anything on it needed doing.

Cartridges with no part mapped are counted on a single line rather than given
one each. They cannot be dropped, since that would quietly shorten the order,
and they cannot be ordered from here either - the job they represent is
mapping them, which is one job however many there are.

Bands and the order horizon are decided server-side, next to the arithmetic
that produces them, so a heading cannot disagree with what got added to the
list.

Checked against a fleet of 43 dev printers with real part mappings, driven by
a stub Zabbix - live Zabbix is not reachable from the dev box.
2026-08-13 13:08:39 -04:00
cproudlock
a64796f060 Document the upload, proof and forecast endpoints
Some checks failed
CI / backend (push) Failing after 9s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 10s
CI / migrations-mysql (push) Failing after 7s
Ten routes shipped over the last few commits without reaching the inventory,
which is the file everything else is generated from - openapi.json, the /api/docs
browser, and the per-site build. An endpoint absent from it is invisible to
anyone integrating against the API even though it answers requests.

The auth notes matter more than the paths. API-REFERENCE tells a deployment
reviewer that essentially every GET is optional-auth; two of these are not, and
an exception buried in a table of 372 operations is one nobody reads. The
installer download and the warranty proof are called out with why: one is
licensed vendor software, the other carries pricing and a service tag. The
application image is listed with the fully public endpoints, since tiles render
before login.
2026-08-12 12:06:25 -04:00
cproudlock
959db2922b Release 0.8.1
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
The installer on the share was stamped 0.8.0 and contained sixteen commits that
were not in the v0.8.0 tag. A build that misreports its own version is exactly
what the version check in docs/RELEASING-WINDOWS.md exists to prevent, and it
would have left two sites unable to say what they were running.

Everything in 0.8.1 landed after v0.8.0 was tagged this morning, driven by two
sites entering real data for the first time: the blank-code 500, the two-slide
display that never rotated, modals discarding a part-filled form, filters
returning an empty page, model photos that could not be saved, and the shared
equipment catalog that lets a new site start with vendors, models and printer
supply part numbers already present.

CHANGELOG gains a 0.8.1 section, and the OpenAPI document follows __version__
rather than being restated.
2026-08-05 14:50:06 -04:00
cproudlock
89e880afc3 Release 0.8.0
Some checks failed
CI / backend (push) Failing after 6s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 6s
The Windows installer has never shipped under a version: v0.7.0 was tagged
before any of it existed, so every build handed out so far stamped a server with
0.7.0. Two servers running different builds were indistinguishable, and the
installer logged each upgrade as "same version already installed" rather than
recording what changed. This cuts the release that fixes that.

0.8.0 rather than a patch: the air-gapped installer is a new capability, and
pre-1.0 semantic versioning puts that in the minor slot (ADR-007).

CHANGELOG gains a 0.8.0 section covering the twelve defects a real Windows
Server 2019 install surfaced, the move from inferring "is this a re-run of my
install?" to recording it, and the operator documentation.

deploy/site-profile-universal.json is now in the repository. Released builds
were being produced from a profile in a temporary directory, so the next release
could not have been reproduced once that file was cleaned up.
docs/RELEASING-WINDOWS.md points at the committed profile and says why.

scripts/gen_openapi.py reads __version__ out of shopdb/__init__.py instead of
restating it. Its hardcoded copy had already drifted a release behind, which is
the same mistake that once shipped an installer stamped with the wrong version.
2026-08-05 07:34:05 -04:00
cproudlock
b507884ad6 api: serve interactive OpenAPI docs at /api/docs (offline) + llms.txt
Generate docs/openapi.json (3.1, 362 operations) from the API inventory via
scripts/gen_openapi.py, and serve it with a self-hosted Redoc bundle at
/api/docs - no CDN, works on the air-gapped box. Also serve docs/llms.txt (a
concise LLM entrypoint) at /api/docs/llms.txt. New core 'docs' blueprint;
staticdocs/ excluded from the naming check (vendored minified JS).
2026-07-30 07:51:12 -04:00