48 Commits

Author	SHA1	Message	Date
cproudlock	a46058a4dd	Close the last three restore-gate gaps, and make the gate test the real gate ... Three defects that all shared a shape: the restore path looked checked and was not. An unreadable data revision was treated as a legacy set. backup.sh discards the mysql error when reading alembic_version, so a set could be written with an empty datarev; the compatibility check then fell through to the code-to-code head comparison - the very check datarev was introduced to replace - and badged the set restorable on no evidence at all. A manifest that CARRIES the key and left it empty is now refused, while one with no key at all is still the genuine legacy case and keeps the fallback. backup.sh records the failure in run history rather than passing it over in a log line, since the set it just wrote cannot be restored. Dumps were bound to the schema they came from. mysqldump --databases bakes CREATE DATABASE and USE <name> into the file, so restoring onto a site whose MYSQL_DATABASE differs loaded every row into the dump's schema, left the live database untouched, and reported success. New sets are dumped without it and load into the schema named at restore time. Existing sets are still bound, so the gate reads which schema they name and refuses a mismatch instead of restoring beside the live data. Also dropped --no-tablespaces into the main dump: it needs the PROCESS privilege the app user does not have on MySQL 8. The gate's own test carried a copy of the gate. test-restore-datarev.sh re-implemented the predicate in its own shell functions, so it would have stayed green through any drift in restore.sh - including the plugin-chain hole fixed in the previous commit, which it could never have caught. restore.sh now takes RESTORE_CHECK_ONLY, which stops it immediately after the gates having touched nothing, plus overridable paths for the migration and plugin trees. The test drives the shipped script and asserts on its exit code, so the verdict under test IS the verdict that ships. Confirmed non-vacuous by planting a violation: disabling the plugin revision check turns the suite red. Nine cases now, up from five, covering both plugin-chain directions, a plugin the build does not have, and the schema binding. Moving the schema check earlier was prompted by that test: it had been placed after the safety copy, so it needed a live database to reach a verdict it can reach for free. It sits with the other gates now and stops before anything is written. Round-tripped against a real stack after the dump format change: backup, mutate, restore, data back to backup-time state, and the new dumps confirmed to carry no schema name. Suite 2607 passed / 9 skipped. Scope: product	2026-09-06 18:31:37 -04:00
cproudlock	e2705d732a	Gate a restore on every schema chain, not just the core one ... The compatibility gate read alembic_version and nothing else, but plugin schemas move independently of core. Roll the code back to an older plugin revision and the core head can be identical, so a dump carrying plugin tables the code has never heard of passed the gate, loaded - destroying what was there - and only then failed in 'flask plugin upgrade-all' with "Can't locate revision". A failed restore that has already replaced what it was restoring over is the worst outcome this feature can produce, and it was reachable from an ordinary rollback. Backups now stamp the applied revision of every alembic_version_<plugin> table alongside the core one, read from the dumped database the same way. restore.sh checks each against that plugin's own versions directory and refuses before touching anything, naming the plugin and the revision to deploy past. A plugin in the dump that this build does not have at all is refused for the same reason. The list endpoint's compatibility badge asks the same question, so the UI cannot offer a set the restore will reject. Sets taken before the stamp existed carry no dataplugins and keep their old core-only verdict. Their plugin revisions are unknowable, and making every existing set unrestorable to close this would be a worse trade than the gap. Proven against a real stack, reproducing the exact case: core head identical, endpoints data stamped at a revision the build does not have. The restore refused, the live row was still there afterwards, and it stopped early enough that the pre-restore safety copy was never needed. The same set with its true revision restored normally. Suite 2605 passed / 9 skipped, with the gate covered both ways plus the legacy fallback. Scope: product	2026-09-06 18:02:39 -04:00
cproudlock	fa6b99bf91	Finish a restore without a browser, from the set that was asked for ... Closes the three defects in the imported restore path, at Ben's request. Each was proven against a real stack before and after: two backup sets with different data and different instance files, then a restore of the OLDER one, which is the case that was broken. A restore no longer depends on a tab staying open. The backup sidecar has no Flask, so it loads the dump and leaves a sentinel for the api container to run the migrations. The only thing that ever acted on that sentinel was the poll loop of the modal that queued the restore, so closing the tab - or letting a token expire - during the minutes a load takes left the data at the old revision under new code, every query on a newer column failing, and nothing retrying. A watcher in the api container now claims and runs it. The endpoint no longer runs the upgrade at all; it only makes sure something is pending, which is also how a failed upgrade is retried. That takes Alembic out of the request path, where it could outrun the gunicorn worker timeout and be killed mid-chain, leaving a half-applied revision the next attempt trips over. The claim is a rename, because rename(2) is atomic: several workers and a browser poll can race the same sentinel and exactly one wins. A failed upgrade parks itself as .upgrade-failed rather than re-arming, since putting it back would rerun the same broken chain on every poll forever. Instance files come from the set being restored. The restic call asked for 'latest' regardless of which set was named, so restoring anything but the newest paired that set's DATABASE with the NEWEST snapshot's uploads and plugins.json - a mismatch nothing downstream could detect. Now scoped by the set's own tag, which backup.sh has always written. The tree is replaced rather than restored over, so files created after the snapshot are gone afterwards: a restore has to put the site back to the snapshot, not merge into it. Done as a staged swap rather than 'restic restore --delete --target /', because that aims a delete pass at the container root and the cost of getting its filter wrong is the whole filesystem. Nothing destructive happens before a copy is taken. The dump carries DROP TABLE per table with no transaction around it, and the instance tree is replaced wholesale, so a wrong set or a failure part-way through left nothing to go back to. restore.sh now writes a pre-restore-<stamp>/ set - dump, instance tarball and a README saying what it is and how to load it - and refuses to touch anything if that copy cannot be written. Two more found while proving the above, both of which stopped the overlay working at all: * Every script under scripts/backup/ was committed non-executable, while every other .sh in the repo is 755. The compose entrypoints are /opt/backup/watch.sh and backup.sh from a read-only bind mount, so the backup and watcher containers could not start from a clean checkout on any machine, and nothing could fix it at runtime. * Dockerfile.backup could not build without the corporate CA staged. A plain COPY of the gitignored cert fails outright; the trailing-glob form does too, because BuildKit fails a COPY whose pattern matches nothing - which is exactly the case being handled. Copying the directory and testing for the file inside it is the form that works both ways. Verified by building the image with no cert present. Suite 2600 passed / 9 skipped. The finalizer has unit tests covering the atomic claim, the parked failure and the endpoint no longer running the upgrade. Scope: product	2026-09-06 17:38:10 -04:00
cproudlock	83190422c9	Act on the review of the wilm import ... Six findings from a four-agent review of the imported work, plus the contract bump the import owed. Nothing here changes what the imported features do. The contract version was not bumped for a change that widens what a plugin may declare. The asset-panel vocabulary gained two tab groups (serviceprocess, reports), a datetime column format, a map.body key and a tabs section that renders as a table. 0.29.0 bumped for exactly this class of change when the optional group was added, so this is 0.31.0 with a ledger row. All four are optional and resolve server-side when absent, so a plugin declaring none of them is unaffected. Site Backups now answers honestly where it has nothing behind it. The overlay is a compose profile: a separate container writes sets onto a shared volume and the api container only reads them. A Windows/IIS install runs no such container and the default root resolves to a drive-letter path nothing creates, so Back Up Now either failed on PermissionError or reported a run queued that would never happen, under a banner reading "No verified backup yet" - which describes a backup that failed, not a feature that was never installed. The three write routes refuse with the same 404 shape the configuration-policy routes use for a disabled feature, the list and status routes report whether a store exists, and the page says so instead of offering buttons. Two listAll callers still read a response that is not there. listAll resolves to a plain array through fetchAllPages, so response.data.data throws: PC Types settings rendered empty, and in Configuration Policies the throw happened inside the shared try, so PC types, applications, the action catalog and monitors were all left unassigned by one bad read. The agent package upload cannot succeed on IIS as shipped. MAX_CONTENT_LENGTH went to 256 MB to admit the ~69 MB tray binary, but IIS request filtering caps a body at 30,000,000 bytes and answers 413 before waitress sees the request, so web.config needs the matching requestLimits. The backup image could not be built by anyone outside a TLS-intercepting network. Dockerfile.backup did a plain COPY of a gitignored corporate root CA, and a COPY of a missing file fails the build. The trailing glob makes it a no-op when the cert is absent. endpointstates.domain was the one new string column written without a clamp. Under STRICT_TRANS_TABLES an over-long value raises Data too long and aborts the whole check-in, so a single malformed report would stop a machine reporting telemetry and events entirely. Not fixed here, and left with upstream: the restore path finalizes its schema upgrade only from a browser poll loop, so closing the tab mid-restore leaves the data behind the code; it restores instance files from the newest snapshot rather than the set that was asked for; and it takes no safety dump before the destructive load. All three are in the Docker-only overlay, which this site does not run, and none can be validated here. Scope: product	2026-09-06 16:47:47 -04:00
cproudlock	6e36df841e	Import the wilm endpoint telemetry, PC Detail rework and backup overlay ... Ports 37 of Benjamin Baker's commits from the wilm branch, mirrored from the corporate GitHub at 7905917c. wilm rides the publication lineage and shares no ancestor with wj, so this is a filtered tree patch rather than a merge: 94 files, and every path the publication export strips (docs/, tools/, mcp/, CLAUDE.md, start-*.sh, .gitea/, the three excluded tests) is held back, because a raw diff would delete each of them from this tree. What comes across: * Endpoint telemetry, phase 2. Physical disk health from two sources, NIC link state, boot type, Defender health, processor/memory/chassis inventory, the listening TCP set, mapped network drives and Windows Update history. Five additive migrations, 0012 through 0016, off the 0011 head. * Mapped drives are collected by a probe running in the interactive user's session, because they do not exist in the agent's session 0. Two Windows sources can disagree about one drive and the worse status wins, so a half-broken share is never rendered as healthy. * Windows events are reported once each, keyed on the channel's own record id, instead of being re-sent whenever a report overlapped the last one. * PC Detail rework: Details cards flow, the hand-written Network card gives way to a full address history, Storage and disk health merge into one card with two sections, Operator Reports and Corporate Tools get their own tabs. * An opt-in restic backup overlay with a third container, restore gated on the dump's own data revision rather than the code head, and backup/restore history. * The agent's version is read from the binary's PE version rather than trusted from the csproj, and the Fleet view gains an Agent Version column. * The enqueue route now asks caller_holds instead of the owner's permissions, which closes the scoped-token elevation on run_script, update_app and system_restart. Held back deliberately: * agent/installer/install-shopdbagent.ps1 stays deleted, as upstream deleted it. The installer that reaches a bay is generated by _install_script() in plugins/endpoints/api/siteagent.py; the standalone script had stopped being shipped and was collecting fixes no endpoint ever ran. * The .gitignore hunk adding docs/*.md. Correct on the publication lineage, where docs/ becomes the wiki and its absence is how the gates detect a published checkout. Here it would silently ignore every new doc. * Their deletions of docs/DEV-BUILD-TEST.md and docs/PLUGIN-HOOKS.md. Upstream those were stubs; here PLUGIN-HOOKS.md is the 31KB document CLAUDE.md points readers at. * Five zero-byte files committed by an accidental `git add -A`, a run log from the author's own machine, and docs/notes/, which is working scratch that says so in its own header and belongs in devnotes/ on this side. Carried with the import so it lands green: the 17 new routes are described in docs/api-inventory.json with the spec and project map regenerated, and two new frontend specs read fixtures through fileURLToPath(URL) which node rejects under jsdom's own URL global. Suite 2591 passed / 9 skipped; frontend 391 passed; naming, style and doc gates clean. Scope: product	2026-09-06 16:47:24 -04:00
cproudlock	13be0ef90f	Import the wilm agent store and WINC adoption work, with four fixes ... Eleven commits from the sister site: a reversible Maximo/WINC machine-adoption import, a versioned agent binary store with upload and rollback, remote-BuildKit CI, tray version reporting, and UI fixes. Applied as a patch again - the published mirror rewrites hashes, so the branches have unrelated histories and git refuses to merge them. Reviewed by three agents before merging. Four things fixed here. THE PUBLISH WORKFLOW WAS DEAD. The new remote-BuildKit step in publish-images.yml sits at column 0 instead of indented under steps:, so the whole file is invalid YAML and GitHub would refuse it. Their entire GHCR distribution channel - which Start-ShopDBDev.ps1 and the compose files consume - was broken at the source and clearly never run. Indented; all five workflows now parse. THE IMPORT'S APPROVAL CHECK COULD NOT FAIL. commit() built the plan and the approval packet from the same upload and then asked the writer whether they matched, so packet_matches_plan compared a hash against itself. The documented contract - "a human approves a specific, frozen plan, and the writer can prove nothing drifted" - did not exist in the web path: an admin could preview file A and commit file B, gated only by the needs-create group NAMES they ticked. The preview's planhash is now required at commit and checked against a freshly rebuilt plan. Their tests committed without previewing, which is what let this through; they go through the real flow now, and three new tests pin it - all confirmed to fail with the check stubbed out. endpoints.agentpackage QUIETLY GREW. It used to mean "wrap the staged exe in a ZIP"; with the upload routes it also means "choose the executable every future package installs and runs as SYSTEM". Before this, staging needed filesystem access to the server. The permission description still said only "generate and download a package", so an admin granting it could not see what they were granting. Reworded. NOTHING RECORDED WHAT WAS STAGED, OR BY WHOM. The manifest held version, size and timestamp; re-uploading a version replaces the bytes in place under the same label. A sha256 of the stored file is now recorded, and the upload is audit-logged with it - after an incident there was previously no answer to who put that binary on the fleet. Also documented the nine new routes in docs/api-inventory.json (that gate cannot run on the mirror, so it falls to us at promotion time again), regenerated the spec and the project map, and kept our own docs where they conflicted: docs/ is scrubbed from the mirror, so their DEV-BUILD-TEST.md still had the dev-be channel mapping we corrected, and their PLUGIN-HOOKS.md was a new 45-line file rather than our 737-line canonical one. Their journal section is appended to ours. Contract version comes across as 0.30.0 for a genuinely additive surface. Suite 2369, frontend 324, gate green. Scope: product	2026-09-03 07:54:14 -04:00
cproudlock	e98e15259e	Unpick one site's conventions from the shipped bundle, and bind the launch-path ... check to the file it actually runs Two clusters the five-agent review found. Both are about a product that another plant has to be able to install. ADR-015: the shipped monitor bundle was the originating plant's fleet convention frozen into product seed. The business policy was seeded as "RPCA Shopfloor App Monitors". RPCA is that plant's facility designator, the row is source='shipped', and the API refused to rename it - so every adopting site would carry another plant's label at the top of its policy list forever. 0007 now seeds the neutral name, and 0010 renames it on installs that already stamped 0007, guarded so it cannot collide with a policy a site made itself. Downgrade is one-way on purpose: putting another site's designator back is not a state worth reaching. Which PC type the bundle attaches to was the literal 'Shopfloor PC'. A plant whose type is "Shop Floor Workstation" silently got no shipped monitors at all, and one using that name for something else got all of them. It is a setting now, defaulting to the seeded name so nothing changes here, and a blank value attaches the bundle to nothing - the right neutral answer for a site that does not want it. Shipped rows refused every edit, including isactive and launchpath. A plant whose eDNC is on D:, or that runs no UDC, could not correct the path or switch the monitor off, and then carried permanent false drift for it. ADR-015 allows seed data BECAUSE a site can edit it afterwards. Paths and the active flag are now editable; kind and displayname stay fixed, because that pair is the identity 0007's bundle matches on. Also corrected two false claims in 0007's docstring: the policy tables belong to the computers plugin, not the core baseline, and a guarded no-op does NOT reconcile later - Alembic stamps the revision either way, so the bundle would simply never seed. LaunchPathPolicy was purely lexical. GetFullPath is string normalization that never touches the disk, so the root check constrained the NAME while File.Exists and CreateProcess followed reparse points to somewhere else. A junction under a writable directory in an approved root - creating one needs no privilege, only write access to the parent, and vendor trees under Program Files are not reliably locked down - turned WebUI monitor-edit rights into code execution as the console operator. It now resolves the real target, leaf and directory chain both, and re-checks THAT against the roots, rejecting if resolution fails. Alternate data streams passed the whole policy on net8.0: an interior colon no longer throws in GetFullPath, so "readme.txt:payload.exe" reported an .exe extension, passed the root check, and existed. Rejected explicitly now. Six tests added for the shipped-row and pctype behaviour. The Core project compiles clean. The launch-path change has NOT been exercised against real junctions, ADS or 8.3 names - that needs the win11 VM, and per the reviewer it wants a planted-violation suite there rather than my say-so. Still open from the review: monitor revocation does not take effect until agent restart, ActionPicker is keyboard-inoperable, the lightbox has no focus trap, one attached monitor grants generic any-target process control, and screenshots are retained unencrypted in the user profile. Scope: product	2026-09-02 15:44:17 -04:00
cproudlock	b5540e631c	Merge the wilm endpoint-monitor and channel-distribution work into main ... Six commits from wilm-import: the squashed import, route documentation, the channel remap onto our tag-and-trunk model, the project map, parallel test support, and the four smaller review findings. Merged rather than cherry-picked: the published mirror rewrites hashes, so the commits do not apply one at a time against our history even though the squashed diff applies clean. Scope: product # Conflicts: # docs/PROJECT-MAP.md	2026-09-02 15:19:55 -04:00
cproudlock	28bf7ea1bf	Regenerate the project map for the imported plugins and tests ... Scope: product	2026-09-02 14:26:10 -04:00
cproudlock	3587a35ad6	Regenerate the project map ... Test count had drifted from the tree. The map is generated; a stale one fails scripts/gen_project_map.py --check in CI. Scope: product	2026-09-02 12:17:24 -04:00
cproudlock	2122c3b45c	Tab the PC detail page, and let a panel say which tab it belongs to ... The page had grown to roughly ten cards in one flow, and three more surfaces were wanted on it: installed software, live metrics, and the service controls. Stacking those into the same column would have buried what the page is for. Tabs were mocked and rejected on 2026-08-28, on the grounds that the strip duplicated the sidebar and cost hero space. That objection is answered by where the strip sits: the hero and the triage band stay above it and never move, so "is this machine OK" is still answered without a click, and the strip partitions the detail underneath rather than navigating anywhere. WHY A PANEL DECLARES ITS OWN GROUP Most cards on a detail page are not the page's. They arrive through the ADR-010 get_asset_panels hook, so a page that sorted panels into tabs by knowing their ids would defeat the hook it is built on. Panels now carry an optional group instead, and the page only knows the five group names. Resolution happens server-side, in the asset-panels route, so that an absent or misspelled group becomes 'details' before the client ever sees it. The client renders known groups only; an unrecognised value that survived to it would be a panel that exists and is displayed nowhere. Contract 0.28.0 -> 0.29.0, additive - every existing plugin keeps working untouched, and the endpoints status panel is deliberately left ungrouped so the default path is exercised by a shipped plugin and not only by tests. A tab is offered only when something fills it. An empty tab reads as broken rather than as "this site does not collect that", so the strip is built from the groups that actually returned data, and a tab that disappears on refetch falls back to Details rather than leaving the page blank. AssetDetailTabs is core, not a PCDetail component: machines, printers, network devices and measuring tools all have the same shape, and the shared asset-detail shell they need starts here. Scope: product	2026-08-31 14:24:16 -04:00
cproudlock	df083776b3	Offer the search box to whoever can actually search ... The sidebar box was drawn for anyone signed in, on the reasoning that offering it to a signed-out reader gives them a control that can only answer 401. That stopped being the right test when the guest role became able to hold assets.view: the box was hidden from exactly the visitor the grant was reopened for, and a guest could reach search only by typing /search?q= by hand. The client cannot decide this. Its hasPermission getter reads the signed-in user's permission list and a guest has no user at all, so asking it about a guest answers no for everyone. get_navigation already had the problem and already solved it by letting the server decide, so capabilities does the same thing and reports what the CALLER holds. It reports booleans about the caller and never what the guest role holds in general, so it tells an anonymous reader nothing the rendered page does not. Public because it is the pre-login call that decides whether the box is drawn at all. Members who lack assets.view now lose a box that only ever 401'd for them. On a failed call the box stays hidden rather than falling back to the old signed-in test, which would put it back in front of those same members. Scope: product	2026-08-31 09:56:38 -04:00
cproudlock	d9eb7a74db	Regenerate the project map after promotion	2026-08-31 09:06:25 -04:00
cproudlock	dbc43162c5	Pin the approved content digest on an endpoint command ... Delivery descriptors for run_script and update_app are resolved when the agent claims a command, not when an operator queues it, and the command row recorded no digest: the enqueue path resolved the descriptor purely to validate it and threw the result away. So what an operator approved and what an endpoint runs were two separate lookups against mutable content, minutes or hours apart. Anyone able to replace a script body or an installer package could do so in the gap. The claim then re-resolves, the descriptor faithfully describes the new bytes, the agent verifies correctly, and it runs as LocalSystem. The audit trail names the operator who queued the original. The gate on the verb and the gate on the payload were also different grants: pushing needs endpoints.deploy, but replacing what gets pushed needs only applications.edit. pinnedsha256 records the digest as it stood at enqueue, and the claim refuses a descriptor that has moved, failing the push with an audit row instead of shipping it. Null for actions that deliver no content, and for rows queued before this migration, which claim as before. A descriptor that vanished entirely is not treated as tampering: that path already sends the command through undescribed so the agent fails closed. Separately, a result may now only finalize a command that is actually running. That needs no agent change and closes two things: reporting on a command that was never claimed, and overwriting a recorded failure with a success. The agent now sends its hostname on the result callback so the server can check it owns the command it is finalizing. The shared collector key does not say who is calling, so without it any agent can post a result for any command id. The server treats the parameter as optional for agents that predate this and enforces it when present, so there is no flag day. This is a floor, not a fix: only a per-endpoint credential makes the caller's identity provable, and that is tracked separately. The .NET side is unbuilt here and needs a build plus a win11 VM run. Scope: product (cherry picked from commit d64b7122ab)	2026-08-31 09:06:25 -04:00
cproudlock	5f27f81e88	Clear the review blockers on the merged wilm work ... A multi-agent review of the merged branch produced 29 findings that survived adversarial verification. This clears the blockers and the four worth fixing before it reaches main. The design held up well: path-traversal defence on all four serving routes is clean, the agent's SHA-256 check fails closed, enqueue authority is proportionate and audited, and no site literals or credential leaks in the new C#. THE TESTS NEVER RAN pytest.ini sets testpaths = tests, so plugins/endpoints/tests/ was collected ZERO times. Fifty-five lines of delivery assertions were dead code, and run directly five of them failed. plugins/printedparts/tests/ had been invisible for longer. That is the worst shape a gap can take: the suite is green, the coverage is imaginary, and the collector auth on a download route could be deleted without anything going red. Both files moved under tests/. tests/test_no_hidden_tests.py now fails if a test file appears anywhere pytest will not look; it caught a staged copy inside the installer bundle on its first run, and was verified by planting a violation. The five failures were stale expectations, and they split by cause. His: the applications panel made six panels where his test said five, and 'applications' belongs in KNOWN_PAYLOAD_KEYS now that it is persisted - his own test contradicted his own feature. Ours: we renamed the events card to activity, moved the fleet-wide cards to endpoints.fleet, and made the list route a bare route('') which Flask rejects unless the blueprint is mounted under a prefix. FOUR FIXES The snapshot table's key was a 32-bit autoincrement on a table rewritten wholesale every check-in, so it climbs at (endpoints x apps) per report and exhausts on MySQL. Widened while the table is unreleased. Widening it broke every insert on SQLite, which only autoincrements a column typed exactly INTEGER PRIMARY KEY, so it carries a dialect variant - and nothing caught either fault, because no test inserted into that table at all. One does now. Ingest truncated displayname but not displayversion or publisher. On strict MySQL an over-long registry value fails the whole check-in transaction, so one machine with a malformed Uninstall entry goes silent entirely. SQLite does not enforce lengths, so the suite would never have shown it. update_app's version: branch pinned AppVersion.sha256, but that route resolves the version to its APPLICATION and serves application.installpath - so the digest described an artifact the route never sends, and where it was null the agent had nothing to verify and failed open. It now pins the file actually served. The same branch ended the delivery URL at the id, so an MSI arrived with no extension and could not be routed through msiexec. It carries the filename now. Agent packaging moved off require_role('admin') to a new endpoints.agentpackage permission. The generated ZIP carries a live collector key, so this is a heavier grant than deploying to one endpoint - and a role cannot be given to one person for one job the way a permission can. Full suite 2231 passed (2198 before, plus the 22 that never ran and 11 new), frontend 314.	2026-08-30 17:17:13 -04:00
cproudlock	7a8acbbcb5	Regenerate the project map after promotion	2026-08-30 11:12:09 -04:00
cproudlock	13c46af5ce	Rebuild the dashboard around what needs a person ... The board was one card per source in a fixed grid, and both halves of that hurt. Every source got the same small box whatever it had to say. Warranties showed 5 of 50 behind "and 45 more"; the crash card truncated at 4; a one-row card sat beside a four-row card and left a hole. Nothing had room, which is the complaint that started this: not enough space to relay the data. And a box per source flattened priority. An enforcement failure and a warranty notice looked equally urgent because each owned a card, while the largest type on the page belonged to counts that are true every day and need nobody. WHAT THE PAGE IS NOW A severity rollup, one worst-first queue of everything that needs a person, and a row of context beneath it. The queue merges every critical and warning row and sorts them, so reading order is work order. It is full width, which is where the room came from. Rows are capped per source, because fifty expiring warranties must not bury two crashed PCs, and what did not fit is named per source with a link to it - a bare "and 45 more" that leads nowhere is worse than saying nothing. The rollup counts ROWS, not cards, and filters the board in place rather than navigating away, so the counts and the rows cannot disagree. Severity, position and permission were already declared per card and already sorted on. The old layout threw that away; this only stops it. WHAT WENT, AND WHY IT SHOULD NOT COME BACK Four asset tiles plus an "All assets" tile: machines + PCs + printers + network IS all assets, so the page showed a total beside its own addends at the same weight. It is now a line under a rule, where a total belongs. The printer strip. It carried the printer count a third time - the tile row and "Total Printers" were already the same number - and "Online" equalled total, so three tiles held one fact. A genuinely low cartridge already reaches the queue through the printers-supplies card, which is the moment anyone wants it. Three cards whose whole job was to display 0. Recent Devices: the largest block on the page, listing five machines that were all "In Use" with business unit "TBD". Recency is not a reason to look at something. CONTRACT 0.26.0: 'shortname' Cards may declare a short label for their rows in the merged queue. Their 'title' is written as a heading - "Endpoints with crashes or shutdowns" - and as a row badge it ran wider than the hostname it qualified, putting the badge rather than the asset at the start of every line. CSS-truncating the title was tried first and was worse: four consecutive rows reading "ENDPOINTS WITH CR..." say nothing at all. Optional and defaults to the title, so a plugin that does not declare one is unaffected. Additive field, minor bump. ALSO: "silent 1 days" A meta suffix is written for the plural case and the card cannot know the value. One row saying it was a shrug; the merged queue puts every quiet endpoint on one screen, so it said it five times in a column and read as broken. Now singularised at exactly 1, on the first word, keeping what follows: "1 event (48h)", "silent 1 day". The test asserting ' ms' survives is what caught the first version turning it into ' m'. DashboardRow.vue is extracted because the queue and the context panels render the same row, and the map tooltip, link fallbacks and chips are too easy to get subtly different in a second copy. Verified on screen against live data. Full suite 2178 passed, frontend 301.	2026-08-29 21:05:48 -04:00
cproudlock	b8fc2e9e33	Upgrade the eight shipped dependencies carrying advisories ... scan_sbom.py reported 50 advisory matches against shipped components, 16 of them at or above HIGH. This clears all of them: the scan now reports 0 shipped findings and the gate passes. pyjwt 2.12.1 -> 2.13.0 idna 3.13 -> 3.19 axios 1.13.2 -> 1.20.0 (29 advisories) postcss 8.5.6 -> 8.5.26 dompurify 3.4.11 -> 3.4.14 nanoid 3.3.11 -> 3.3.18 follow-redirects 1.15.11 -> 1.16.0 form-data 4.0.5 -> 4.0.6 Every one is an in-major bump inside a range the manifests already allowed, so this is a lockfile refresh rather than a migration. The Python side was compiled with --upgrade-package for each name so nothing else moved; the diff is two version lines and their hashes. nanoid is worth recording: it is transitive under postcss at ^3.3.11, and its advisories are fixed on both the 3.x and 5.x branches. 3.3.18 is the fix that is actually reachable. The declared floors for axios and dompurify were raised to the versions in the lockfile. They were ^1.6.0 and ^3.4.11, which still nominally admitted the vulnerable range - npm resolves to the highest match so an install was never going to pick one, but the manifest should state the minimum safe version rather than rely on that. axios 1.20 adds optional proxy-agent support, which pulls https-proxy-agent and agent-base into the tree. Both are Node-only and neither reaches the browser bundle; the build was checked for them. The main chunk grew 256kB to 271kB (85.7kB to 90.5kB gzipped) from the rest of the axios delta. The PyJWT entry in the allowlist was removed rather than kept. It stopped matching the moment the upgrade landed, and an allowlist that accumulates entries for advisories that no longer apply is one nobody rereads. The reasoning is preserved as the worked example in docs/DEPENDENCY-SECURITY.md. Verified: full suite 2161 passed, frontend 273 passed, build clean, npm audit --omit=dev reports 0 vulnerabilities.	2026-08-29 17:42:35 -04:00
cproudlock	794bb4668f	Add vulnerability scanning on top of the SBOM ... The SBOM was generated and installed onto every server, and nothing read it. "Are we exposed to this advisory" was still a person going and looking, which meant it happened when somebody remembered to. scripts/scan_sbom.py queries OSV.dev by purl - no account, no API key, no agent, no vulnerability database to maintain on a box that cannot reach the internet. The SBOM is already the input OSV wants, so this is a lookup against what shipped rather than a second scan of the tree that could disagree with it. Three decisions worth recording: Shipped and build-only findings are both reported, but only shipped ones can fail a build. Two thirds of the components here are build tooling the SBOM already marks scope 'excluded', and the highest-severity match in the whole tree today is a CRITICAL against the test runner. A gate that fails a release over that trains everybody to bypass it, and then it catches nothing. Severity is taken from the advisory, never derived. Where there is only a CVSS vector the report says UNKNOWN, which is shown and never gates. A guessed score is worse than an honest gap because somebody would act on it. The allowlist is a record rather than a mute button: each entry carries the reason, the accepter, the date, and what would invalidate the reasoning. The first entry is a worked example - a PyJWT algorithm-confusion advisory that needs both mixed algorithm families and a raw JSON JWK as the key, and this application configures neither. The report ends with a per-component upgrade rollup, since that is the part somebody acts on. A target on a different major version is labelled rather than listed beside patch bumps as though the two cost the same. Current state: 8 shipped components have fixes available, none of them exploitable through a path this application opens today.	2026-08-29 17:18:58 -04:00
cproudlock	0baed4cc80	Close the gaps against the design: critical tile, filters, policy card, action placement ... Four things the mockup showed and the first cut did not have. FLEET - A 'with critical events' tile, from the stability route that existed and nothing called. Clicking it narrows to rows whose health is not ok - client side, because the health rollup is assembled per row after pagination, so it narrows the PAGE rather than the fleet. Worth knowing before the count is read as fleet-wide. - PC type and Location filters. The list route already accepted both; there was simply no control. Both use listAll, not list: perpage is clamped to 100 server-side, so a long location list would lose its tail in a dropdown while still looking complete. - The machine cell shows number and name together where they differ, so '3105 Okuma' rather than a bare number. PC DETAIL - A Policy card, contributed by the computers plugin. It matters because the effective policy already narrows the command allowlist, so without it the reason a control is unavailable on a given PC is written down nowhere. Provenance, so it sits in the reference column, not the triage band. Its route returns an EMPTY field list rather than the feature-gate 404 when policies are disabled. The panel contract hides a card with no fields, so it disappears on sites that have not enabled the feature - a 404 would have rendered a broken card on most of them. That was flagged in the design review as a snag; this is the fix. - Refresh Status and Restart Computer moved out of the page header into the triage band. They act on the MACHINE; Print Label, Edit, Retire and Back to List act on the RECORD, and mixing them put the most consequential button on the page - it halts whatever machine the PC serves - between printing a sticker and editing a form. In the band it sits beside the liveness chip and the machine it would take down. The band takes them through an 'actions' slot rather than knowing about PC verbs, so a measuring tool can put 'Calibrate' there without the core component learning anything about computers. Full suite 2124 passed, vitest 260 passed, gates green, spec at 333 paths.	2026-08-28 23:00:25 -04:00
cproudlock	414a7243d7	Merge events and commands into one Activity feed, and lead with live state ... Two changes to the PC page, both about what a responder meets first. LIVE STATE FIRST. The contributed panels moved to the head of the card flow. The grid is a two-column FLOW (column-count, with display:contents on the column wrappers), so a card cannot be pinned to a side and only document order decides where it lands - putting the panels first fills the left column with endpoint status, services, processes, interfaces and activity, and leaves identity, hardware, network and location to the right. Panels still self-order among themselves by declared position and still self-hide when no plugin answers for the asset type. ONE ACTIVITY FEED replaces the Recent events card. Events and commands answer the same question - what happened to this box - and reading them as two lists meant interleaving by eye. GET /api/endpoints/asset/<id>/activity merges them newest-first, capped at 15. Both are keyed by assetid, so it is two indexed reads and a merge sort rather than a join, and it renders through the existing ADR-010 list renderer with no bespoke frontend work. A failed command is marked critical, because that is the row somebody needs to notice. Not collapsed by default, unlike the card it replaces. The core AuditLog is deliberately NOT a third source: its rows carry the action as the entity name and the asset only inside a details string, so including it would mean parsing prose to find out which machine a row belongs to. Requester names are withheld from callers who cannot themselves issue commands, via caller_holds. A read-only viewer - a guest, on a site that grants this panel - gets the history without a roster of who restarted what, and the lookup is skipped entirely rather than fetched and discarded. requestedby holds a user ID, so names resolve through one batched query, not one per row. Verified on dev telemetry: a queued command shows as 'Collect now / Pending by admin' above the Windows events, ordered by time. Full suite 2124 passed, vitest 260 passed, naming/contract/parity gates green, spec regenerated at 332 paths.	2026-08-28 22:34:59 -04:00
cproudlock	54f0857e13	Split the fleet roster from per-endpoint reads, and stop offering controls nobody can use ... Twelve endpoints routes shared one permission, which made a reasonable policy impossible to express. A site that wants signed-out visitors to see live status on a PC page had to hand them the entire fleet - including every logged-in username on the floor - because the per-asset panels and the fleet-wide list were gated on the same string. endpoints.fleet now gates the list and the three dashboard routes. endpoints.view keeps the per-asset reads: status, services, processes, interfaces, events, command history. The name is load-bearing. guest_grantable_permissions() opens any '.view' or '.read' permission to the guest role, so calling this one 'endpoints.fleet' rather than 'endpoints.fleetview' is what keeps the roster off the public surface while per-asset telemetry stays grantable. Verified both ways. Also: the services panel emitted Start/Stop row actions to every reader, because row_actions_for() never consulted the caller. Clicking was always refused server-side, so nothing was exploitable, but showing a read-only viewer controls over a shop-floor PC is the wrong signal - and it stops being cosmetic the moment a site grants that panel to guests. That needed a contract addition rather than a local check: caller_holds() on shopdb.api answers whether THIS caller holds a permission without refusing the request, which is the complement to require_permission for a surface that serves everyone but should only offer what the caller can act on. Guests are included, since a signed-out visitor holds whatever the guest role was granted. Reaching into shopdb.utils.authz instead would repeat the ADR-002 violation the contract gate caught earlier in this branch. Contract 0.24.0 -> 0.25.0. DEPLOYMENT: endpoints.fleet is a new permission. Run 'flask seed permissions' and grant it to the roles that should keep seeing the Fleet page. Until then that page is admin-only, since admins bypass every check - a safe direction, but not an automatic one. Full suite 2124 passed, naming and contract gates green, map and spec regenerated.	2026-08-28 21:14:27 -04:00
cproudlock	784964c907	Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import ... Ports the corporate-GitHub wilm branch onto main, keeping the two branches separate. Thirteen upstream commits arrive with their authorship intact; the review fixes and the portability work below are on top. What arrives: endpoints plugin - a ShopDB agent reporting endpoint state, telemetry, services and interfaces, with a command channel the agent long-polls. Ingest stays on the generic collector route (ADR-006); the agent key is a UI-managed single-scope token whose revocation is an instant kill switch. computers policies - Configuration and Business Policies (RMM-style), scope trees validated at write time, actions checked against the endpoints command allowlist. Feature-gated: every route 404s while the setting is off, so enabling it can never half-expose the surface. asset CSV import - templates, a collector script carrying the site path, and a validate-then-commit import. A plugin can own a virtual column plus a post-write row hook, which is how a PC's machine number becomes a controls link without a hand-built relationships sheet. agent/ - the Windows agent and its installer. Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22), mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability table had no rows for any of them and now does. MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the only JSON columns in the product, and 5.6 has no JSON type: the migration failed with a bare syntax error near the word JSON. They are TEXT now, through a JSONText type decorator, so all seventeen call sites still read and write dicts. The convention elsewhere is already TEXT (guides.blocksjson, geenforce.manifestjson). Review fixes carried on top: - six policy GETs were jwt_required(optional=True), which authenticates a caller without checking anything; they require computers.view now, and six redundant jwt_required decorators are gone - the agent installer took a default BaseUrl and CollectorKey, so a mistyped run silently pointed an endpoint at the wrong instance; both are mandatory and a non-https BaseUrl throws unless -AllowInsecureHttp is passed - a committing CSV import wrote no audit row Documentation: the 34 new routes are described in docs/api-inventory.json, which is hand-maintained on purpose, and the generated spec now covers 331 paths and 477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row said "use MySQL 8 for dev" and now gives the actual fix, including the trap that 5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error. Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.	2026-08-28 16:17:17 -04:00
cproudlock	8953ce8210	Ticking a section off crosses out the section ... It struck the heading and left everything underneath looking untouched, which is the opposite of what somebody ticking "this section is done" means. The strike now covers every block under that heading, down to the next heading of the same or higher rank - so a top-level section takes its sub-sections with it. That needed the rank, and the HTML importer was throwing it away: h2, h3 and h4 all arrived as a flat 'heading', so a parent heading had no children to cross out. It was the visible half of a real loss - a converted article kept its words and lost its shape. Headings now carry their level and render at it, and a guide written before this reads as flat, exactly as it did. Code blocks, tables and images inside a finished section fade but are not struck. A line through a command makes it unreadable, and re-reading the command is usually the reason to look back at a step already done. The five imported guides were re-imported so they carry their structure.	2026-08-27 15:42:46 -04:00
cproudlock	afce24379c	Reading anything needs a permission ... A sweep of the URL map found 165 GET routes that answered anybody, with no token at all: the whole asset inventory, every machine, PC, printer and network device, the USB checkout history and who is holding which drive, the staff directory with its photographs, every core report. Guest access had been argued about one feature-flag at a time while the front door stood open. Each of those now carries the read permission it belongs to. That also makes the guest role mean something: a site can open exactly the reads it chooses, and nothing follows by accident. Thirty-three endpoints stay public, and each is named with its reason rather than left to be inferred: the app shell and its bootstrap, the settings allowlist (the handler already filters an anonymous caller down to branding and a few keys), images fetched by <img src> which carries no Authorization header, what a display kiosk renders before anybody signs in, the parts and USB kiosks, the client scripts that run as SYSTEM before a user session exists, and a secure-send link whose unguessable token IS the authorisation. tests/test_core/test_public_read_surface.py is the standing gate, the companion to the mutating-route sweep that has guarded writes for months. A new public read fails CI until somebody adds it to that list in front of a reviewer. Four further reads need a login but no permission, because the handler decides from who you are - your own user record, your own API tokens, the role catalogue the grid renders - and the same test proves each still refuses an anonymous caller. Verified not vacuous by ungating one route and watching the sweep name it. The employees plugin gained employees.view; the directory is names, SSOs, photographs and reporting lines, and it had no permission of any kind. Around ninety tests asserted payload shapes against endpoints that answered anybody. They are about what the answer says, not about who may ask, so they are signed in now rather than the gate being loosened. The ones that assert an anonymous caller is refused were left exactly as they were.	2026-08-27 14:26:06 -04:00
cproudlock	2331c03664	Tick a guide's sections off as you work through it ... Only checklist items could be ticked, which answers "what have I done" but not the question a long procedure actually raises: where was I. Somebody looks away to do the thing the guide asked for and looks back at a wall of text. Every step and every heading now carries a checkbox while reading. Ticking one strikes it through and fades it rather than hiding it, because somebody scanning back up the page has to still be able to read what they did. The number is struck through with it, but the step's own "Step 3" label is not - that is the address of the thing, not part of what was done. Headings count, not just steps. A guide converted from an HTML article often has fifty headings and no steps at all, and those are exactly the guides long enough to lose your place in. That also means the card's "N steps to tick" stops reading as a broken feature on a guide that is entirely tickable. Reading only, as the checklists already were: in the editor a tick would be an edit, and an author clicking around their own draft would quietly change what the next reader sees as done. Progress stays per browser and per version - shopfloor PCs share a login, so one technician's ticks must not greet the next person on that bay, and step 4 of version 3 is not step 4 of version 2.	2026-08-27 13:02:24 -04:00
cproudlock	151d7c4f44	Anyone not signed in holds the guest role ... Whether somebody with no account may see something was being answered one feature at a time: a securesend_allow_guests setting, an enable flag per kiosk, and very nearly a third for guides - each with its own settings page, its own vocabulary and its own hand-written branch inside a route. Every one of them answered a permissions question, and this app already administers permissions per role. So a caller with no token is not nobody: they hold the seeded `guest` role. require_permission consults it when there is no identity, which is the entire mechanism. The role is seeded holding NOTHING, so an upgrade opens no door; a site opens one by ticking a box in the same grid it uses for every other role, and closes it the same way. Revocation is read on every request rather than cached, because taking guest access away is what somebody does the moment they realise what they opened, and it has to bite immediately. The redundant @jwt_required() above require_permission is gone from 184 routes. It was belt-and-braces before; now it is actively wrong, because it refuses a request with no token before the permission check can have an opinion, so a grant would appear to do nothing. What a guest may hold is bounded. Any .view or .read permission is grantable. A WRITE has to be blessed by whoever owns it, through the new get_guest_grantable_permissions hook (contract 0.21.0), and Secure Send is the only case that exists: sending a password to a vendor is the one write this product means to offer somebody who never signs in. Everything else is refused by the API and greyed out in the grid rather than warned about - a checkbox that quietly makes deleting an asset anonymous is not a warning anybody reads twice. A guest write records no creator: the trail says when and from where, never who, and that is the reason for the bar. GE-Enforce and Slides gained read permissions so they can be opened to guests without handing out the manage right. Their read routes accept either through require_any_permission, so nothing an existing manager could reach has moved. The navigation endpoint now returns only what the caller can actually reach. Without that a signed-out visitor gets a sidebar of the whole app where every link bounces off the login page, which reads as a broken site rather than as a closed door. A nav item may name several permissions; holding any one is enough, which is what GE-Enforce and Slides need now that a viewer and a manager hold different strings. Two things this found on the way: current_user is a LocalProxy, so `current_user is None` is False for a request with nobody behind it - harmless under a required JWT, an AttributeError the moment verification became optional. And an image pasted into a guide before its first save was never attached to it, so it was visible to editors and invisible to every reader; saving now attaches it, and `flask guides fix-images` repairs what is stored. The guides guest setting written earlier today is gone. Granting guides.view to the guest role replaces it, which is the point.	2026-08-27 12:51:58 -04:00
cproudlock	8531559f22	A guide can be carried to another site ... Guides are written on one site and needed on all of them, and the sites that matter are air-gapped from wherever the writing happens. Until now a procedure was stranded in the database it was typed into: the HTML importer could rebuild one from its original article, but anything authored in the editor had nowhere to go. flask guides export writes one .guide.json per guide, with every image the blocks reference carried inside it as base64. One file on purpose - a bundle that needs its images copied alongside it is a bundle that will eventually arrive without them. flask guides import reads them back, a directory at a time, and refuses anything that is not a bundle before it writes a row rather than half-importing a stray .json sitting in the same folder. What travels is the content: title, summary, tags, blocks, images. What does not is ids, revision history, authorship and publication state, because those belong to whichever database issued them - an imported guide starts as a draft at revision 1, owned by whoever ran the import. The SLUG does travel: it is the address somebody pastes into a work order, and the same guide should answer to the same name on every site. --replace updates the guide already carrying that slug instead of creating a second one, which is how a corrected guide is re-shipped. It drops the stored images first: the incoming bundle is the whole truth for that guide, and a guide re-shipped monthly would otherwise accumulate orphans nothing points at. Bundles refuse a version they are too old to read rather than importing the parts they recognise. A newer site's guide is not a guide this one can be trusted to render. Proven by round-tripping a five-guide export back into the same site, including one carrying two PNG screenshots, and reading it in a browser.	2026-08-27 11:46:50 -04:00
cproudlock	c8efadefba	An HTML article can become a guide ... A site adopting this app arrives holding procedures already written: styled HTML pages that get mailed around, opened off a share, and edited by whoever still has the file. Those pages are exactly what a guide is for, and retyping one into the editor is how a good procedure stays on a share forever. flask guides import-html reads the page and produces blocks. It recognises the vocabulary those pages actually use - a step box with its own number, an alert in one of three colours, a code block with a copy button, a checkbox list, a table with a header row - and maps each to the block that already renders it, lifting embedded images into guideimages so a document stops carrying its screenshots as base64. Page furniture goes: the header, the footer, and the table of contents a guide builds for itself from its own headings. The conversion is lossy on purpose. Colour, fonts, width and nested layout are dropped, because storing blocks rather than markup is what makes dark mode, printing and tickable steps work at all. So an import is always a DRAFT and the default is a dry run that reports what it would produce and writes nothing. Blocks go through the same validator the API uses: an import must not be able to store something the editor could not have produced. The parser is tolerant, because real articles are not well-formed. An unclosed <p> is normal HTML and the browser closes it at the next block element; without the same rule one unclosed tag swallows the rest of the article into a single paragraph, which is what it did before the test caught it. Proven end to end against four real articles, including one carrying two PNG screenshots, read back in a browser.	2026-08-27 11:27:19 -04:00
cproudlock	f59aca5a1b	USB check-in / check-out audit, as a report rather than a paged list ... CMMC removable-media tracking is an audit obligation, and answering "who had this drive, and when" meant paging through the check-in/out list by hand. GET /api/usb/reports/audit returns the events - a take and a return are separate rows, because a count cannot answer the question an audit actually asks - with summary figures riding along for the people who only want the shape of it, and a CSV for the people who want it in a spreadsheet. The window is named rather than a day count: today, last month, six months, a year, all time. Nobody should have to work out that a year is 365. Both directory modes answer in one voice. The external path reads the cmmc_usb checkinoutlog; self-hosted expands one usbcheckouts row into its two ends, so a returned drive is distinguishable from one still out, and the window filter runs in SQL against either end of the loan - a drive taken before the window and returned inside it is a check-in the window has to show. Rows come back from the mode-specific half; the CSV, the summary and the response shape are built once in the route. Ordering is total in both modes. Timestamp alone ties for two events in the same second, and the same query then returns them in a different order each run, which reads as the log being unstable.	2026-08-27 10:20:21 -04:00
cproudlock	95ad4f1ef3	Docs: the USB kiosk, its endpoints and why the list costs a badge ... Changelog, the plugin table, the three kiosk endpoints in the API inventory, and regenerated openapi.json / PROJECT-MAP.	2026-08-27 08:14:02 -04:00
cproudlock	90c3c3102b	Docs: guides in the indexes, the stalest-kind rule, and the changelog ... Guides was shipped without an entry in PLUGINS.md or START-HERE, so a site adopting the product had no way to find out it exists. BACKUP-KINDS.md described a host's badge as coming from the most recently confirmed kind. It has not worked that way since the fleet report was reworked: the freshest revision per (host, kind) is taken, and then the STALEST of those kinds decides, because reporting the freshest would let a healthy NTLARS backup mask a part-marker capture that stopped a month ago. Stalest among kinds that can be JUDGED - a kind never confirmed is unmeasurable, not stale, and treating it as the worst value made a host with a working backup show no badge at all. The backups row also still advertised UDC and CMM kinds that do not exist. api-inventory, openapi.json and PROJECT-MAP regenerated.	2026-08-26 15:06:20 -04:00
cproudlock	580c0d0c80	Record the tab title and Guides packaging fixes in the changelog	2026-08-26 13:13:41 -04:00
cproudlock	6106f4c5f4	Stop shipping duplicate vendors, models and operating systems ... The seed catalog is what a new site inherits, so its duplicates propagate. Removed the four space-stripped twins it carried: vendor 'DellInc.', model 'OptiPlex7080', and operating systems 'MicrosoftWindows10Enterprise' and 'Windows10', each of which had a correctly spaced original already present. 'OptiPlex5000' was NOT a duplicate - no spaced twin existed - so it was renamed to 'OptiPlex 5000' rather than dropped, matching every sibling in the list. Its vendor moved to 'Dell Inc.', which this site treats as the one canonical Dell. Vendors 53 to 51, models 128 to 127, operating systems 14 to 12. Verified afterwards: no exact duplicate rows, no near-duplicates, no orphaned model-to-vendor or supply-to-model reference.	2026-08-24 14:49:31 -04:00
cproudlock	7fb6bdba07	Regenerate the project map for the new script tests	2026-08-24 14:32:31 -04:00
cproudlock	90da0a1c09	Write down the plugin nobody documented, and stop pointing at a deleted branch ... PLUGINS.md listed 14 of the 15 bundled plugins. `backups` had no row: the per-asset configuration backups the collector takes off shopfloor machines, kind-pluggable, with view and download as separate permissions because a .reg is a restorable artifact. Added, with the pointer that separates it from BACKUP-RESTORE.md, which is the database and a different subject. PLUGIN-LAB-PRINTEDPARTS said the finished implementation lives on branch feat/printedparts-plugin. That branch was deleted in the v0.8.0 cleanup; only the 17 lab-stage tags survived, and they are enough. Says so now, and warns that the tags are a record of contract 0.11-0.13 rather than today's conventions. Its manifest example carried core_version >=0.12.0 against a shipped >=0.16.0. The printer-assignment proposal still said the client script is not built. It shipped and released, across v0.11.3 and v0.12.0, and the collector half that contradicted the assigned-vs-observed split was retired in adc7ef4. PROJECT-MAP regenerated for the test count.	2026-08-24 14:05:24 -04:00
cproudlock	adc7ef43f9	Stop the collector being able to set a printer assignment ... The printer feature rests on one rule: a collector report is OBSERVED state and never ASSIGNED state. If a drifted bay's own state can become what it is told to install, enforcement stops meaning anything, because a bay that installed the wrong printer makes itself right simply by reporting it. _sync_printer_links broke that rule. It ran on every collector PC push and, when the payload carried defaultprinter or printers, wrote defaultprinter and connectedto rows from the report - defaultprinter being one of the two types _reconcile_edges is meant to own alone, from authenticated routes a human calls. It was recorded as dormant. It was not. The collector schema ADVERTISED both keys and the example client in COLLECTOR-INTEGRATION.md set $payload['defaultprinter'], so the documentation was recruiting callers for it - while the same document, under "Observed is not assigned", promised that nothing on this path writes a usesprinter or defaultprinter row. The document contradicted itself and the code followed the wrong half. Nothing in the fleet armed it, because the first-party reporter never sent those fields, but any third-party client written from the docs would have. Retired the function, its _sync_one helper and the now-unused PRINTER_LINK_ORIGIN label, and removed both keys from the schema. IGNORED, NOT REJECTED. Unknown payload keys are not validated, so a client written against the old docs keeps pushing successfully and only its printer keys stop doing anything. The response warns, naming the keys and pointing at POST /api/collector/printers for reporting queues and seed-from-observed for adopting them. Dropping them in silence would leave an integrator believing assignments were still being written, which is a worse failure than the one being fixed. The docs were the larger half of this. The schema table rows are gone, the "PC -> printer relationship sync" section is replaced by a note saying what happens now to anyone still sending the keys, and the example client no longer enumerates Win32_Printer at all - it points at Report-PrintersToShopDB.ps1, which posts to the observed endpoint and posts NOTHING on a failed enumeration, since an empty list would erase the host's rows. test_collector_printers.py pinned the old behaviour, so it now pins the opposite: the keys create no relationships, the push still succeeds and says why, a payload without them stays quiet, a human-made default survives a report naming a different printer, and the schema no longer offers the keys.	2026-08-24 11:25:02 -04:00
cproudlock	adb30c8875	Release 0.12.0 ... Printers become a property of the bay, and the container stops throwing away the site's own files every time it is updated. Printer assignment is now end to end. The assignment belongs to the MACHINE and reaches whichever PC controls it, so a reimaged or swapped bay comes back with its printers and nothing had to be saved off the old box. One picker serves both forms rather than two implementations of the same override. A relationship type can declare itself singular, so setting a second default REPLACES the first instead of silently losing to the older row. Alongside the assigned half there is now an observed half: a bay reports what it actually has, kept strictly apart from what it is told to have, because a drifted bay's own state becoming its desired state would make every configuration error permanent. The client script corrects a drifted queue in place rather than only installing a missing one, and a driver rollout can be spread across waves so 300 bays do not pull 30 GB through one five-minute window. DOCKER SITES SHOULD READ THIS PARAGRAPH. The compose stack never persisted the instance directory, so `docker compose build api && up -d api` - the update path the docs themselves gave - discarded plugins.json and every upload with it. It is a volume now, and DEPLOY.md carries the one-time rescue for a stack that predates it. The image also could not be built at all: the frontend stage never copied the plugin staging script its own prebuild hook runs, so every build since that script landed failed. Air-gapped sites need a fresh offline bundle, because the stack moves to MySQL 8.4 LTS and an existing tarball carries only the 8.0 image. The Windows installer already bundled MySQL 8.4, so this closes a gap between the two halves rather than moving anyone. Adds a topology migration guide: IIS to Docker, Docker to a new host, and back. The data moves cleanly; what costs time is that the server address is baked into GE-Enforce manifests, the generated collector script, the printer client scripts and printed QR codes. 1736 tests green. Contract stays at 0.20.0 - nothing under shopdb/api changed.	2026-08-20 08:45:56 -04:00
cproudlock	2d09fa3201	Collect what bays actually have, separately from what they are told to have ... ShopDB knew what a bay SHOULD have and nothing about what it DOES. Adding the observed half makes a rollout a review instead of a typing exercise: the floor reports itself in, you look, and you adopt. The collection uses the mechanism that already exists rather than a new one. POST /api/collector/printers dispatches to the printers plugin's apply_collector_payload, the same ADR-006 hook the computers and backups plugins implement. New client script, new plugin-owned table, no new transport and no new credential. OBSERVED AND ASSIGNED STAY APART, and that is the point rather than a detail. A collector report can never write an assignment row: _reconcile_edges is the only function that writes usesprinter/defaultprinter, it has two call sites, and both are authenticated routes a human calls. If a drifted bay's own state were allowed to become what it is told to install, every configuration error would become permanent the next time that PC checked in. Seeding an assignment from observed state is explicit - POST /assignments/seed-from-observed - because a rollout adopts many machines at once. It routes through the same _reconcile_edges as the editor, so there is one write path with two doors, and a queue matching no known printer is REFUSED rather than guessed into an assignment. That last rule is the lesson from the measuring tools: adopting on a weak key produced 43 duplicate instruments. Two fixes on top of what the agents built. The replace deleted a host's previous rows by exact case-folded name while the read path treats a short name and its FQDN as one machine, so a PC that changed spelling appeared to hold every queue twice - which reads as drift that is not there. And the client sent 'reportedat' where the declared schema said 'observedat'. Also here: the legacy loader now imports machines.printerid, the classic system's record of each machine's default printer, which it silently dropped - the production import would have lost every one. And Set-ShopdbPrinters.ps1 finally registers the per-user logon task, staging Apply-ShopdbDefaultPrinter.ps1 to C:\ProgramData first because the share it lives on is mounted only during the enforcement cycle and the task runs at logon when it is gone. VALIDATED ON WINDOWS 11 (build 26200), not just on Linux pwsh, which parses these scripts happily and executes none of the spooler branches. The reporter: posts a correct payload with the X-API-Key header; resolves BaseUrl and CollectorKey from HKLM when given no arguments; suppresses the virtual queues by port; resolves port addresses; and reads the CONSOLE USER's default out of HKU rather than SYSTEM's own, which is a different and usually wrong answer. Two results matter more than the rest. With the spooler stopped, both the cmdlet and the CIM path fail and the script posts NOTHING - verified against a capture server that recorded zero requests, where an empty list would instead have erased that host's observed rows and read as a bay that lost its printers. A genuinely empty host still posts [], because that is a real and different fact. The logon task registers as the Users group at Limited, and falls back to the well-known SID S-1-5-32-545 when the group name will not resolve, as it will not on localised Windows. It was then run with the source directory RENAMED AWAY, to stand in for the share being unmounted, and it still moved the user's default - which is the whole reason the script is staged to C:\ProgramData rather than run from where it lives. The guarantees against damage were re-checked rather than assumed: an empty assignment changes nothing, an unreachable server changes nothing, -WhatIfOnly leaves no queue, no task, no staged file and no registry value behind, and a drifted queue is repointed IN PLACE with Set-Printer so whoever has it as their default keeps it. Not covered by any of this: the driver-staging path, which needs a real vendor package rather than the class drivers a VM ships with.	2026-08-19 15:32:18 -04:00
cproudlock	72b3904f71	Release 0.11.3 ... The last of the buildings-and-levels bugs, and the shop-floor clients brought into the product. Anyone on 0.11.0 through 0.11.2 should take this. Every hover mini-map in the product reported "this asset has a position but no level" - the levelid prop added in 0.11.0 was passed by none of its seven call sites - and the map PDF printed markers from every floor onto one sheet, which nobody can correct once it is carried onto the floor. The legacy import loader, still to run against production, created markers with no level at all. The gate that should have caught all three asked whether a FILE mentions levelid rather than whether each position does. It now checks per occurrence. Also: printers can be assigned to a MACHINE and reach whichever PC controls it, so a reimaged bay reinstalls its own printers with nothing saved off the old PC; printerdrivers can name a vendor, so two rows cover 41 of 44 printers instead of twenty-one near-duplicates; and the collector reporter and EventSaver now live in the repository with no site baked into either, the reporter generated per site on request. The client scripts were validated on Windows 11 against a live ShopDB, not only by the suite: a bay with no rows of its own created both queues from its machine's assignment, bound them to the right universal drivers, and set the per-user default. The version and the changelog are the release; the detail is in the entry.	2026-08-19 10:56:52 -04:00
cproudlock	8bde89c47e	Release 0.11.2 ... Bug fixes for the buildings-and-levels work in 0.11.0, every one of them found by using the feature rather than by the suite. Anyone running 0.11.0 or 0.11.1 with more than one level should take this release: the map drew markers from every level on whichever floor plan happened to be showing, which is the failure ADR-017 exists to prevent, and it was in the one component that draws the map. The viewer also had no way to choose a level, and the editor never accepted a click at all - its handler was bound only if the map was already a picker at mount, and the editor opens with nothing selected. Also: a level's native size could not be set (the settings page had no height field), the same search could return different rows because fourteen searchers truncated without an ORDER BY, and both map pages carried a scrollbar from subtracting the wrong page chrome from the viewport height. No schema change, and the plugin contract stays at 0.20.0. The version and the changelog are the release; the detail is in the entry.	2026-08-17 15:46:38 -04:00
cproudlock	dd503be4ba	Release 0.11.1 ... A patch release. The pagination-cap fixes, and the documentation that missed the 0.11.0 installer by a few minutes. docs/UPGRADE.md is one of the documents the Windows installer puts on the server, and its buildings-and-levels section was committed after 0.11.0 was tagged. So 0.11.0 shipped the page an operator reads WHILE upgrading without the guidance about that upgrade - which matters most on an air-gapped box, where there is no other way to reach it. The installer now also ships FLOOR-MAP.md for the same reason the Windows runbooks ship. No application code changed on the server side and the plugin contract stays at 0.20.0. The version and the changelog are the release; the detail is in the entry.	2026-08-17 14:38:59 -04:00
cproudlock	24647aa8e8	Release 0.11.0 ... The floor map became a set of drawings instead of one picture, and search now reaches what the forms have been collecting all along. A site can hold more than one building and a building more than one level, and every map position records which drawing its coordinates belong to. An existing single-map site upgrades into one building and one default level carrying the blueprint it already had, and nothing moves on screen. Landmark recalibration moves a level's markers onto a redrawn plan, solved per axis from the landmarks and never from the image dimensions, dry run by default, snapshotted before it applies. Two identifiers the UI collects were findable in almost no way and two plugins' records in no way at all: gaugelabreference matched only measuring tools, maintenancereference matched nothing anywhere, and USB devices and printed items were absent from search entirely because neither is an asset. All four are searchable now, from the global bar and from every list's own search box. Contract 0.20.0 is additive: a plugin that writes a map position must write its level with it. The version and the changelog are the release; the detail is in the entry.	2026-08-17 13:23:49 -04:00
cproudlock	aede5ac621	Match the optional identifiers in every list's search box ... gaugelabreference and maintenancereference are collected on the form and shown on the detail page for all five asset types, and no list's ?search= matched either. Someone holding a physical tag could read it off the machine, type it into the list they were already looking at, and get nothing back - while global search, fixed in the previous commit, found it. The clause is copy-pasted once per plugin, which is how all five came to omit fields their own forms collect, so the test is one parametrized pass over all five lists rather than five per-plugin tests that would drift the same way. It also pins that widening the clause did not turn the box into a pass-through. The api-inventory entries enumerate the fields each search matches, so all five were stale the moment the queries changed; updated with the OpenAPI spec. Not touched: the collector does not send either identifier, correctly - a lab-assigned tag is not something an agent on a PC can discover. The legacy import loader does not map them either, and the classic schema has no gauge-named column to map from.	2026-08-17 13:16:22 -04:00
cproudlock	3324dbd91e	Buildings and levels for the floor map, and make every identifier searchable ... The map was one picture of one floor. A second floor was added, the blueprint changed size, and machines moved, so a position now records WHICH DRAWING its coordinates belong to. Buildings and levels (ADR-017). Each level owns its blueprint per theme and its own native pixel size; assets.mapx/mapy are pixels of assets.levelid, not of the site. A position whose level is unknown renders "level unknown" and is never drawn on the default level, because a marker on the wrong floor plan looks entirely correct while pointing at the wrong place. Repositioning in bulk: filter by unplaced, needs-review or level, search, place, confirm. Landmark recalibration solves the transform PER AXIS from landmark pairs and never from image dimensions - the canvas grew taller without rescaling, so a dimension-derived scale would stretch Y by 1.57 and be wrong everywhere. It defaults to a dry run, reports what would land off the drawing, snapshots before applying, and clears mapverifiedat because a transform is a guess awaiting review. Snapshots restore, including the level and the review state, and a restore snapshots first so an undo is undoable. Search: gaugelabreference was matched only for measuring tools and maintenancereference was matched nowhere at all, for any asset type, while Settings happily offers both identifiers on machines and PCs. A tag an operator is told to record has to be findable or it is a write-only field. USB devices and printed items were unreachable from search entirely - neither is an asset, so the generic asset search could not see them and no searcher existed; they now match on serial, asset tag, label, bin code and gage-lab tag, honouring isactive, with Settings toggles and result labels to match. The retired-application rule was half a rule: GET /api/knowledgebase hid articles whose topic application is retired while global search still returned them and printed the retired application as the subject. A filter is only real if every path that reaches the row applies it. Contract to 0.20.0 (additive): Asset gained levelid and mapverifiedat, Location gained levelid, and resolve_asset_position returns the levelid belonging to whichever source supplied the coordinates. The five plugins that write a map position are re-pinned. The install-list text format gained levelid as a NINTH field, appended, because the shipped Pascal installer reads fields 0-7 by index. That installer still compiles in one drawing's dimensions and bundles one blueprint, so its map is accurate for the default level only; /api/maplevels is deliberately unauthenticated so it can read both at runtime once rebuilt. Recorded in PRINTER-INSTALLER.md section 6 along with the other known gaps. Migration 7d33 converts an existing single-map site into one building and one default level carrying the old map_* settings, then assigns every placed asset and location to it. Nothing moves on screen. Old settings rows are kept so a rollback still finds them. Verified end to end on MySQL 5.6 from a production-shaped database.	2026-08-17 12:55:51 -04:00
cproudlock	7d9a54ca0f	Release 0.10.0 ... A privilege-escalation fix in the Windows installer, the multi-site portability work, and a documentation and API surface that a second site can rely on. The installer created C:\shopdb-flask, C:\Python314 and C:\MySQL84 without breaking inheritance from the drive root, whose default DACL grants Authenticated Users an inherit-only Modify. Two files inside those roots are executed by someone else - shopdb-admin.ps1 self-elevates, and instance config.py is loaded unconditionally by the app - so any logged-on user could replace either and wait. Verified fixed as SYSTEM in the Windows 11 VM against the shipped functions. Contract 0.19.0 is breaking: get_dashboard_widgets returns data and shape, not a component name. The change itself shipped earlier without a bump, with the contract docs still teaching the dead shape; both are corrected. The version and the changelog are the release; the detail is in the entry.	2026-08-17 08:01:52 -04:00
cproudlock	96df19702e	docs: one manual runbook, and ADR statuses that mean something ... DEPLOY-WINDOWS-IIS was a second copy of the manual IIS procedure that had diverged from the first: a different MySQL version (8.0, which reached end of life in April), a different port, a different plugin list, and a profile file that does not exist. Two runbooks for one procedure means a reader follows whichever they found, and one of them was wrong. INSTALL-WINDOWS-IIS covers everything it did plus a preflight step and the subpath method, so the one section it uniquely had - redeploying a hand-built server - is folded in there, with the plugin-chain step it was missing and a note to back up first, and the duplicate is gone. Everything that pointed at it now points at the survivor. Three ADR statuses said something untrue. ADR-013 said PROPOSED while half of it had shipped and ADR-014 had been accepted on top of it. A decision that has been implemented and depended upon is not proposed, and leaving one that way devalues every other status in the index. The catalog half is still unbuilt, which is the ordinary state of an accepted decision: accepted means settled, not delivered. ADR-016 said ACCEPTED for a design where nothing is built - the endpoint and permissions it describes do not exist, so a reader goes looking for them. The status stands, because the decision does; the header now says so plainly and points at where today's credentials actually live. ADR-003 and ADR-004 were ACCEPTED with their own Decision lines still opening "**PROPOSED:**", which reads as though the decision was never taken. And the dashboard proposal carried Status: ACCEPTED, which belongs to a decision record. A proposal is a proposal; the contract it produced is the ADR.	2026-08-14 16:20:59 -04:00
cproudlock	8f9805d964	Generate the facts that go stale, and record the fortnight in the changelog ... CLAUDE.md is read at the start of every session and claimed contract 0.16.0 and 1159 tests while the code was at 0.18.0 and 1567, along with a plugin count and a chain head that were both wrong. Hand-written facts about a moving codebase are wrong within a fortnight, and a reader who cannot tell which lines are stale has to re-derive all of them. So they are derived. scripts/gen_project_map.py reads the versions, the plugin inventory, every Alembic chain head, the ADR index with status and the size of the codebase out of the code, and --check fails a build where the committed map no longer matches. CLAUDE.md now points at it, leads with the multi-site rule, and lists the gates to run instead of restating the conventions they enforce. The changelog's Unreleased section covered 8 of the 46 commits since 0.9.0 and had no Changed, Fixed or Security sections at all. It now carries the whole fortnight, including both contract bumps - which had never been recorded even though ADR-002 makes contract versions their own series.	2026-08-14 13:47:48 -04:00