b1e5dd734df8009ce5d435d85be6be839dfb099c
381 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
b1e5dd734d |
Draw a dual-bay secondary where it actually is, and make two counts add up
Three map and aggregation defects. The first is a behaviour change you asked for; the other two are the 2026-09-03 run's findings. A dualpath secondary bay was excluded from /api/assets/map unconditionally while the collapse setting is on, so positioning it was a silent no-op: the form took the pin, stored mapx/mapy, and the map never drew it. One marker per pair is right when the pair sits in one place, and that stays the default - but a dual-bay machine whose second bay is physically elsewhere is exactly what positioning it separately means. Now: a secondary with its own position draws its own marker; one without is represented by the primary's, which carries the combined '2007 / 2008' label. Nothing extra is needed for the fallback, because the feed already requires mapx/mapy - an unpositioned bay never reaches the loop. The combined label is now suppressed when the partner draws its own marker. Labelling two separate pins '2007 / 2008' would say one machine is in two places, which is worse than either bay being unlabelled. Verified against a live instance: unpositioned secondary gives one marker labelled with both bays; positioned gives two markers each with its own name; clearing the position puts it back to one. That third case matters - a one-way door would be its own bug. machines/dashboard/summary applied the dualpath exclusion to total and bytype but not to bystatus, so the status bars summed to more than the headline above them, by exactly the number of secondary bays. A breakdown that does not add up to its own total is an afternoon of reconciling. The measuring-tool calibrationstatus filter ran AFTER pagination. It is a derived value so it cannot be a SQL filter, but applying it to the current page meant asking for 'overdue' with one overdue tool in sixteen returned an empty page 1, a total of 16, eight pages, and the row itself on page 3. When that filter is in play the matching set is now built and filtered before paging - the same tradeoff the old comment named, spent on correct answers instead of a cheaper query. Seven tests. The pagination one verified against the committed version, which fails it. Scope: product |
||
|
cproudlock
|
45acaf0b8f |
Resolve an asset type by name, and send a map position with its level
Wave 5, the asset forms. Both defects are the same mistake in different clothes: a form deciding something it should have been told. Four forms carried a hardcoded asset-type id - const COMPUTER_ASSETTYPEID = 2, and the same for machines, printers and network devices - to tell CustomFieldsInputs which type's fields to render. Those ids come from seed order. On the instance they were written against, computer was 2; on a freshly seeded database computer is 1 and 2 is network_device. So the PC form asked for another type's custom fields, rendered them, and dropped everything typed into them behind a success toast, because the values were keyed to a type the record does not have. Verified in a browser against a correctly built dist: the form now requests assettypeid=1 on an instance where computer is 1. The name is the stable thing - assettype is a fixed vocabulary and the API already publishes the mapping - so composables/assetTypeIds.js resolves it once per page load. A failed load leaves it undefined, which renders no custom-field panel: a missing panel is honest, a guessed id is what this replaces. Three of the five forms that emit a map position sent mapx and mapy without levelid. ADR-017 is explicit that a position without its level cannot be drawn on the right floor plan, so a picked pin saved a null level and rendered nowhere - while the form displayed "Position: x, y" back, because it kept the value it had never sent. The ADR-017 gate should have caught that and did not: it globbed '*.py' only, so it policed what the server emits and never what the browser sends. It now covers .vue and .js too, and accepts both quoting styles. Verified by planting: drop levelid from PCForm again and the gate names that exact line. Five composable tests, including a fixture with a DIFFERENT seed order - the case the constants got wrong. Still open from this area, and deliberately not in this commit: the PC form writes an INHERITED location back as the record's own. Fixing it needs the API to distinguish an owned locationid from a derived one, which is a contract change rather than a form fix. Scope: product |
||
|
cproudlock
|
ffced2f822 |
Keep a share path inside the share, a UTC timestamp in UTC, and a public map public
Three confirmed findings from the 2026-09-03 run, each independent.
GE-Enforce scope names are directory names on the imaging share, and create_scope
checked only that the name was non-empty. "../escape" wrote a manifest one level
above the share root, "../../escape2" two, and "/tmp/anywhere" wherever it liked
- proven by writing the files and then deleting them. The exposure is the
permission split rather than the traversal itself: geenforce.manage creates the
scope and geenforce.publish writes the file, so a manage-only author could aim a
publisher's write at any path that process can reach. Names are now a single
safe path segment, checked at creation AND again at the write (rows predating the
check exist, and the write is the line that acts), with the resolved target
confirmed to sit under the share root.
Agent timestamps were converted with astimezone(tz=None), which means the
SERVER's local zone, not UTC. An agent reporting 17:26:16Z was stored as
13:26:16 on this EDT box and re-served labelled 'Z', while the same row's
server-written lastreportedat stayed true UTC - one row carrying two time bases
four hours apart. The dashboard's 48-hour stability window compares a real UTC
cutoff against these values, so events left it four hours early. Invisible on a
UTC server, which is how it survived, and nothing in the Dockerfile, the compose
files or the Windows deploy pins TZ. AGENT-CONTRACT.md mandates UTC on the wire,
so a naive value is now taken as UTC rather than guessed at.
The three /api/maplevels reads went back to optional-auth. ADR-017 rule 6, the
API inventory (auth "none"), the module's own docstring and the comment in
PrinterInstallerMap.vue all say they are public, because the printer installer
map runs before anyone logs in and needs a blueprint to draw on.
|
||
|
cproudlock
|
13e992d863 |
Ask what the caller may do, not what its owner may do
Wave 4, first half. A personal API token scoped to endpoints.command alone could queue system_restart, run_script and update_app against a shop-floor PC. require_permission has always suspended the admin bypass for a scoped token, and tests pin that. What nothing covered was a route checking a SECOND permission by hand: the endpoints command route asked current_user.haspermission(elevated), which reads the token's OWNER. An admin minting a narrow token for a script therefore handed out remote restart, script execution and app deployment - the exact bypass require_permission exists to prevent, reintroduced one hand-rolled check at a time. caller_holds is now scope-aware, so the answer is about the caller rather than the person behind it: the owner must hold the permission AND, when the request carries a scoped token, the scope list must name it. An unscoped token is the person and is unchanged; a session is unchanged; guests still answer from the guest role. Both hand-rolled checks in the tree now use it - the endpoints elevated gate and the secure-send tools.secrets check, which had the same shape and the same consequence at a smaller scale. Two tests, one proving the refusal and one proving an unscoped token still passes (a fix that quietly broke every service token would be worse than the bug). Verified by planting the owner-based check back: the first fails. Reported by the fleet-agent lane of the 2026-09-03 test run and confirmed by an independent verifier that minted its own token and queued its own commands. Scope: product |
||
|
cproudlock
|
debef39a18 |
Answer a nonsense filter with a 400, and sweep every list route for the rest
Wave 3. Six findings, one cause: `int(request.args.get('typeid'))` with nothing
catching the ValueError. The caller sent a bad argument and got
500 INTERNAL_ERROR plus a traceback in the log, which says the server is broken
when the truth is that 'abc' is not a number.
shopdb/utils/queryargs.intarg reads it and raises QueryArgError, which an
app-level handler answers as a 400 naming the argument. App-level on purpose: a
route should not have to remember, and forgetting was the entire defect. The
policy differs from get_pagination_params, which falls back to a default for a
bad page number - a nonsense page is harmless and the caller still wants their
list, but silently ignoring a nonsense FILTER returns rows they believe they
excluded.
The gate matters more than the six fixes. tests/test_core/test_bad_query_args.py
sweeps all 131 registered no-parameter GET routes under /api/ with twelve
argument shapes and fails on any 5xx. It found sites the test run had not
reported - statusid and locationid on /api/assets, the second asset-list route,
limit on the employee lookup and the printer supply history - because the
pattern is how a list filter gets written rather than a property of any one
route. Verified non-vacuous by planting the old `int(...)` back into
/api/machines: four tests fail.
Four usb routes are skipped by exact path, with the reason: they reach the
EXTERNAL cmmc_usb database, so their 500 in a test environment is a missing
database rather than an argument fault. That is the complete set - every usb
GET route calling cmmc_usb_connection - not a convenient subset.
One correction worth recording: the first pass rewrote every
`if x := request.args.get(...)` in the touched files, including text arguments,
so a search for 'ZZNOTHINGMATCHES' briefly became a 400. Caught by
test_list_search_identifiers. Only id-shaped arguments are integers.
Scope: product
|
||
|
cproudlock
|
12392a361e |
Refuse a value the database cannot store, instead of quietly changing it
Wave 2 of the remediation plan. Eight findings from the feature test run were the same defect twice: a write that reports success while storing something other than what was sent. The first half is a date parser that swallowed its own error. Two plugins carried an identical private _parse_date returning None on ValueError, so an unparseable date and a deliberately cleared one produced the same write. PUT /api/warranty/<id> with enddate '30-06-2027' answered 200 "Warranty updated" and stored NULL, dropping the record out of every expiry surface on the strength of a typo; the measuring-tool calibration date did the same and flipped the tool to 'unknown', out of the calibration report and the map overlay. shopdb/utils/datevalues.py now holds one strict parser on the contract surface: an explicitly empty value clears the field, anything that is not a date is refused with a 400 naming the field, and those are different intents with different answers. The vendor-lookup paths keep a lenient wrapper on purpose - a provider changing its date shape must not fail the refresh of an otherwise good record, and no person typed that value. The second half is why the truncation findings were possible at all. MySQL was running non-strict: sql_mode on the 5.6 server is NO_ENGINE_SUBSTITUTION with no STRICT_TRANS_TABLES, so it accepted writes it could not store and adjusted them. A 257-character username became its first 100 characters (and the next distinct one then collided with it); an over-long asset number was cut at 50; a malformed releasedate became '0000-00-00' and permanently broke every read of that application. All with a 201. The mode is now set on the CONNECTION rather than left to the server, so a site whose my.cnf disagrees still gets it. That trades silent corruption for a DataError, which uncaught is a 500 - and a 500 says the server is broken when the truth is that the value does not fit. An error handler maps it to a 400 naming the column where MySQL names one. Verified by hand against MySQL 5.6: POST /api/users with a 257-character username now answers 400 "That value is too long for username." and stores nothing, where it previously stored a truncated row and answered 201. Twelve tests. The date guard is verified by planting: restore the swallowing parser and four of them fail. The DataError tests raise the error directly because the suite runs on SQLite, which does not enforce VARCHAR length at all - there is no value that would trigger it there, which is precisely why this went unnoticed. Scope: product |
||
|
cproudlock
|
aa8028f96e |
Let the row refuse the write both kiosks were deciding in Python
Wave 1 of the remediation plan for the 2026-09-03 feature test run. Two kiosks, one shape of defect, one critical consequence. Both kiosks read the state, decided in Python, and then wrote. Nothing stops two requests interleaving between the read and the write, so both see the same pre-state and both proceed. Reproduced on MySQL 5.6 over HTTP, five concurrent takes of 5 from a bin holding 5: all five were accepted and quantityonhand landed on -20, because each request wrote its own arithmetic. The ledger had recorded every one of them, so the cached quantity and the ledger disagreed permanently, which is the number the shop floor actually reads. The USB kiosk had the same shape: concurrent scans of one drive all succeeded and inserted several open custody rows, so one drive was recorded out to two people and the documented 409 never fired. The comparison now lives in the WHERE clause, where the database settles it at write time. printedparts._move_quantity issues UPDATE ... WHERE quantityonhand >= :n and treats rowcount 0 as the refusal, raising InsufficientStock, which take and adjust turn back into the message they already gave - carrying the quantity the database holds now, not the stale one the request read. The low-stock alert derives its before-value from what the write produced for the same reason. Both USB paths claim the device with ischeckedout (self-hosted) or status (external cmmc_usb) in the WHERE clause; the external one claims BEFORE writing the log line, so a refused claim leaves nothing to undo. Check-in is guarded the same way. After the fix, the same five concurrent takes: one accepted, four refused, quantityonhand 0 and the ledger sum 0. Check-in also closes EVERY open row for the device rather than the newest. One device can only be out once, so a second open row is corruption, and closing only the newest left the older ones open forever - the device then read as returned while its custody history said otherwise. Separately, usbcheckouts now records who handed a drive back (checkinsso/checkinname, migration usb0004checkinidentity). Returning for a colleague is a supported flow that the kiosk asks to confirm, but the row it wrote attributed the check-in to the HOLDER, so the custody history and the audit report both named somebody who was not there. Nullable and deliberately not backfilled: copying the holder's badge into old rows would manufacture evidence for the exact claim this stops making. Four tests. Both guards verified by planting: strip the condition out of either UPDATE and its test fails. Scope: product |
||
|
cproudlock
|
405a2ef919 |
Cap request bodies, close a hole in the gate, fix to_dict and two settings pages
Six fixes that were ours regardless of what the sister site does. Four came out of our own reviews; two you reported from the settings pages. MAX_CONTENT_LENGTH was set nowhere. Every upload route caps its own payload, but each of those checks runs after the body has arrived: the first touch of request.form or request.files makes Werkzeug parse the whole multipart and spool the file to a temp file, so a route can reject the content and still have paid for the transfer and the disk. Three separate reviews found that shape in three different routes, which makes it a property of the app rather than of any handler. 128 MB, clearing the 64 MB agent binary with room for the envelope. The naming gate excluded devnotes with --exclude-dir, which matches a directory of that NAME at any depth - so a force-added plugins/<plugin>/devnotes/ was invisible to every check in the file, ADR-015 included. Replaced with a path anchored post-filter. Verified by planting the same site literal in root devnotes/ AND in plugins/endpoints/devnotes/: the gate now reports exactly one, the nested one. Before, it reported none. BaseModel.to_dict walked __table__.columns and published every one, so a model without an override published whatever it carried - including columns added later by someone not thinking about this method. User has no override, so User.to_dict() returned its bcrypt hash. Nothing calls it in a response path today, which is luck rather than design. A blanket allowlist would empty the output of all 34 models relying on the default, so this pins a floor instead: named secret-bearing columns are never emitted, and a model may extend the set. Eight tests, including a sweep over every mapped model. ActionPicker's Esc handlers had .prevent but not .stop, so the event bubbled to the wrapper and closed the whole picker instead of returning to the monitor list. The branding page reported "error loading image" on the site and badge logos. Two bugs stacked. The load failure was real: the preview used the stored value as a raw src, and that value is root-relative, so under /shopdb or /ops it resolved against the SERVER root and 404d - siteSettings.js wraps every logo in withBase() and this page was the one place that did not. With that fixed two of three were still invisible, because the shipped GE mark is fill="white" for the dark sidebar and was rendering white-on-white. The preview tile now sits on a mid-tone checker so artwork of any colour reads against it. The floor-map levels table ran off the right with its cells crushed - nine columns of inputs, no scroll wrapper, and .data-table is styled nowhere. My first attempt reached for the global .table-container and made it WORSE: that class also sets white-space: nowrap, and a full sentence of hint text lives in one of the cells, so nowrap forced the table wider still. It now scrolls in a local wrapper, the hint wraps, and the name input has a width instead of collapsing to one character. Suite 2377, frontend 324, gate green. Scope: product |
||
|
cproudlock
|
13be0ef90f |
Import the wilm agent store and WINC adoption work, with four fixes
Eleven commits from the sister site: a reversible Maximo/WINC machine-adoption import, a versioned agent binary store with upload and rollback, remote-BuildKit CI, tray version reporting, and UI fixes. Applied as a patch again - the published mirror rewrites hashes, so the branches have unrelated histories and git refuses to merge them. Reviewed by three agents before merging. Four things fixed here. THE PUBLISH WORKFLOW WAS DEAD. The new remote-BuildKit step in publish-images.yml sits at column 0 instead of indented under steps:, so the whole file is invalid YAML and GitHub would refuse it. Their entire GHCR distribution channel - which Start-ShopDBDev.ps1 and the compose files consume - was broken at the source and clearly never run. Indented; all five workflows now parse. THE IMPORT'S APPROVAL CHECK COULD NOT FAIL. commit() built the plan and the approval packet from the same upload and then asked the writer whether they matched, so packet_matches_plan compared a hash against itself. The documented contract - "a human approves a specific, frozen plan, and the writer can prove nothing drifted" - did not exist in the web path: an admin could preview file A and commit file B, gated only by the needs-create group NAMES they ticked. The preview's planhash is now required at commit and checked against a freshly rebuilt plan. Their tests committed without previewing, which is what let this through; they go through the real flow now, and three new tests pin it - all confirmed to fail with the check stubbed out. endpoints.agentpackage QUIETLY GREW. It used to mean "wrap the staged exe in a ZIP"; with the upload routes it also means "choose the executable every future package installs and runs as SYSTEM". Before this, staging needed filesystem access to the server. The permission description still said only "generate and download a package", so an admin granting it could not see what they were granting. Reworded. NOTHING RECORDED WHAT WAS STAGED, OR BY WHOM. The manifest held version, size and timestamp; re-uploading a version replaces the bytes in place under the same label. A sha256 of the stored file is now recorded, and the upload is audit-logged with it - after an incident there was previously no answer to who put that binary on the fleet. Also documented the nine new routes in docs/api-inventory.json (that gate cannot run on the mirror, so it falls to us at promotion time again), regenerated the spec and the project map, and kept our own docs where they conflicted: docs/ is scrubbed from the mirror, so their DEV-BUILD-TEST.md still had the dev-be channel mapping we corrected, and their PLUGIN-HOOKS.md was a new 45-line file rather than our 737-line canonical one. Their journal section is appended to ours. Contract version comes across as 0.30.0 for a genuinely additive surface. Suite 2369, frontend 324, gate green. Scope: product |
||
|
cproudlock
|
7f6aaf2e64 |
Gate the USB kiosk the same way, and make an unreachable setting a test failure
The USB kiosk had a feature flag and nothing else. A flag is a decision, not a credential: with the kiosk on, anyone who could reach the app could move a drive between checked-out and checked-in and sign it with a badge they guessed. Same reasoning that closed the parts kiosk, smaller blast radius - which is why Darkwing did not raise it and why it was worth asking about anyway. Both kiosks now share one mechanism. The device token moved out of the printedparts api wrapper into a request interceptor that attaches X-Kiosk-Token by URL prefix, with one localStorage key per kiosk - they are separate screens in separate rooms, and a shared token would mean provisioning one grants the other. claimKioskToken() is shared too, so both views take the token from a ?token= URL once and strip it from the address bar identically. Then the part that matters more than either fix. I told you to set the new settings in the UI, and that was impossible: every plugin settings page carries a hardcoded key list, and nothing had added them. printedparts_kiosk_enabled, printedparts_kiosk_token and usb_kiosk_token were declared, seeded and read correctly, and rendered by no page - changeable only by writing SQL. tests/test_plugin_settings_reachable.py now fails when any declared setting appears nowhere in its plugin's frontend. It found seven MORE that were already in that state, all pre-existing: collector_corporate_ranges computers_configpolicies_enabled computers_machinelink_alerts computers_quietreporthours geenforce_reportstaleminutes printers_dashboardpercent warranty_expiringdays All seven now have fields. I first parked them in a KNOWN_GAPS ratchet, then fixed them and deleted the ratchet - a backlog list in a new check is a way of agreeing the check does not apply yet. There is no backlog; the check is absolute. Two of my own mistakes on the way, both caught by tests rather than by reading: the blanket rename of the USB guard also rewrote the call inside _kiosk_open, so it called itself until the stack ran out; and the first version of the reachable check skipped all 17 plugins because it looked for get_settings() when the method is get_settings_defaults() - it passed by testing nothing, which is the exact shape it exists to catch. Suite 2336, frontend 316, build clean. Scope: product |
||
|
cproudlock
|
e4db112296 |
Use a valuetype the settings column actually documents
printedparts_kiosk_token declared valuetype 'secret'. The column documents string, boolean, integer and json, and that was the only 'secret' in the codebase - it worked only because Setting.get falls through to the raw string when it recognises nothing. Masking never depended on it: settings.py::_is_secret keys on the NAME containing 'token', so the value is still hidden in API reads and in the audit log. Changed to string, which is what it is. Scope: product |
||
|
cproudlock
|
744bc4341a |
Shut the parts kiosk behind a device token, and stop the health probe leaking
Both from the second Darkwing scan. Every claim in it was verified against the
code before acting.
The kiosk was the product's only unauthenticated write, justified by a decision
record saying it was "physically rate-limited: it serves a touch screen on the
shop floor". Nothing in code enforced the physical half, and an HTTP route is
reachable from wherever the app is. What that left open:
- _kiosk_find_item falls back to matching bare digits against the numeric
tail of every active item, so 1, 2, 3 walk the catalogue without knowing a
barcode;
- a badge is a short numeric SSO, and the route answered 422 for an unknown
one and 200 for a real one - an oracle for sorting guesses into employees;
- the write then decremented stock and stamped the ledger, which IS the audit
trail, with whatever SSO the caller supplied;
- quantity was bounded only by what was on hand, so one request emptied a bin.
A badge is attribution, never authorization. Both kiosk routes now need the
site's kiosk flag AND a per-device token, and 404 rather than 403 without them
so a caller does not learn the door is there. The READ is gated too: gating only
the write would have left the enumeration oracle running against a kiosk a site
had switched off.
The token is per-DEVICE, not per-page, because a kiosk is a browser on a touch
screen and a token served inside the page is readable by anyone who can load it.
An admin opens the screen once as /parts-kiosk?token=..., the page moves it into
that browser's localStorage and strips it from the address bar.
Also: quantity is capped per take, JSON true no longer passes as 1
(isinstance(True, int) is True), and every badge failure on the anonymous route
returns one message. The gated restock and adjust handlers keep the specific
message - there the person is known and the detail is help, not an oracle. I
changed the wrong one of those three handlers first; caught by the test.
/api/dashboard/health formatted str(e) straight into the body of a route
PUBLIC_ENDPOINTS declares open because "monitoring has no account". A pymysql
OperationalError names the database account, the app host's address, the driver
and the failing statement. It is logged server-side now and the body carries a
coarse status. The version goes with it - it told an anonymous caller which
release to look up. Status stays 200 either way: a monitor keyed on the body's
status field is the contract here, and turning a degraded database into a
transport failure is a separate decision from not leaking the exception.
Nine tests. The three gate tests were confirmed to fail with the gate stubbed
out, rather than assumed to cover it. Suite 2325, frontend 316, build clean.
Scope: product
|
||
|
cproudlock
|
e98e15259e |
Unpick one site's conventions from the shipped bundle, and bind the launch-path
check to the file it actually runs Two clusters the five-agent review found. Both are about a product that another plant has to be able to install. ADR-015: the shipped monitor bundle was the originating plant's fleet convention frozen into product seed. The business policy was seeded as "RPCA Shopfloor App Monitors". RPCA is that plant's facility designator, the row is source='shipped', and the API refused to rename it - so every adopting site would carry another plant's label at the top of its policy list forever. 0007 now seeds the neutral name, and 0010 renames it on installs that already stamped 0007, guarded so it cannot collide with a policy a site made itself. Downgrade is one-way on purpose: putting another site's designator back is not a state worth reaching. Which PC type the bundle attaches to was the literal 'Shopfloor PC'. A plant whose type is "Shop Floor Workstation" silently got no shipped monitors at all, and one using that name for something else got all of them. It is a setting now, defaulting to the seeded name so nothing changes here, and a blank value attaches the bundle to nothing - the right neutral answer for a site that does not want it. Shipped rows refused every edit, including isactive and launchpath. A plant whose eDNC is on D:, or that runs no UDC, could not correct the path or switch the monitor off, and then carried permanent false drift for it. ADR-015 allows seed data BECAUSE a site can edit it afterwards. Paths and the active flag are now editable; kind and displayname stay fixed, because that pair is the identity 0007's bundle matches on. Also corrected two false claims in 0007's docstring: the policy tables belong to the computers plugin, not the core baseline, and a guarded no-op does NOT reconcile later - Alembic stamps the revision either way, so the bundle would simply never seed. LaunchPathPolicy was purely lexical. GetFullPath is string normalization that never touches the disk, so the root check constrained the NAME while File.Exists and CreateProcess followed reparse points to somewhere else. A junction under a writable directory in an approved root - creating one needs no privilege, only write access to the parent, and vendor trees under Program Files are not reliably locked down - turned WebUI monitor-edit rights into code execution as the console operator. It now resolves the real target, leaf and directory chain both, and re-checks THAT against the roots, rejecting if resolution fails. Alternate data streams passed the whole policy on net8.0: an interior colon no longer throws in GetFullPath, so "readme.txt:payload.exe" reported an .exe extension, passed the root check, and existed. Rejected explicitly now. Six tests added for the shipped-row and pctype behaviour. The Core project compiles clean. The launch-path change has NOT been exercised against real junctions, ADS or 8.3 names - that needs the win11 VM, and per the reviewer it wants a planted-violation suite there rather than my say-so. Still open from the review: monitor revocation does not take effect until agent restart, ActionPicker is keyboard-inoperable, the lightbox has no focus trap, one attached monitor grants generic any-target process control, and screenshots are retained unencrypted in the user profile. Scope: product |
||
|
cproudlock
|
88a683dad3 |
Act on the five-agent review of the wilm import
Findings the reviewers verified, fixed here. The larger cluster they found is NOT fixed and is listed at the end. Frontend. reportImageUrl is called from the template, so it ran on every render with no in-flight guard: a report whose fetch was pending was re-requested, and because each resolution reassigns the map the whole grid re-rendered and re-fired the rest, costing up to N-squared requests for N thumbnails. Two blobs resolving for one key orphaned the first object URL where revoke could not reach it, and a failure was never recorded so a broken image retried forever. Guarded, deduplicated, failures remembered. Three CSS variables did not exist, so their hardcoded fallbacks always won: --bg-alt (the "No screenshot" tile, near-black in light mode), and --warning-text/-bg/-border (dark khaki on a dark modal). Replaced with real tokens. The DEV badge's literal #c62828 is var(--danger). Installer and launcher. The generated site installer ran docker compose with no -p, so the project name defaulted to the package folder - which carries the version. Installing the next version created a new project with new empty volumes and the site came up looking wiped, old data stranded under the old project. Pinned to shopdb-<channel>. Start-ShopDBDev deleted any existing -TargetDir that lacked a .git, recursively and without asking. For a script handed round as "save it anywhere and run it", a mistyped -TargetDir pointed at Documents would have destroyed it. It now only reclaims a directory that is empty or carries this launcher's own marker, and refuses otherwise. Dev secrets came from Get-Random, which is System.Random: a 32-bit seed, and all four values off one stream, so recovering one recovers the rest. They sign JWTs for a stack a pilot may point at real imported data. CSPRNG now, and the API port binds to loopback like the DB rather than every interface. Two of my own from earlier in this session. The operator-report size check sat below request.form, and the first access to that parses and spools the whole multipart - so it ran after the thing it prevents. Moved above. And the channel remap missed build-channel-installer.ps1 entirely, where it set BOTH channels to main: a "stable" package built from the trunk, the exact mislabel the remap was for. stable now resolves the newest v* tag. Also removed a zero-byte docs/ShopDB-Planner-Tasks-v2.csv that rode in. NOT FIXED - needs a decision, and it is the biggest thing the review found. Migrations 0006/0007 freeze the originating site's fleet convention into product seed: a shipped policy literally named "RPCA Shopfloor App Monitors", a hardcoded 'Shopfloor PC' type deciding who gets the shipped watch-list, and shipped rows the API refuses to edit or deactivate - including their launch paths. A site whose eDNC is on another drive cannot correct it and gets permanent false drift. Those are stamped migrations, so this gets harder to undo at every install. See the review notes before anyone upgrades a second site through 0006-0007. Suite 2314 passing, frontend 316, all three PowerShell files parse. Scope: product |
||
|
cproudlock
|
b5540e631c |
Merge the wilm endpoint-monitor and channel-distribution work into main
Six commits from wilm-import: the squashed import, route documentation, the channel remap onto our tag-and-trunk model, the project map, parallel test support, and the four smaller review findings. Merged rather than cherry-picked: the published mirror rewrites hashes, so the commits do not apply one at a time against our history even though the squashed diff applies clean. Scope: product # Conflicts: # docs/PROJECT-MAP.md |
||
|
cproudlock
|
720f4ddbb2 |
Close the four smaller findings from the wilm import review
None blocked the merge; all four were mechanisms that did not mechanise. _RUNNING_STATES existed twice, in api/routes.py and services/reconciler.py, with a comment asking a human to keep them in lockstep. The panel and the reconciler read the same agent snapshot, so a drift between the copies would paint a service green and flag it as drift on the same page. One definition now, owned by the reconciler and imported by routes - api -> services is the direction the plugin layout already runs in, and the reconciler's import of routes is deferred, so there is no cycle. The operator-report upload checked its 10 MB cap AFTER upload.read() had already pulled the whole body into memory, so the check could not prevent the thing it was there to prevent. content_length is now an early out and the read itself is bounded to cap+1, which is enough to know the cap was exceeded without taking a body that lied about its length. The multipart envelope carries the other form fields too, so the early out allows 64 KB of slack rather than rejecting an image that is just under. The same handler forced the extension to .png and the mimetype to image/png whatever the bytes were, then served the file back under that type. The upload headers (CSP sandbox + nosniff) make the document case inert, so this was never the hole it looks like, but nothing is gained by storing a file we already know is not what it claims. The PNG signature is now checked. The lightbox bound Escape and the arrow keys to a div and never focused it, so every one of those bindings was dead until the user happened to click the backdrop. It takes focus on open and hands it back to the thumbnail on close. On the two upload tests, honestly: the PNG one was confirmed to fail with the check removed. The oversize one passes either way, because the old code did reject an oversized image - it just did so after reading all of it. That test pins the response, not the fix. Whether the read is bounded is not observable through the response, so nothing here proves it; the change stands on reading the handler. Suite 2311 passing, frontend 316 passing, build clean. Scope: product |
||
|
cproudlock
|
f3a291a28d |
Stop a request-supplied host reaching the generated installer script
Found by the Darkwing scan of main (run 33654922078). Its own review dismissed
the finding as not remotely exploitable and it is right about that: nothing can
make a victim's browser attach a custom header, and deploy/windows/web.config
does no output caching. It still recommended the hardening, and the reasoning
for the dismissal is a property of OUR IIS config, not of the product - every
site runs its own instance behind its own proxy under ADR-004.
_batch_base_url() interpolated a caller-supplied host into the single-quoted
PowerShell -Uri literal of the .bat this route hands out. site_base_url ships
blank, so that fallback is the default path on a fresh install, not an edge
case, and GET /api/printers/install-batch is public.
Three rules now:
- the host is validated as a hostname before it is interpolated, so the quote
that would end the literal never reaches it,
- X-Forwarded-Host is only believed when TRUSTED_PROXY_COUNT declares a proxy
-- the rule shopdb/utils/clientip.py already applies to X-Forwarded-For.
That was centralised for this exact reason and did not cover the Host
variant,
- a request supplying neither usable header gets a relative base, which still
downloads correctly from the browser that fetched the script.
Three tests pin it, and all three were confirmed to fail with the old body
restored rather than assumed to cover it.
Scope: product
|
||
|
cproudlock
|
56ed89bb7b |
Document the ten new endpoint routes and undo the stray re-indent
Two of the three things holding the wilm import out of main. The monitor CRUD, the two resolver reads and the three operator-report routes had no entry in docs/api-inventory.json, so test_api_inventory_parity failed. The gate cannot run on the published mirror - the test ships but its data file is scrubbed, and it skips on the file's absence - which is why the branch that added the routes never saw it. Cost lands here at promotion time instead. /api/endpoints is registered both with and without a trailing slash and both forms are served, so both are described rather than guessing which one counts. Spec regenerated: 353 paths, 504 operations. plugins/endpoints/plugin.py had a contiguous run of continuation lines shifted by one space. Python accepted it and git diff -w showed nothing, but it put deletions in a diff that is otherwise purely additive. Restored; the file now differs from the published main by additions only. Scope: product |
||
|
cproudlock
|
27b2817fd7 |
Import the wilm endpoint-monitor and channel-distribution work
Everything on origin/wilm that is not on the published main, applied as one
change because the published history has rewritten hashes: per-commit
cherry-pick conflicts against our main even though the squashed diff applies
clean.
Contents: endpoint monitor catalog + policy attachment + warn-only drift
reconciler, operator issue reports from the tray, agent launch-path allowlist,
and the GHCR channel-image distribution path for dev workstations.
NOT READY FOR main. Open items:
- 10 new routes are absent from docs/api-inventory.json, so
test_api_inventory_parity fails. That gate SKIPS on the published mirror
(the test ships, its data file is scrubbed), which is why wilm never saw it.
- dev-bootstrap.ps1 and Start-ShopDBDev.ps1 hardcode the ge-aero GitHub org.
- plugins/endpoints/plugin.py carries a whitespace-only re-indent.
Scope: product
|
||
|
cproudlock
|
1480b30e8c |
Let a person edit the asset fields the pages already display
Eight fields across three asset types were shown on the detail page and accepted by the API, but had no input on the form. A wrong value therefore needed database access to correct, which is not a thing anybody should need for a device category or a maintenance date. The shape is the same in each case: a column was added, the detail card was written, the API allowlist was updated, and the form was missed. Nothing was broken - the values arrive by import or collector - so nothing complained. PCs gain azureobjectid and azuredevicecategory. azuredeviceid was already there, which is what made the omission easy to miss: the page showed two ids and the form offered one. The two are not interchangeable and the pairing is counterintuitive - the DEVICE id opens Intune, the OBJECT id opens Entra - so each input says where it goes. Machines gain the three maintenance fields. These matter beyond the immediate fix: they are the columns a measuring instrument's calibration dates map onto when metrology moves to machines, and until now there was no way to enter a calibration date on a machine at all. Printers gain iscolor, isduplex and isnetwork as a capability group. isnetwork defaults to true in the form to match the API's create default; a new printer claiming otherwise would be wrong for nearly all of them. Checked before making any of these editable that nothing writes them automatically - a hand edit silently reverted by the next collector poll would be worse than no input at all. The printers collector reports queues and supplies, not capabilities, and nothing writes machine maintenance dates, which is exactly why they are all empty. The two genuinely collector-owned computer fields, lastreporteddate and lastboottime, are deliberately still not editable. Scope: product |
||
|
cproudlock
|
c8e0de4dc1 |
Let the server decide which NIC is a PC's address, and keep the rest
A CMM reported 100.0.0.1 as its address and never reported 10.48.255.210 at all. Both halves of that were the same mistake. THE BAY WAS CHOOSING. The collector schema took a single `ipaddress`, so the script picked one NIC and dropped the others - preferring an address inside a configured range, else whichever interface carried a default route. A bay cannot make that judgement: it does not know the site's addressing. On this CMM the Marvell AQtion 10Gbit link to its controller is a real hardware NIC, up, and carries a default route of its own, so it satisfied every test the script had. It won, and the corporate address was discarded before the request was even built - which is why it is not in ShopDB even as a second row. So the bay now reports EVERY connected hardware NIC with a routable IPv4, and takes no view. The server picks the primary from collector_corporate_ranges, a new per-site setting, and writes a Communication row for each of the others. Naming the ranges is what turns the answer from a guess into a decision; with none set it still falls back to the default-route flag and then to the first reported, both guesses, both better than before only because nothing is thrown away now. Rows are keyed on the MAC where the bay sends one, because an address moves between adapters and a MAC does not. A NIC that is down at check-in is left alone rather than deleted - being down is not evidence the record is wrong, and hand-entered gateway, DNS and serial settings live on those same rows. Exactly one row ends up flagged primary, and only a row this report touched can claim it, so a check-in cannot silently demote an address set by hand. -AllowedRanges and the CollectorRanges registry value are now accepted and ignored, logging a line that says where the setting moved. The parameter stays because a scheduled task still passing it would otherwise fail on an unknown argument. A bay on an older collector sends one address and no interfaces; that keeps working and is pinned by a test. Scope: product |
||
|
cproudlock
|
241d8f40fc |
Add two commands for cleaning up computer types
Twenty computer types existed and six had a machine on them. Three of the empty ones were near-duplicates of live types - Shopfloor beside Shopfloor PC, CMM beside CMM PC, Engineer Workstation beside Engineer - which is the kind of list that grows because deleting from it by hand is nerve-wracking. retype-by-role types a PC by WHAT IT CONTROLS. A metrology box has an active relationship to a measuring_tool asset, so the fact is already recorded and does not need inferring from a hostname or a tool's name. The imaging scope is the authoritative source and stays so, but only 17 of roughly 292 PCs have ever reported one, so it cannot answer for the fleet yet. Relationships can: 49 PCs drive a measuring tool, and 36 of them are currently typed as ordinary shopfloor machines. prune-types deletes a type only when nothing is on it AND the live pctype map does not name it. The second guard is the one worth having: the map is what the collector assigns from at check-in, so deleting a mapped type would leave every future check-in logging 'mapped ComputerType not found' instead of typing the machine. Both report by default and write only with --apply, and prune prints why it is keeping each type it keeps rather than only what it removes. Also fixes `computers stats`, which counted the computer type named 'Shopfloor' - the EMPTY duplicate - rather than 'Shopfloor PC'. It has been reporting 0 shopfloor PCs for as long as both rows existed, which is the same duplication these commands clear up. Scope: product |
||
|
cproudlock
|
1bf4efa9e1 |
Link the Entra ids on a PC to the Azure portal
The Entra Device Info card showed two GUIDs and nothing to do with them.
Copying one into the portal by hand is the kind of errand a detail page exists
to remove.
Each link sits BESIDE the id it is built from rather than in a row at the top
of the card, because the pairing is counterintuitive: the DEVICE id opens
Intune and the OBJECT id opens Entra. A separate action row would have to
re-label which is which, and would still leave the reader matching them up.
BOTH URLS ARE SETTINGS WITH AN EMPTY DEFAULT, and that is the whole design.
The host is a tenant fact, not a product one - this site is on the US
Government cloud, portal.azure.us, where a commercial-cloud default would send
people to a page that does not exist. Microsoft also moves these blade paths,
and a site that hits that can fix it by editing a setting rather than waiting
for a release. Empty means no link and the id still renders as text, so a site
that configures nothing is unchanged. The example values live in each
setting's description, per cloud.
Same shape as the ServiceNow URL templates already in integrations, down to
the {token} placeholder, so there is one idiom for "deep link into a system we
do not own" rather than two.
Scope: product
|
||
|
cproudlock
|
2feb569686 |
Rename Metrics to Monitors and move it second
"Metrics" promised numbers. The tab holds tracked services, tracked processes and network interfaces - things being WATCHED and their current state, only one of which is a measurement. Monitors is what the panels actually are, and it matches the word the agent's own config already uses for them. Moved to second, ahead of Software. What is being watched on this box is the question asked right after "what is it", and well before its application inventory - which is reference data somebody goes looking for deliberately. Tab order is now Details, Monitors, Software, Activity. Contract 0.29.0 keeps its number. It is unreleased, so the group vocabulary it introduces is still being settled and nothing can yet depend on the old name. Both the server and the client fall back to 'details' for a group they do not recognise, so a site that deploys these halves out of step gets a misplaced card rather than a missing one. Scope: product |
||
|
cproudlock
|
c71183375f |
Fix four things found by looking at the tabbed page
SOFTWARE OPENED COLLAPSED. The installed-software panel was collapsible and started that way, because it used to be low-priority reference data near the bottom of one long page. It is now the Software tab's whole reason to exist, and somebody who clicked that tab has already said what they came for. Made it neither collapsible nor collapsed. TRACKED SERVICES VANISHED FROM METRICS. The group is decided by the SERVER, so a new frontend against a server still serving the old vocabulary puts a panel in a group the build does not render, and it is fetched, held, and shown nowhere - with nothing logged. Moving services out of 'actions' is exactly that shape. The client now falls back to 'details' for any group it does not render, the same way the server does, so a version-skewed pair degrades to a misplaced card instead of a missing one. A TABLE PANEL WITH NOTHING IN IT HID ITSELF. panelVisible honoured `empty` for list panels and ignored it for tables, for no reason anybody wrote down. A declared empty message is an answer - "No installed software reported." - where an absent card is a question. It matters more now that a panel can be the only thing in a tab, because the tab disappears with it. THE TAB STRIP HAD A VERTICAL SCROLLBAR. Per the overflow spec, when one axis is not `visible` the other computes from `visible` to `auto`, so `overflow-x: auto` alone made the strip scrollable vertically too and the -1px margin the tabs use to sit on the border was enough to trip it. Also moved .detail-full into the global stylesheet next to .content-grid, per frontend/CLAUDE.md: it is detail-page layout, and the other four detail pages need it when they adopt tabs. Scope: product |
||
|
cproudlock
|
f4b2ed7613 |
Drop the Actions tab, and give wide panels the width instead of a scrollbar
Four changes from looking at the built page. ACTIONS HELD ONE CARD. Tracked services was the only thing in it, and it was the wrong split anyway: a service row reads as state first and a Start/Stop second, so separating the buttons from the state they act on made a tab that answered no question on its own. Services move to Metrics, beside the rest of what the endpoint reports, and the group is gone from the contract. WIDE PANELS SPAN, THEY DO NOT SCROLL. Network interfaces carries adapter, address, MAC, speed and state; at half page width in the two-column flow it either squeezed every cell to three lines or offered a horizontal scrollbar. A scrollbar for data that just needs room is a layout failure. Table panels now take column-span, which is the multi-column layout's own answer, so short fact cards stay two-up around them. The wrapper keeps overflow-x only as a floor for a genuinely narrow viewport. SOFTWARE AND ACTIVITY RUN FULL WIDTH. Both are single long lists, and a list split into two balanced columns reads as two unrelated lists with no way to tell where the first ended. Details and Metrics keep the two-column flow because they are short cards, which is what that flow is good at. Contract 0.29.0 keeps its number: it is unreleased, and the group vocabulary it introduces has simply lost one value before anybody could depend on it. Scope: product |
||
|
cproudlock
|
2122c3b45c |
Tab the PC detail page, and let a panel say which tab it belongs to
The page had grown to roughly ten cards in one flow, and three more surfaces were wanted on it: installed software, live metrics, and the service controls. Stacking those into the same column would have buried what the page is for. Tabs were mocked and rejected on 2026-08-28, on the grounds that the strip duplicated the sidebar and cost hero space. That objection is answered by where the strip sits: the hero and the triage band stay above it and never move, so "is this machine OK" is still answered without a click, and the strip partitions the detail underneath rather than navigating anywhere. WHY A PANEL DECLARES ITS OWN GROUP Most cards on a detail page are not the page's. They arrive through the ADR-010 get_asset_panels hook, so a page that sorted panels into tabs by knowing their ids would defeat the hook it is built on. Panels now carry an optional group instead, and the page only knows the five group names. Resolution happens server-side, in the asset-panels route, so that an absent or misspelled group becomes 'details' before the client ever sees it. The client renders known groups only; an unrecognised value that survived to it would be a panel that exists and is displayed nowhere. Contract 0.28.0 -> 0.29.0, additive - every existing plugin keeps working untouched, and the endpoints status panel is deliberately left ungrouped so the default path is exercised by a shipped plugin and not only by tests. A tab is offered only when something fills it. An empty tab reads as broken rather than as "this site does not collect that", so the strip is built from the groups that actually returned data, and a tab that disappears on refetch falls back to Details rather than leaving the page blank. AssetDetailTabs is core, not a PCDetail component: machines, printers, network devices and measuring tools all have the same shape, and the shared asset-detail shell they need starts here. Scope: product |
||
|
cproudlock
|
ae7b30a056 |
Stop naming remote-access schemes as if they were constants
A link is built by formatting that protocol's own linktemplate, and the
templates are rows in accessprotocols that a site can edit. So no scheme
is a fixed fact about the product, and writing one down as though it were
misleads whoever reads it next.
Two places did. The guest-grant docstring said the detail route resolves
"working rdp:// and vnc:// links", which is wrong for the row we actually
ship: VNC is com.realvnc.vncviewer.connect://{host}:{port}, the RealVNC
handler, not vnc://. And the protocol admin form offered vnc://{host}:{port}
as its placeholder, in the one screen where somebody authors a new template,
so the example contradicted the seeded data it sits next to.
Neither changes behaviour. _computer_access_links already reads the
template from the database and the frontend only ever renders the resolved
link, so nothing in the code assumed a scheme - only the prose did.
Scope: product
|
||
|
cproudlock
|
723a4ff19b |
Declare the guest grants that already shipped
Inverting the grantable rule from the .view suffix to explicit declaration dropped every permission that had qualified by how it was spelled. Two of them were shipped guest features with grants already in the field. guides.view: the nav item and list_guides both reason in comments about a signed-out reader holding it, and the route is written around one - drafts are filtered out, search matches the published revision so draft wording cannot leak through a hit, and the author branches cannot open for a caller with no user id. tools.view: without it a guest holding tools.secrets has permission to use Secure Send and no way to reach the page, because tools.view is what puts the entry in the nav. The tools themselves run in the browser and touch no data. The failure was worse than a missing checkbox. update_role refuses the WHOLE request when any submitted permission is undeclared, and the role grid resubmits what the role already holds, so a site that granted either of these under the old rule could no longer save the guest role at all - including to tick assets.view, which is the grant the shop floor needs to look up an asset number without signing in. The test asserts both stay grantable, so narrowing the set again means looking at what is already granted in the field rather than only at what the catalogue declares. Scope: product |
||
|
cproudlock
|
dd52666d6b |
Order the supplies card by how empty the cartridge is
Rows sorted on iscritical then on printer NAME. Critical is everything
at or below 5%, so a cartridge at 5% and one at 0% tied on the first key
and the alphabet decided which came first.
Wrong order on its own, and worse than it looks: the dashboard queue
keeps only the first few rows per source, so a printer that was
genuinely empty was truncated off the card while a fuller one held its
place. Reported from the floor as "starts at 5% and hides one at 0%",
which is exactly the two bugs compounding.
remaining is already normalised for waste cartridges - a full waste bin
reads as low remaining - so one ascending sort is worst-first for every
supply type, and iscritical falls out of it rather than needing its own
key. Name only breaks ties.
worstremaining is carried on the row because the supplies list is
rendered to text ("Black 0%") before it reaches the client, so the
number the sort needs is gone by then.
The toner forecast report was already worst-first (it sorts on daysleft)
and is unchanged.
Scope: product
(cherry picked from commit
|
||
|
cproudlock
|
dbc43162c5 |
Pin the approved content digest on an endpoint command
Delivery descriptors for run_script and update_app are resolved when the
agent claims a command, not when an operator queues it, and the command
row recorded no digest: the enqueue path resolved the descriptor purely
to validate it and threw the result away.
So what an operator approved and what an endpoint runs were two separate
lookups against mutable content, minutes or hours apart. Anyone able to
replace a script body or an installer package could do so in the gap.
The claim then re-resolves, the descriptor faithfully describes the new
bytes, the agent verifies correctly, and it runs as LocalSystem. The
audit trail names the operator who queued the original.
The gate on the verb and the gate on the payload were also different
grants: pushing needs endpoints.deploy, but replacing what gets pushed
needs only applications.edit.
pinnedsha256 records the digest as it stood at enqueue, and the claim
refuses a descriptor that has moved, failing the push with an audit row
instead of shipping it. Null for actions that deliver no content, and
for rows queued before this migration, which claim as before. A
descriptor that vanished entirely is not treated as tampering: that path
already sends the command through undescribed so the agent fails closed.
Separately, a result may now only finalize a command that is actually
running. That needs no agent change and closes two things: reporting on
a command that was never claimed, and overwriting a recorded failure
with a success.
The agent now sends its hostname on the result callback so the server
can check it owns the command it is finalizing. The shared collector key
does not say who is calling, so without it any agent can post a result
for any command id. The server treats the parameter as optional for
agents that predate this and enforces it when present, so there is no
flag day. This is a floor, not a fix: only a per-endpoint credential
makes the caller's identity provable, and that is tracked separately.
The .NET side is unbuilt here and needs a build plus a win11 VM run.
Scope: product
(cherry picked from commit
|
||
|
cproudlock
|
ccd8221adf |
Key the client address on the connection, not a header the caller sends
client_ip() read the first X-Forwarded-For hop unconditionally. The
login rate limit and the account lockout are keyed on that value and the
audit log records it, so rotating the header per request bought an
unbounded budget and an untraceable trail.
This walked around TRUSTED_PROXY_COUNT, which exists to decide exactly
this. The app leaves ProxyFix off by default and says why, ten lines of
comment away: ProxyFix trusts X-Forwarded-*, a client can send those
itself, and the login limiter is keyed on the address. Then client_ip
read the header anyway, so the stated protection never applied.
Resolution is now from remote_addr, which ProxyFix has already filled
from the header wherever a proxy is genuinely configured. Both cases end
at the same trustworthy source, and the port still comes off because ARR
forwards clientip:port and ProxyFix passes it through verbatim.
geenforce and tools had each hand-rolled the same mistake for their own
limiters. Rather than duplicate security-relevant logic a third time,
client_ip and strip_port join the shopdb.api surface (contract 0.28.0).
geenforce's _trusted_client_ip, which already refused the raw header for
its IP allowlist, is untouched and was the model for this.
The stability table was also missing rows for 0.26.0 and 0.27.0.
INSTALL-WINDOWS-IIS.md never mentioned TRUSTED_PROXY_COUNT, so a site
installed from it runs behind IIS with the setting at 0: every request
looks like it came from IIS, the whole site shares one rate-limit
bucket, and the audit log records the proxy for every action. It now
says so, and says not to set it when nothing is in front.
Three test modules simulated separate callers by setting
X-Forwarded-For, which is the header a real attacker rotates to escape
the limit. One of them, test_client_ip_prefers_forwarded_first_hop_
stripped, asserted the defect outright. The suite could not have caught
this. They now drive REMOTE_ADDR, with a regression guard that forging
the header buys nothing.
Scope: product
(cherry picked from commit
|
||
|
cproudlock
|
85e8a7b877 |
Decide guest grants explicitly instead of by permission name
guest_grantable_permissions() opened any permission whose name ended
'.view' or '.read' to the guest role, on the reasoning that reading is
the safe default. That is true of a permission gating a list of asset
numbers and false of one gating bytes, and a suffix cannot tell them
apart. Twenty-four of eighty-five qualified and none had been decided:
applications.view reached the SYSTEM-context PowerShell bodies and
the installer binaries pushed to bays
backups.view returned the full payload that the separate
backups.download permission exists to protect
assets.view gated a global search that queries the staff
directory, USB history and network inventory with
no per-domain check of its own
audit.view usernames, addresses, the before/after of every
settings.view change, and every setting row - both admin
category, opened by naming alone
users.view gates nothing at all; list_users is
require_role('admin')
Every entry is now declared, by CORE_GUEST_GRANTABLE or by the plugin
that owns the permission, having read what its routes actually return.
Adding one is a code change, which is the point: it puts the decision
in a diff rather than in a suffix.
The two routes serving script bodies and installer packages move to a
new applications.download permission, which is not grantable, so a site
can still open the catalogue metadata to its floor without the bytes
going with it.
The declared set preserves the grants this site had already chosen:
applications.view, computers.view, endpoints.view, reports.view,
tools.secrets, warranty.view.
Two known gaps are recorded on the hooks that expose them rather than
silently carried: the computer serializer publishes every column plus
resolved rdp:// and vnc:// links, and the per-asset endpoint panels are
keyed on a small integer so walking ids reconstructs much of the roster
that endpoints.fleet exists to protect. Neither is widened here.
A shipped test was granting guests audit.view and passing.
Scope: product
(cherry picked from commit
|
||
|
cproudlock
|
5f27f81e88 |
Clear the review blockers on the merged wilm work
A multi-agent review of the merged branch produced 29 findings that survived
adversarial verification. This clears the blockers and the four worth fixing
before it reaches main. The design held up well: path-traversal defence on all
four serving routes is clean, the agent's SHA-256 check fails closed, enqueue
authority is proportionate and audited, and no site literals or credential leaks
in the new C#.
THE TESTS NEVER RAN
pytest.ini sets testpaths = tests, so plugins/endpoints/tests/ was collected ZERO
times. Fifty-five lines of delivery assertions were dead code, and run directly
five of them failed. plugins/printedparts/tests/ had been invisible for longer.
That is the worst shape a gap can take: the suite is green, the coverage is
imaginary, and the collector auth on a download route could be deleted without
anything going red.
Both files moved under tests/. tests/test_no_hidden_tests.py now fails if a test
file appears anywhere pytest will not look; it caught a staged copy inside the
installer bundle on its first run, and was verified by planting a violation.
The five failures were stale expectations, and they split by cause. His: the
applications panel made six panels where his test said five, and 'applications'
belongs in KNOWN_PAYLOAD_KEYS now that it is persisted - his own test contradicted
his own feature. Ours: we renamed the events card to activity, moved the
fleet-wide cards to endpoints.fleet, and made the list route a bare route('')
which Flask rejects unless the blueprint is mounted under a prefix.
FOUR FIXES
The snapshot table's key was a 32-bit autoincrement on a table rewritten
wholesale every check-in, so it climbs at (endpoints x apps) per report and
exhausts on MySQL. Widened while the table is unreleased. Widening it broke every
insert on SQLite, which only autoincrements a column typed exactly INTEGER
PRIMARY KEY, so it carries a dialect variant - and nothing caught either fault,
because no test inserted into that table at all. One does now.
Ingest truncated displayname but not displayversion or publisher. On strict MySQL
an over-long registry value fails the whole check-in transaction, so one machine
with a malformed Uninstall entry goes silent entirely. SQLite does not enforce
lengths, so the suite would never have shown it.
update_app's version: branch pinned AppVersion.sha256, but that route resolves the
version to its APPLICATION and serves application.installpath - so the digest
described an artifact the route never sends, and where it was null the agent had
nothing to verify and failed open. It now pins the file actually served.
The same branch ended the delivery URL at the id, so an MSI arrived with no
extension and could not be routed through msiexec. It carries the filename now.
Agent packaging moved off require_role('admin') to a new endpoints.agentpackage
permission. The generated ZIP carries a live collector key, so this is a heavier
grant than deploying to one endpoint - and a role cannot be given to one person
for one job the way a permission can.
Full suite 2231 passed (2198 before, plus the 22 that never ran and 11 new),
frontend 314.
|
||
benjamin.baker
|
4590156f30 |
Add endpoint software delivery: run_script and update_app push
Adds the two delivery actions to the endpoints command allowlist, each gated on a new endpoints.deploy permission ON TOP OF endpoints.command so pushing software is a heavier grant than a routine toggle. Enqueue validates update_app appversionid ownership and installer presence before queueing. New collector-authenticated download routes let the agent fetch installer packages and script bodies by digest rather than pre-staging them or sending bodies over the command wire. Wires the scriptsApi client, the siteagent generator routes, and a temporary dev-only Delivery test bench (view/route/nav) for exercising the path while the production UI is built. |
||
|
benjamin.baker
|
5f8061c55d |
Self-heal endpoint command queue wedged by a dead in-flight command
The command queue is at-most-one-in-flight, but nothing ever expired a stuck command. When an agent claimed a command (RUNNING) then died before reporting a result - killed, uninstalled, or offline mid-flight - the row stayed in-flight forever and every operator action for that endpoint returned 409 'already in flight'. Reap in-flight rows past a staleness ceiling (15m for RUNNING, 60m for PENDING) to a new STATUS_EXPIRED at enqueue time, inside the same transaction, so the queue self-heals lazily the next time someone queues work. No background job and no schema migration - status is free-text. |
||
|
benjamin.baker
|
902339baf9 |
Persist and surface endpoint installed-software (SBOM)
The endpoint agent already reports an installed-software list (SBOM), but ingress dropped it: the collector schema had no entry for it and there was nowhere to store it, so operators could not see what is installed on a PC. Persist it as a per-asset snapshot in a new endpointapplications table, replaced on every report (current inventory is what operators ask for, not install history). Add applications to the collector schema and KNOWN_PAYLOAD_KEYS so ingress stops discarding it, a read route for the raw collection, and a collapsed-by-default Installed software panel to surface it. appid is nullable groundwork for a future Managed Applications link; no matcher is wired yet. |
||
|
benjamin.baker
|
7040832565 |
Add agentless adoption tool for endpoints
Standalone one-shot enrollment tool that creates a ShopDB PC + Endpoint entry using only the endpoints collector key, reusing the existing collector adopt path. Collects identity, WMI hardware, and installed-software SBOM; requires admin. Server-side packaging route attaches to the existing endpoints blueprint; no core contract change. |
||
|
benjamin.baker
|
cfb0b9ca4e |
collect installed-software SBOM from endpoint agent
The agent now reports installed applications from both Uninstall registry hives (64-bit and WOW6432Node) as {displayname, displayversion, publisher}, mirroring the Assess-Orchestrator applications collection. Registry-only, no Win32_Product/WMIC self-repair and no AppX. The server logs it as an unpersisted key until the SBOM store is built.
|
||
|
benjamin.baker
|
57845e1a9a |
log ignored agent collector keys for later UI enhancements
The endpoint agent now sends disk, bios, memory, processor, osinfo, and domain telemetry. The server does not persist these yet, so ingest logs any unpersisted payload keys with a compact preview. This makes incoming telemetry observable in the logs before any UI or schema work, so we can confirm the agent is sending it and enhance the UI later. |
||
|
cproudlock
|
b5fc5c01d1 |
Stay quiet when a site never asked for supply monitoring
Splitting "cannot see the cartridges" from "nothing is low" was right, but it treated two unlike cases as one. isconfigured means enabled AND a URL AND a token, so its absence means the site did not ask for supply monitoring at all - and a permanent "Supply monitoring is not configured" line is a card reporting nothing wrong every day, which is how a board teaches people to stop reading it. It matters for sites adopting ShopDB without Zabbix, who would otherwise inherit a warning about a feature they declined. Not configured now returns a plain empty and the card hides. Only configured and unanswerable reports itself, because that is something meant to work that is not working. Also withdraws an unfounded inference. A dev box has no route to Zabbix by design, so "not reachable" there is the expected state and says nothing about what production can see. The value of this change is not that it revealed a production fault; it is that when production's Zabbix does go down, the board will say so instead of displaying "Low Supplies 0". Four tests pin the three cases apart: unreachable reports, unconfigured stays quiet, and a real zero from a reachable Zabbix is not flagged as blindness. The fourth holds the envelope's rows, since carrying a reason changed the response shape. Verified they are not vacuous by restoring the original branch in place - the unreachable test fails against it. Full suite 2182 passed. |
||
|
cproudlock
|
b80bd8731f |
Filter the queue by check, and say when a check cannot answer
TWO CONTROLS, ONE BOARD Severity answers "how bad". It could not answer "what kind", and on a fleet where eight endpoints go quiet at once, being able to set those aside and look at the four crashes is the difference between a board and a list. Chips per contributing check, coloured by its severity, composing with the severity rollup above them. Both narrow what is already loaded rather than refetching, so no combination can disagree with the counts. Picking a severity that excludes the active chip clears the chip: two controls lit above an empty queue, neither obviously wrong, is worse than losing the narrower one. A chip counts every row, not the rows the queue had room for. A chip reading 5 above a "3 more" line would be its own small lie. "LOW SUPPLIES 0" WAS NOT TRUE Asked what happened to toner replacement, and the answer was worse than the layout question that prompted it. _get_low_supplies_data returned the same empty result for three different situations: Zabbix not configured, Zabbix unreachable, and every cartridge full. The dashboard strip therefore displayed "Low Supplies 0 / Critical 0" while the monitoring was dead - and this instance is exactly that case, with zabbix_enabled true against a URL nothing answers on. So the strip removed in the previous commit was not information. It was reassurance, and it read identically whether the printers were fine or nobody could see them. That is the failure this codebase already names elsewhere: an empty card and a broken card must not look the same. The endpoint now distinguishes the three, and returns an envelope rather than a bare list so a card can say WHY it has no rows. cardVisible shows such a card whatever its empty rule, because "I cannot tell" is not "nothing to report". It renders as a notice line, not a queue row: there is no asset to act on, and burying "the monitor is dead" among the things it failed to find would be the same mistake in a new place. Note what this does NOT fix: the Zabbix URL on this instance is a dev placeholder. The board will now say so instead of implying the cartridges are full, but somebody still has to point it at a real server - and it is worth checking what production points at, since nothing would have shown there either. Frontend 314 passed (13 new), full suite 2178. The response shape changed from a list to an envelope; no test depended on the old shape. |
||
|
cproudlock
|
13c46af5ce |
Rebuild the dashboard around what needs a person
The board was one card per source in a fixed grid, and both halves of that hurt. Every source got the same small box whatever it had to say. Warranties showed 5 of 50 behind "and 45 more"; the crash card truncated at 4; a one-row card sat beside a four-row card and left a hole. Nothing had room, which is the complaint that started this: not enough space to relay the data. And a box per source flattened priority. An enforcement failure and a warranty notice looked equally urgent because each owned a card, while the largest type on the page belonged to counts that are true every day and need nobody. WHAT THE PAGE IS NOW A severity rollup, one worst-first queue of everything that needs a person, and a row of context beneath it. The queue merges every critical and warning row and sorts them, so reading order is work order. It is full width, which is where the room came from. Rows are capped per source, because fifty expiring warranties must not bury two crashed PCs, and what did not fit is named per source with a link to it - a bare "and 45 more" that leads nowhere is worse than saying nothing. The rollup counts ROWS, not cards, and filters the board in place rather than navigating away, so the counts and the rows cannot disagree. Severity, position and permission were already declared per card and already sorted on. The old layout threw that away; this only stops it. WHAT WENT, AND WHY IT SHOULD NOT COME BACK Four asset tiles plus an "All assets" tile: machines + PCs + printers + network IS all assets, so the page showed a total beside its own addends at the same weight. It is now a line under a rule, where a total belongs. The printer strip. It carried the printer count a third time - the tile row and "Total Printers" were already the same number - and "Online" equalled total, so three tiles held one fact. A genuinely low cartridge already reaches the queue through the printers-supplies card, which is the moment anyone wants it. Three cards whose whole job was to display 0. Recent Devices: the largest block on the page, listing five machines that were all "In Use" with business unit "TBD". Recency is not a reason to look at something. CONTRACT 0.26.0: 'shortname' Cards may declare a short label for their rows in the merged queue. Their 'title' is written as a heading - "Endpoints with crashes or shutdowns" - and as a row badge it ran wider than the hostname it qualified, putting the badge rather than the asset at the start of every line. CSS-truncating the title was tried first and was worse: four consecutive rows reading "ENDPOINTS WITH CR..." say nothing at all. Optional and defaults to the title, so a plugin that does not declare one is unaffected. Additive field, minor bump. ALSO: "silent 1 days" A meta suffix is written for the plural case and the card cannot know the value. One row saying it was a shrug; the merged queue puts every quiet endpoint on one screen, so it said it five times in a column and read as broken. Now singularised at exactly 1, on the first word, keeping what follows: "1 event (48h)", "silent 1 day". The test asserting ' ms' survives is what caught the first version turning it into ' m'. DashboardRow.vue is extracted because the queue and the context panels render the same row, and the map tooltip, link fallbacks and chips are too easy to get subtly different in a second copy. Verified on screen against live data. Full suite 2178 passed, frontend 301. |
||
|
cproudlock
|
78017eee2a |
Put the dual-bay sibling in the triage band, and fix the gage counterpart
TWO STRIPS OF CHROME SAYING THE SAME KIND OF THING
The machine page opened with a banner - "Dual-bay machine - sibling bay: 2022" -
directly above a hero whose band said "Controlled by". Both answer "what am I
looking at", and splitting them across a banner and a band cost a strip of
vertical space to say so.
The sibling is now a chip in the band beside the controlling PC. The band takes
an `extraitems` prop rather than learning what dualpath is: the pairing is
resolved by the machines plugin, not from the relationships endpoint, so core
cannot derive it, and a dualpath-shaped prop on a core component would be the
machines plugin leaking upward. A page passes [{label, links:[{text, to}]}] and
the band renders it with its own markup, which also keeps the chips visually
identical to the ones it derives itself.
The label stays "Dual-bay sibling" rather than "Dualpath". The banner spelled
the meaning out because a dualpath pair is ONE physical machine in two bays; a
chip reading "Dualpath" would have dropped that.
THE GAGE PAGE FILTERED FOR A COUNTERPART THAT CANNOT EXIST
MeasuringToolDetail was wired counterparttype="machine" labelled "Used on".
Wrong on both counts. The import synthesizes one measuring-tool asset per
metrology PC and a 'controls' edge PC -> tool (stage_metrology), so a gage's
counterpart is a COMPUTER, exactly as on the machine page. Filtering for a
machine peer meant the band rendered nothing on every gage that has a PC.
It looked merely empty rather than wrong because this dev database holds one
measuring tool with no relationships at all, so there was nothing to be missing.
A hand-held gage genuinely has no such edge and correctly shows nothing; a
metrology station now shows the PC that drives it.
The relationships card stays. It is the only surface with Add Relationship and
the per-row delete, and it lists types the band filters out - the band is
triage, the card is management. It does now repeat what the band shows, which
is worth revisiting separately.
Verified on screen. Frontend 280 passed.
|
||
|
cproudlock
|
c22c81afa5 |
Show the controls link from both ends, and fix the dead chip links
THE RECIPROCAL LINKS BUG A PC's page showed "Serves machine 7803" at the top. The machine's page showed nothing equivalent, though the same edge was right there in the relationships card at the bottom. That asymmetry read as missing data and was reported as such. It was neither missing data nor an API fault. The endpoint returns both directions and always did; AssetTriageBand already walked outgoing AND incoming and deduped, with a comment explaining that production holds reciprocal rows. It then discarded every peer whose asset type was not 'machine', and hardcoded the label "Serves machine". So the component could only ever work from the PC side, and it was mounted only on the PC page. The counterpart type and label are now props, defaulting to the old values so the PC page is unchanged. The machine page mounts the band with counterparttype="computer" and the label "Controlled by". Eliminated with evidence rather than assumed, because this bug had already been misdiagnosed twice: the endpoint runs two queries and returns both arrays; the generic card renders both; that route has no pagination to clamp; and a single permission gate cannot admit one direction and not the other. THE RELATIONSHIP TYPE WAS ALSO HARDCODED Not every asset pair is linked by 'controls'. A machine reaches a printer through usesprinter and defaultprinter, so the band was blank on the printer page - the one page where the link is not obvious from the asset itself. The types are now a prop, and the printer page passes its own. THE CHIPS WERE DEAD LINKS Found while adopting the band more widely. It linked to /assets/<assetid>. There is no /assets/:id route and no catch-all, so every chip in the band was dead - and this change would have propagated that to four more pages. AssetRelationships.vue already resolved this correctly and privately. That function is now frontend/src/utils/assetRoute.js and both components use it, so a third caller cannot get it wrong again. Two things it gets right that are easy to miss: the detail routes key on pluginid, not assetid, so passing assetid opens the wrong record; and the lookup is now case-insensitive, because csvimport.py seeds 'Computer' and 'Machine' capitalised while this site's database holds them lowercase - a case-sensitive match produced dead links at whichever site seeded the other way. The band now carries pluginid and assettypename on each chip, which the resolver needs. TESTS assetRoute had none and neither did the band. Seven cover the resolver, including the assetid-vs-pluginid confusion and the seeding-case difference. On the backend, nothing asserted that the TARGET of a directional edge receives it in `incoming` - only that rows carry isdirectional. That missing assertion is this bug's exact shape, so it is now pinned: the endpoint was right all along, and the next such report should not cost anyone a re-diagnosis of the API. Verified on screen: the machine page shows "Controlled by G6K76CW3ESF" in the hero, the PC page is unchanged, and a page with no such link renders no empty band. Full suite 2162 passed, frontend 280 passed. |
||
|
cproudlock
|
d4105c2289 |
Say what a service state actually is, and make the badges readable
Three colour faults on the PC page, each hiding the next. ACTIVITY BADGES WERE ALL GREY. The list renderer reads a badge colour from a ROW FIELD, and the activity panel declared none, so every severity rendered identically. The feed's whole job is to make a critical event visible and it was painting it the same as a logon. The route now emits severitycolor - info grey, warning yellow, critical red - and the panel declares it. PAUSED AND THE TRANSITIONAL STATES WERE RED. statecolor was green-or-red-or- grey, so a paused service looked like a stopped one. They are different: a paused service was paused deliberately, and one stuck in Start Pending is failing to start on a retry cycle - which is exactly what raises event 7011, the thing behind the critical counts on the Fleet page. Both are amber now. NAMED COLOURS ALWAYS GOT WHITE TEXT. readableText only parsed hex and returned white for anything else, but panels emit NAMES. Green, red and grey are dark enough that white worked by luck; amber is not, so the fix above shipped as white-on-yellow and could not be read. Names resolve to hex before the contrast test, so this holds for any named colour anywhere in the app rather than these two panels. colorStyle had no test file at all. It has one now. Every one of these was found by looking at the rendered page. None of them would fail a test that checked markup. Full suite 2131 passed, vitest 273 passed. |
||
|
cproudlock
|
60c35728e8 |
Import USB records off an external cmmc_usb database, so a site can retire it
A site running external mode keeps its devices and check-in/out history in a separate cmmc_usb database. There was no way to bring that in, so the choice was to keep the second database forever or lose the history. This adds 'flask usb import-external' - dry run by default, --commit to write - which reads through the same connection settings the routes already use and never writes to the source. Two things it has to reconcile, and they are why it is a command rather than a few INSERTs. The schemas disagree about SHAPE. External is an event log, one row per action; self-hosted is a paired row carrying both times. Pairing walks each device in time order, and real logs after years of use are untidy. The rule is that nothing is invented and nothing vanishes: a drive still out becomes an open checkout and is NOT flagged (that is just a drive somebody has), a return with no matching issue is skipped and named, and two issues in a row leave the first open rather than inventing a return time nobody recorded. Identity is the other. usbdevices.serialnumber is unique and NOT NULL and the external devices table has no serial, so device_id becomes it. Blank or duplicated ids stop that device rather than inventing a key. external device_owner has no column here, so it is appended to the device notes rather than dropped. Also adds usbcheckouts.scannedviruses, the one field with nowhere to land. waswiped already means what external calls sanitized - selfhosted.py was mapping it - but scanned_viruses was HARDCODED None in the log output: the API promised the field and had nothing behind it, so an imported history would have rendered blank. It now reads and writes properly. Nullable, never defaulted to false. An unrecorded scan is not a failed scan, and for removable-media records those are very different claims. Full suite 2131 passed; seven new tests pin the pairing rules. |
||
|
cproudlock
|
99c40ae7c3 |
Say WHICH critical event, and stop the feed hiding it
Reported from production: health showed '418 critical' while the ten most recent events were all info. Both were true, which is the problem. The health rollup counts critical events in a 48h window; the activity feed shows the fifteen most recent events of ANY severity. A box that logged 418 service timeouts overnight and ordinary logons since produces exactly that pair - a page contradicting itself with no way to tell why. Two fixes. The rollup now names the DOMINANT event instead of totalling: 'Service timeout x418' rather than '418 critical'. Counting by event id costs nothing (one more GROUP BY column) and turns a number into a lead. This is the same reasoning that made health a phrase rather than a score; it stopped one step short. The activity feed now fetches the newest criticals separately and merges them, and keeps up to five of them when trimming to the cap. They hold their place in time but survive the cut, so what health is reporting always appears in the feed that exists to explain it. Verified against the reported shape: 20 service timeouts two hours old under 12 newer logons. Before, the feed was fifteen logons; now it carries five critical rows and health reads 'Service timeout x20'. Worth noting the underlying signal is real - several hundred criticals on one endpoint in 48h is a box crash-looping or a service failing on a retry cycle. The display was hiding it, not inventing it. Full suite 2124 passed, naming and parity gates green. |
||
|
cproudlock
|
b1b9cdda72 |
Add bulk actions to the Fleet list, with a confirmation that names machines
Select endpoints, then Collect now or Restart across the selection. Restart is
gated on endpoints.restart, both are re-checked server-side.
The confirmation names the MACHINES, not just the hostnames:
Restart 2 computer(s)?
GJX9B2Z3ESF, HOST-7
This STOPS the machines these PCs serve:
DT Office
A dialog listing eight hostnames tells an operator nothing about what they are
about to halt, and that is the whole consequence of the action on a shop floor.
Where no machine is recorded it says so, because 'none listed' and 'we did not
check' are different things at 6am. Only possible now the Machine column exists.
A 409 is reported as a SKIP, not a failure. The queue permits one command in
flight per endpoint, so a bulk action across eight PCs where two are already
busy says '6 queued' and '2 skipped - a command is already in flight'. Folding
those into an error count would make a working button look broken.
Selections clear on every reload: the rows they referred to may not be on the
new page, and acting on an invisible selection is how bulk tools surprise
people.
No per-row buttons. They were built first, and a tenth column on a nowrap table
pushed itself off the edge - selecting one row costs a single click and leaves
one command path instead of two.
Full suite 2124 passed, vitest 260 passed, naming gate green.
|
||
|
cproudlock
|
057cee1f02 |
Match the Fleet tile radius to the global card
The tiles were 6px while .card is 0.25rem, which left Fleet marginally rounder than every other page. The global card treatment is a separate decision and has been deferred; until it is made, this page should look like the rest of the app rather than half-way to a mockup. |