64e932ce5c9675d2f788fae82fff5aaec41bf529
148 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
6e36df841e |
Import the wilm endpoint telemetry, PC Detail rework and backup overlay
Ports 37 of Benjamin Baker's commits from the wilm branch, mirrored from the
corporate GitHub at
|
||
|
cproudlock
|
a1802a2bf5 |
Match the number on the bin, and let an unset kiosk say so
The last two kiosk findings. Keypad entry collected EVERY digit in an item's code, and the shipped prefix contains one: '3DP0004' collapsed to 30004, so an operator typing the 4 printed on the label was told no part matches that barcode and had to work out that the leading 3 belonged to the prefix. The number on a label is the number at the end, so the trailing digit run is what is compared now. An ambiguous number is still refused rather than guessed - taking from the wrong bin is worse than being asked to scan the label. A parts kiosk that had never been provisioned drew its normal "Scan the barcode on the bin" screen and answered "Not found" to every scan, which reads as "we do not stock that part" rather than "this screen was never set up". The server cannot say which: the routes answer a bare 404 on purpose, so a caller without the device token does not learn the door exists, and that stays. But the BROWSER knows whether it holds a token, and saying so discloses nothing - an unprovisioned screen now says it is not set up and how to fix it. For the case the client genuinely cannot distinguish - kiosk switched off versus a part that is not stocked, both a 404 by design - three misses in a row add a line suggesting the kiosk may have been switched off. Truthful without the server having to leak anything. Three tests, verified by planting the all-digits match back. Scope: product |
||
|
cproudlock
|
f4d3827214 |
Record how a PC is reached, and stop pages implying they can do things
Three findings, all about a screen or a log saying something that is not so. Editing a PC wrote an audit entry for its named fields and nothing for the two changes that matter most when somebody investigates later: the primary IP address, which is how the machine is found, and the remote-access protocols and ports, which is how somebody connects to it. Both are on the same form, in the same request, and both were invisible afterwards - a PC that moved address had no history of moving, and a PC that gained a VNC listener had no record of gaining one. Both are recorded now, and re-sending the same value still records nothing, because an audit log full of non-events is one nobody reads. While proving that, a silent discard turned up in the same lines: both the create and the update path did `if ip_comtype:` around the IP write and did nothing at all when the 'IP' communication type row was missing. Somebody typed an address, saved, got a 200, and the address was gone. A missing reference row is a seeding gap, not a reason to drop what was typed, so it is created on demand. The PCs list rendered a 403 as "No computers found", which tells a person without the permission that the fleet is empty and sends them looking for the data rather than for the permission. It now says which permission is missing. The Site Policies and Configuration Policies pages rendered their empty table and an Add button while the policy feature flag was off - every route answers 404 "Policy feature is not enabled", so the button could never work. Both pages now say the feature is off and that the empty list is a consequence of that, not of nothing being configured. Nine tests. The audit entries verified by planting: remove either and two fail. Scope: product |
||
|
cproudlock
|
cbdfcbabb7 |
One badge vocabulary, and images a browser can actually load
Two operator-visible defects and three doc corrections, closing the tail of the 2026-09-03 run. THE BADGE. The two kiosks read the same physical badges and disagreed. The parts kiosk understood both shapes a scanner emits - the bare SSO and the PayNo wrapper 0<digits>BZ - while the USB kiosk compared the raw string to an SSO, so the wrapper resolved to nobody. An operator whose keyboard-wedge scanner emits it could take a 3D-printed part and could not check out a drive, with the same badge, at two cabinets a few metres apart. The shape now lives in shopdb/utils/badges.py on the contract surface, because a plugin may not import another plugin and because a badge is a property of the site rather than of a cabinet. Both kiosks use it, and both store the DIGITS rather than the string that arrived: one row per spelling makes a custody history that cannot be joined to itself. That closed a subtler hole in the same place. str.isdigit() is true for Arabic-Indic and other decimal forms, and int() parses them, so a badge written in those characters was accepted, matched a REAL employee, and was then stored as an SSO string nothing else matches. ASCII digits only. An empty scan and an unreadable one now get different answers, because they are different problems for the person at the cabinet. THE IMAGES. /api/applications/image/<filename> and /api/models/image/<filename> required a permission, and a browser fetching <img src="..."> sends no Authorization header - so the gate did not protect the file, it guaranteed the picture never appeared. Both are ungated now, like the four sibling image routes (branding logo, map blueprint, employee photo, part picture), and registered in PUBLIC_ENDPOINTS with that reasoning. /api/printers/lookup?fqdn= never matched anything: both arguments were compared against Communication.ipaddress, so a hostname lookup 404d on printers whose hostname was on the record. An FQDN now matches the hostname, with a fallback to the short name. The slide manager gained the per-slide hold time. The PATCH route and the feed have always carried `seconds`; nothing called it, so the one documented per-slide control was curl-only. Docs: API-REFERENCE listed the employee lookups and the whole notifications surface as unauthenticated when they answer 401, and called both kiosks "deliberately open" when they are off by default and refuse anything without the site's device token. The installer download says applications.download, which is what the code requires. api-inventory.json said printers/lookup was jwt-optional. Fourteen tests. The badge rule verified by planting the old per-kiosk behaviour: three fail, one in each kiosk. Scope: product |
||
|
cproudlock
|
8fee09e748 |
Validate what a form accepts, and keep a diff to one asset
The validation tail from the 2026-09-03 run. Each is small; together they are the difference between a field that checks its input and one that looks like it does. Subnets accepted any string containing a slash, so 'notanip/xx' was stored - and _devices_in_subnet then swallowed the ValueError from ip_network and returned an empty device list, so the subnet looked real and permanently empty. It is parsed with the same library that has to read it later, a prefix length is required (ipaddress reads a bare 10.1.1.0 as a /32, which is legal and never what somebody typing a subnet meant), and the stored form is normalised so 10.1.1.5/24 and 10.1.1.0/24 cannot become two rows nobody can tell apart. VLAN numbers had no range check: 99999 and -5 were stored. Worse, a non-numeric value reached a duplicate probe that compares against an integer column, which MySQL coerces to 0 - so 'xyz' matched an existing VLAN 0 and the 409 named a row that had nothing to do with it. 1-4094 per 802.1Q, checked before the probe. Deleting a VLAN that subnets still reference now answers 409 like every sibling in-use guard, rather than 400, which told the caller their request was malformed when the request was fine and the state was the problem. An appversion sha256 was stored raw, so an over-long value was truncated to the column width and came back as 64 plausible hex characters - indistinguishable from a real digest, and wrong. A digest is a claim the delivery path checks against, so it is 64 hex characters or it is refused. The releasedate goes through the strict parser for the same reason it did everywhere else. PUT /api/settings/<key> ignored the declared valuetype. "A6-not-a-number" went into an integer key verbatim and every consumer then read 0, because get_typed_value swallows the ValueError - the setting looked set and the behaviour was the default. /api/maplevels told a caller who omitted the level name that 'buildingname and buildingid are required'; buildingname is a field that handler never reads. It now names the fields it actually wants. Restoring a guide revision that is already current writes nothing, because save() dedups on the content hash - and the response still said "Restored version N as a new draft version", describing a revision that does not exist while the unchanged revision list disagreed. GET /api/backups/revisions/<id>/diff?against=<id> accepted any revision id, so pointing it at another asset's revision rendered that asset's configuration as the "before" side of this asset's history: a diff that reads as a change on this machine and describes a different one. Same asset and same kind, or refused. The default branch was always asset-scoped; only the explicit one was not. Two frontend fixes: the per-page selector rendered blank on any list defaulting to a size not in [10,20,50,100] (network devices uses 25), so the operator could not see the current page size; and an emptied PC alias was dropped from the payload instead of sent, so it could not be cleared - the form came back showing the alias that had just been deleted. Twenty-two tests. The backups guard verified by planting. Scope: product |
||
|
cproudlock
|
f6e5f6ca47 |
Store and serve a maintenance date as a date
The machine maintenance panel showed every date one calendar day early on this
site, and would show it a day late east of UTC.
lastmaintenancedate and nextmaintenancedate were DATETIME columns holding
midnight, and BaseModel.to_dict serialises a datetime as isoformat + 'Z', so
2026-12-15 00:00:00 went out as "2026-12-15T00:00:00Z" and the browser
converted it into local time. There is no time in a maintenance date: the form
offers a date picker, the CSV importer takes '2026-06-01', and the value is only
ever compared against other dates. Every sibling already models it correctly -
measuringtools.lastcalibrationdate, warranties.startdate and enddate are all
db.Date - so machines were the odd one out.
Migration 7d40 converts both columns; MySQL truncates a midnight time so no
data moves, and SQLite neither enforces nor needs it.
That exposed two more things. The route was assigning the raw request string to
the column, which a DATE column rejects outright - it now goes through the
strict parser from wave 2, so a mistyped maintenance date is refused with a 400
rather than stored or nulled. And BaseModel.to_dict had no branch for a plain
date, so one reached Flask's encoder and came out as an HTTP-date
("Thu, 15 Jan 2026 00:00:00 GMT") - which a browser parses back into a moment
and can still render as the day before. Dates now serialise as "2026-01-15",
with datetime checked first because it is a subclass of date.
Verified against MySQL on a live instance: the column is DATE, the API serves
2026-01-15, a mistyped date is refused with 400, and a rename on a machine that
has maintenance dates saves - the 500 from the audit-log fix in
|
||
|
cproudlock
|
61a1c0b4fb |
Answer API errors in the envelope, say when supplies are unread, badge each chain
Three more findings from the 2026-09-03 run.
Werkzeug answered 405, 415 and a malformed JSON body itself, in HTML. A client
written against the documented {status,data,meta} contract got a page of markup
with no error code, which is indistinguishable from the server having fallen
over. The 405 had a second edge: on a route the caller cannot otherwise see, a
method-not-allowed CONFIRMS the path exists, so a gated kiosk route meant to
read as absent announced itself by refusing the wrong verb. All three now
answer in the envelope, and only under /api/ - everything else is the SPA and
keeps the framework's behaviour, deep links included.
The Toner Report rendered "0 Printers Checked / 0 Low / 0 Critical" and "No
printers match the selected filter" while the API it had just called said
supply monitoring was unreachable. Zeros read as a clean bill of health for a
fleet nobody has looked at. The page now carries the reason the API gave and
suppresses the counts, because a number nobody measured is not a result. Same
defect shape as the supplies card fixed on 2026-08-29; `unavailable` was
already in the payload and the page ignored it.
Backup revisions badged "current" per (asset, kind). NTLARS settings describe a
MACHINE but are read off the controlling PC, so a machine with two reporting
PCs has two independent chains of the same kind - and only the higher revision
id was badged, so the second PC's newest revision showed as history and lost
its last-verified stamp. The machine looked like one source re-reporting rather
than two. sourcehostname is the chain; a revision without one forms its own,
which keeps the old behaviour for everything that has only ever had one source.
Nine tests. The envelope ones verified by planting: remove the three handlers
and three fail.
Scope: product
|
||
|
cproudlock
|
2fb924f18e |
Let a printer driver carry the name Windows actually matches on
The driver form and the API both dropped `drivername` and `vendorid`. POST returned 201 with both null, PUT accepted them and stored nothing, and no screen offered either field - so there was no interface in the product that could set them. That is not a cosmetic gap. `drivername` is the exact string the INF declares, and Add-PrinterDriver matches on it; `name` is ours to choose and means nothing to Windows. With it null the shipped converger has nothing to match and installs nothing, and the drift check that compares an observed driver name against the expected one could never fire, because the expected value was never stored. `vendorid` is how a universal driver (HP UPD, Xerox GPD) attaches to a whole make instead of needing a near-duplicate row per model. Both are now read on create and update, and the driver form gained the two fields with hints saying what they are for - the INF name box says plainly that leaving it blank means the converger cannot install that driver. A whitespace value clears rather than stores: an empty box means "not known", and a blank string would look set and match nothing. Three tests. Verified by planting: drop the two fields from the create path again and the round-trip test fails. Scope: product |
||
|
cproudlock
|
b1e5dd734d |
Draw a dual-bay secondary where it actually is, and make two counts add up
Three map and aggregation defects. The first is a behaviour change you asked for; the other two are the 2026-09-03 run's findings. A dualpath secondary bay was excluded from /api/assets/map unconditionally while the collapse setting is on, so positioning it was a silent no-op: the form took the pin, stored mapx/mapy, and the map never drew it. One marker per pair is right when the pair sits in one place, and that stays the default - but a dual-bay machine whose second bay is physically elsewhere is exactly what positioning it separately means. Now: a secondary with its own position draws its own marker; one without is represented by the primary's, which carries the combined '2007 / 2008' label. Nothing extra is needed for the fallback, because the feed already requires mapx/mapy - an unpositioned bay never reaches the loop. The combined label is now suppressed when the partner draws its own marker. Labelling two separate pins '2007 / 2008' would say one machine is in two places, which is worse than either bay being unlabelled. Verified against a live instance: unpositioned secondary gives one marker labelled with both bays; positioned gives two markers each with its own name; clearing the position puts it back to one. That third case matters - a one-way door would be its own bug. machines/dashboard/summary applied the dualpath exclusion to total and bytype but not to bystatus, so the status bars summed to more than the headline above them, by exactly the number of secondary bays. A breakdown that does not add up to its own total is an afternoon of reconciling. The measuring-tool calibrationstatus filter ran AFTER pagination. It is a derived value so it cannot be a SQL filter, but applying it to the current page meant asking for 'overdue' with one overdue tool in sixteen returned an empty page 1, a total of 16, eight pages, and the row itself on page 3. When that filter is in play the matching set is now built and filtered before paging - the same tradeoff the old comment named, spent on correct answers instead of a cheaper query. Seven tests. The pagination one verified against the committed version, which fails it. Scope: product |
||
|
cproudlock
|
ffced2f822 |
Keep a share path inside the share, a UTC timestamp in UTC, and a public map public
Three confirmed findings from the 2026-09-03 run, each independent.
GE-Enforce scope names are directory names on the imaging share, and create_scope
checked only that the name was non-empty. "../escape" wrote a manifest one level
above the share root, "../../escape2" two, and "/tmp/anywhere" wherever it liked
- proven by writing the files and then deleting them. The exposure is the
permission split rather than the traversal itself: geenforce.manage creates the
scope and geenforce.publish writes the file, so a manage-only author could aim a
publisher's write at any path that process can reach. Names are now a single
safe path segment, checked at creation AND again at the write (rows predating the
check exist, and the write is the line that acts), with the resolved target
confirmed to sit under the share root.
Agent timestamps were converted with astimezone(tz=None), which means the
SERVER's local zone, not UTC. An agent reporting 17:26:16Z was stored as
13:26:16 on this EDT box and re-served labelled 'Z', while the same row's
server-written lastreportedat stayed true UTC - one row carrying two time bases
four hours apart. The dashboard's 48-hour stability window compares a real UTC
cutoff against these values, so events left it four hours early. Invisible on a
UTC server, which is how it survived, and nothing in the Dockerfile, the compose
files or the Windows deploy pins TZ. AGENT-CONTRACT.md mandates UTC on the wire,
so a naive value is now taken as UTC rather than guessed at.
The three /api/maplevels reads went back to optional-auth. ADR-017 rule 6, the
API inventory (auth "none"), the module's own docstring and the comment in
PrinterInstallerMap.vue all say they are public, because the printer installer
map runs before anyone logs in and needs a blueprint to draw on.
|
||
|
cproudlock
|
5d81f50c33 |
Stop the audit log from being what fails the write
Wave 4, second half. PUT /api/machines/<id> returned 500 and rolled the whole
edit back for any machine with a maintenance date recorded - including a rename
that never touched the date. The Machine edit form could not save such a
machine at all, which is most of them once a maintenance schedule exists.
The route builds {'old': getattr(row, field)} straight off the model, so the
changes dict carries whatever the column holds: a date, a datetime, a Decimal.
AuditLog.changes serialised it with a bare json.dumps, which raises on all
three, and nothing caught it. The audit entry and the edit share a transaction,
so the failure took the edit with it.
json.dumps now has a fallback encoder: dates and datetimes become ISO strings,
Decimals become floats, sets and bytes get something readable, and anything
else becomes its repr rather than an exception. An audit entry that reads a
little awkwardly is worth more than a failed write - the alternative, and what
happened, is losing the change AND the record of it.
Fixed at the audit log rather than in the machines route: every audited route
builds its changes dict the same way, so the next one to audit a date field
would have hit the same wall.
Four tests, including the exact shape the machines route produces. Verified by
planting the bare dumps back: two fail.
Scope: product
|
||
|
cproudlock
|
12392a361e |
Refuse a value the database cannot store, instead of quietly changing it
Wave 2 of the remediation plan. Eight findings from the feature test run were the same defect twice: a write that reports success while storing something other than what was sent. The first half is a date parser that swallowed its own error. Two plugins carried an identical private _parse_date returning None on ValueError, so an unparseable date and a deliberately cleared one produced the same write. PUT /api/warranty/<id> with enddate '30-06-2027' answered 200 "Warranty updated" and stored NULL, dropping the record out of every expiry surface on the strength of a typo; the measuring-tool calibration date did the same and flipped the tool to 'unknown', out of the calibration report and the map overlay. shopdb/utils/datevalues.py now holds one strict parser on the contract surface: an explicitly empty value clears the field, anything that is not a date is refused with a 400 naming the field, and those are different intents with different answers. The vendor-lookup paths keep a lenient wrapper on purpose - a provider changing its date shape must not fail the refresh of an otherwise good record, and no person typed that value. The second half is why the truncation findings were possible at all. MySQL was running non-strict: sql_mode on the 5.6 server is NO_ENGINE_SUBSTITUTION with no STRICT_TRANS_TABLES, so it accepted writes it could not store and adjusted them. A 257-character username became its first 100 characters (and the next distinct one then collided with it); an over-long asset number was cut at 50; a malformed releasedate became '0000-00-00' and permanently broke every read of that application. All with a 201. The mode is now set on the CONNECTION rather than left to the server, so a site whose my.cnf disagrees still gets it. That trades silent corruption for a DataError, which uncaught is a 500 - and a 500 says the server is broken when the truth is that the value does not fit. An error handler maps it to a 400 naming the column where MySQL names one. Verified by hand against MySQL 5.6: POST /api/users with a 257-character username now answers 400 "That value is too long for username." and stores nothing, where it previously stored a truncated row and answered 201. Twelve tests. The date guard is verified by planting: restore the swallowing parser and four of them fail. The DataError tests raise the error directly because the suite runs on SQLite, which does not enforce VARCHAR length at all - there is no value that would trigger it there, which is precisely why this went unnoticed. Scope: product |
||
|
cproudlock
|
aa8028f96e |
Let the row refuse the write both kiosks were deciding in Python
Wave 1 of the remediation plan for the 2026-09-03 feature test run. Two kiosks, one shape of defect, one critical consequence. Both kiosks read the state, decided in Python, and then wrote. Nothing stops two requests interleaving between the read and the write, so both see the same pre-state and both proceed. Reproduced on MySQL 5.6 over HTTP, five concurrent takes of 5 from a bin holding 5: all five were accepted and quantityonhand landed on -20, because each request wrote its own arithmetic. The ledger had recorded every one of them, so the cached quantity and the ledger disagreed permanently, which is the number the shop floor actually reads. The USB kiosk had the same shape: concurrent scans of one drive all succeeded and inserted several open custody rows, so one drive was recorded out to two people and the documented 409 never fired. The comparison now lives in the WHERE clause, where the database settles it at write time. printedparts._move_quantity issues UPDATE ... WHERE quantityonhand >= :n and treats rowcount 0 as the refusal, raising InsufficientStock, which take and adjust turn back into the message they already gave - carrying the quantity the database holds now, not the stale one the request read. The low-stock alert derives its before-value from what the write produced for the same reason. Both USB paths claim the device with ischeckedout (self-hosted) or status (external cmmc_usb) in the WHERE clause; the external one claims BEFORE writing the log line, so a refused claim leaves nothing to undo. Check-in is guarded the same way. After the fix, the same five concurrent takes: one accepted, four refused, quantityonhand 0 and the ledger sum 0. Check-in also closes EVERY open row for the device rather than the newest. One device can only be out once, so a second open row is corruption, and closing only the newest left the older ones open forever - the device then read as returned while its custody history said otherwise. Separately, usbcheckouts now records who handed a drive back (checkinsso/checkinname, migration usb0004checkinidentity). Returning for a colleague is a supported flow that the kiosk asks to confirm, but the row it wrote attributed the check-in to the HOLDER, so the custody history and the audit report both named somebody who was not there. Nullable and deliberately not backfilled: copying the holder's badge into old rows would manufacture evidence for the exact claim this stops making. Four tests. Both guards verified by planting: strip the condition out of either UPDATE and its test fails. Scope: product |
||
|
cproudlock
|
13be0ef90f |
Import the wilm agent store and WINC adoption work, with four fixes
Eleven commits from the sister site: a reversible Maximo/WINC machine-adoption import, a versioned agent binary store with upload and rollback, remote-BuildKit CI, tray version reporting, and UI fixes. Applied as a patch again - the published mirror rewrites hashes, so the branches have unrelated histories and git refuses to merge them. Reviewed by three agents before merging. Four things fixed here. THE PUBLISH WORKFLOW WAS DEAD. The new remote-BuildKit step in publish-images.yml sits at column 0 instead of indented under steps:, so the whole file is invalid YAML and GitHub would refuse it. Their entire GHCR distribution channel - which Start-ShopDBDev.ps1 and the compose files consume - was broken at the source and clearly never run. Indented; all five workflows now parse. THE IMPORT'S APPROVAL CHECK COULD NOT FAIL. commit() built the plan and the approval packet from the same upload and then asked the writer whether they matched, so packet_matches_plan compared a hash against itself. The documented contract - "a human approves a specific, frozen plan, and the writer can prove nothing drifted" - did not exist in the web path: an admin could preview file A and commit file B, gated only by the needs-create group NAMES they ticked. The preview's planhash is now required at commit and checked against a freshly rebuilt plan. Their tests committed without previewing, which is what let this through; they go through the real flow now, and three new tests pin it - all confirmed to fail with the check stubbed out. endpoints.agentpackage QUIETLY GREW. It used to mean "wrap the staged exe in a ZIP"; with the upload routes it also means "choose the executable every future package installs and runs as SYSTEM". Before this, staging needed filesystem access to the server. The permission description still said only "generate and download a package", so an admin granting it could not see what they were granting. Reworded. NOTHING RECORDED WHAT WAS STAGED, OR BY WHOM. The manifest held version, size and timestamp; re-uploading a version replaces the bytes in place under the same label. A sha256 of the stored file is now recorded, and the upload is audit-logged with it - after an incident there was previously no answer to who put that binary on the fleet. Also documented the nine new routes in docs/api-inventory.json (that gate cannot run on the mirror, so it falls to us at promotion time again), regenerated the spec and the project map, and kept our own docs where they conflicted: docs/ is scrubbed from the mirror, so their DEV-BUILD-TEST.md still had the dev-be channel mapping we corrected, and their PLUGIN-HOOKS.md was a new 45-line file rather than our 737-line canonical one. Their journal section is appended to ours. Contract version comes across as 0.30.0 for a genuinely additive surface. Suite 2369, frontend 324, gate green. Scope: product |
||
|
cproudlock
|
7f6aaf2e64 |
Gate the USB kiosk the same way, and make an unreachable setting a test failure
The USB kiosk had a feature flag and nothing else. A flag is a decision, not a credential: with the kiosk on, anyone who could reach the app could move a drive between checked-out and checked-in and sign it with a badge they guessed. Same reasoning that closed the parts kiosk, smaller blast radius - which is why Darkwing did not raise it and why it was worth asking about anyway. Both kiosks now share one mechanism. The device token moved out of the printedparts api wrapper into a request interceptor that attaches X-Kiosk-Token by URL prefix, with one localStorage key per kiosk - they are separate screens in separate rooms, and a shared token would mean provisioning one grants the other. claimKioskToken() is shared too, so both views take the token from a ?token= URL once and strip it from the address bar identically. Then the part that matters more than either fix. I told you to set the new settings in the UI, and that was impossible: every plugin settings page carries a hardcoded key list, and nothing had added them. printedparts_kiosk_enabled, printedparts_kiosk_token and usb_kiosk_token were declared, seeded and read correctly, and rendered by no page - changeable only by writing SQL. tests/test_plugin_settings_reachable.py now fails when any declared setting appears nowhere in its plugin's frontend. It found seven MORE that were already in that state, all pre-existing: collector_corporate_ranges computers_configpolicies_enabled computers_machinelink_alerts computers_quietreporthours geenforce_reportstaleminutes printers_dashboardpercent warranty_expiringdays All seven now have fields. I first parked them in a KNOWN_GAPS ratchet, then fixed them and deleted the ratchet - a backlog list in a new check is a way of agreeing the check does not apply yet. There is no backlog; the check is absolute. Two of my own mistakes on the way, both caught by tests rather than by reading: the blanket rename of the USB guard also rewrote the call inside _kiosk_open, so it called itself until the stack ran out; and the first version of the reachable check skipped all 17 plugins because it looked for get_settings() when the method is get_settings_defaults() - it passed by testing nothing, which is the exact shape it exists to catch. Suite 2336, frontend 316, build clean. Scope: product |
||
|
cproudlock
|
744bc4341a |
Shut the parts kiosk behind a device token, and stop the health probe leaking
Both from the second Darkwing scan. Every claim in it was verified against the
code before acting.
The kiosk was the product's only unauthenticated write, justified by a decision
record saying it was "physically rate-limited: it serves a touch screen on the
shop floor". Nothing in code enforced the physical half, and an HTTP route is
reachable from wherever the app is. What that left open:
- _kiosk_find_item falls back to matching bare digits against the numeric
tail of every active item, so 1, 2, 3 walk the catalogue without knowing a
barcode;
- a badge is a short numeric SSO, and the route answered 422 for an unknown
one and 200 for a real one - an oracle for sorting guesses into employees;
- the write then decremented stock and stamped the ledger, which IS the audit
trail, with whatever SSO the caller supplied;
- quantity was bounded only by what was on hand, so one request emptied a bin.
A badge is attribution, never authorization. Both kiosk routes now need the
site's kiosk flag AND a per-device token, and 404 rather than 403 without them
so a caller does not learn the door is there. The READ is gated too: gating only
the write would have left the enumeration oracle running against a kiosk a site
had switched off.
The token is per-DEVICE, not per-page, because a kiosk is a browser on a touch
screen and a token served inside the page is readable by anyone who can load it.
An admin opens the screen once as /parts-kiosk?token=..., the page moves it into
that browser's localStorage and strips it from the address bar.
Also: quantity is capped per take, JSON true no longer passes as 1
(isinstance(True, int) is True), and every badge failure on the anonymous route
returns one message. The gated restock and adjust handlers keep the specific
message - there the person is known and the detail is help, not an oracle. I
changed the wrong one of those three handlers first; caught by the test.
/api/dashboard/health formatted str(e) straight into the body of a route
PUBLIC_ENDPOINTS declares open because "monitoring has no account". A pymysql
OperationalError names the database account, the app host's address, the driver
and the failing statement. It is logged server-side now and the body carries a
coarse status. The version goes with it - it told an anonymous caller which
release to look up. Status stays 200 either way: a monitor keyed on the body's
status field is the contract here, and turning a degraded database into a
transport failure is a separate decision from not leaking the exception.
Nine tests. The three gate tests were confirmed to fail with the gate stubbed
out, rather than assumed to cover it. Suite 2325, frontend 316, build clean.
Scope: product
|
||
|
cproudlock
|
e98e15259e |
Unpick one site's conventions from the shipped bundle, and bind the launch-path
check to the file it actually runs Two clusters the five-agent review found. Both are about a product that another plant has to be able to install. ADR-015: the shipped monitor bundle was the originating plant's fleet convention frozen into product seed. The business policy was seeded as "RPCA Shopfloor App Monitors". RPCA is that plant's facility designator, the row is source='shipped', and the API refused to rename it - so every adopting site would carry another plant's label at the top of its policy list forever. 0007 now seeds the neutral name, and 0010 renames it on installs that already stamped 0007, guarded so it cannot collide with a policy a site made itself. Downgrade is one-way on purpose: putting another site's designator back is not a state worth reaching. Which PC type the bundle attaches to was the literal 'Shopfloor PC'. A plant whose type is "Shop Floor Workstation" silently got no shipped monitors at all, and one using that name for something else got all of them. It is a setting now, defaulting to the seeded name so nothing changes here, and a blank value attaches the bundle to nothing - the right neutral answer for a site that does not want it. Shipped rows refused every edit, including isactive and launchpath. A plant whose eDNC is on D:, or that runs no UDC, could not correct the path or switch the monitor off, and then carried permanent false drift for it. ADR-015 allows seed data BECAUSE a site can edit it afterwards. Paths and the active flag are now editable; kind and displayname stay fixed, because that pair is the identity 0007's bundle matches on. Also corrected two false claims in 0007's docstring: the policy tables belong to the computers plugin, not the core baseline, and a guarded no-op does NOT reconcile later - Alembic stamps the revision either way, so the bundle would simply never seed. LaunchPathPolicy was purely lexical. GetFullPath is string normalization that never touches the disk, so the root check constrained the NAME while File.Exists and CreateProcess followed reparse points to somewhere else. A junction under a writable directory in an approved root - creating one needs no privilege, only write access to the parent, and vendor trees under Program Files are not reliably locked down - turned WebUI monitor-edit rights into code execution as the console operator. It now resolves the real target, leaf and directory chain both, and re-checks THAT against the roots, rejecting if resolution fails. Alternate data streams passed the whole policy on net8.0: an interior colon no longer throws in GetFullPath, so "readme.txt:payload.exe" reported an .exe extension, passed the root check, and existed. Rejected explicitly now. Six tests added for the shipped-row and pctype behaviour. The Core project compiles clean. The launch-path change has NOT been exercised against real junctions, ADS or 8.3 names - that needs the win11 VM, and per the reviewer it wants a planted-violation suite there rather than my say-so. Still open from the review: monitor revocation does not take effect until agent restart, ActionPicker is keyboard-inoperable, the lightbox has no focus trap, one attached monitor grants generic any-target process control, and screenshots are retained unencrypted in the user profile. Scope: product |
||
|
cproudlock
|
b5540e631c |
Merge the wilm endpoint-monitor and channel-distribution work into main
Six commits from wilm-import: the squashed import, route documentation, the channel remap onto our tag-and-trunk model, the project map, parallel test support, and the four smaller review findings. Merged rather than cherry-picked: the published mirror rewrites hashes, so the commits do not apply one at a time against our history even though the squashed diff applies clean. Scope: product # Conflicts: # docs/PROJECT-MAP.md |
||
|
cproudlock
|
720f4ddbb2 |
Close the four smaller findings from the wilm import review
None blocked the merge; all four were mechanisms that did not mechanise. _RUNNING_STATES existed twice, in api/routes.py and services/reconciler.py, with a comment asking a human to keep them in lockstep. The panel and the reconciler read the same agent snapshot, so a drift between the copies would paint a service green and flag it as drift on the same page. One definition now, owned by the reconciler and imported by routes - api -> services is the direction the plugin layout already runs in, and the reconciler's import of routes is deferred, so there is no cycle. The operator-report upload checked its 10 MB cap AFTER upload.read() had already pulled the whole body into memory, so the check could not prevent the thing it was there to prevent. content_length is now an early out and the read itself is bounded to cap+1, which is enough to know the cap was exceeded without taking a body that lied about its length. The multipart envelope carries the other form fields too, so the early out allows 64 KB of slack rather than rejecting an image that is just under. The same handler forced the extension to .png and the mimetype to image/png whatever the bytes were, then served the file back under that type. The upload headers (CSP sandbox + nosniff) make the document case inert, so this was never the hole it looks like, but nothing is gained by storing a file we already know is not what it claims. The PNG signature is now checked. The lightbox bound Escape and the arrow keys to a div and never focused it, so every one of those bindings was dead until the user happened to click the backdrop. It takes focus on open and hands it back to the thumbnail on close. On the two upload tests, honestly: the PNG one was confirmed to fail with the check removed. The oversize one passes either way, because the old code did reject an oversized image - it just did so after reading all of it. That test pins the response, not the fix. Whether the read is bounded is not observable through the response, so nothing here proves it; the change stands on reading the handler. Suite 2311 passing, frontend 316 passing, build clean. Scope: product |
||
|
cproudlock
|
f3a291a28d |
Stop a request-supplied host reaching the generated installer script
Found by the Darkwing scan of main (run 33654922078). Its own review dismissed
the finding as not remotely exploitable and it is right about that: nothing can
make a victim's browser attach a custom header, and deploy/windows/web.config
does no output caching. It still recommended the hardening, and the reasoning
for the dismissal is a property of OUR IIS config, not of the product - every
site runs its own instance behind its own proxy under ADR-004.
_batch_base_url() interpolated a caller-supplied host into the single-quoted
PowerShell -Uri literal of the .bat this route hands out. site_base_url ships
blank, so that fallback is the default path on a fresh install, not an edge
case, and GET /api/printers/install-batch is public.
Three rules now:
- the host is validated as a hostname before it is interpolated, so the quote
that would end the literal never reaches it,
- X-Forwarded-Host is only believed when TRUSTED_PROXY_COUNT declares a proxy
-- the rule shopdb/utils/clientip.py already applies to X-Forwarded-For.
That was centralised for this exact reason and did not cover the Host
variant,
- a request supplying neither usable header gets a relative base, which still
downloads correctly from the browser that fetched the script.
Three tests pin it, and all three were confirmed to fail with the old body
restored rather than assumed to cover it.
Scope: product
|
||
|
cproudlock
|
27b2817fd7 |
Import the wilm endpoint-monitor and channel-distribution work
Everything on origin/wilm that is not on the published main, applied as one
change because the published history has rewritten hashes: per-commit
cherry-pick conflicts against our main even though the squashed diff applies
clean.
Contents: endpoint monitor catalog + policy attachment + warn-only drift
reconciler, operator issue reports from the tray, agent launch-path allowlist,
and the GHCR channel-image distribution path for dev workstations.
NOT READY FOR main. Open items:
- 10 new routes are absent from docs/api-inventory.json, so
test_api_inventory_parity fails. That gate SKIPS on the published mirror
(the test ships, its data file is scrubbed), which is why wilm never saw it.
- dev-bootstrap.ps1 and Start-ShopDBDev.ps1 hardcode the ge-aero GitHub org.
- plugins/endpoints/plugin.py carries a whitespace-only re-indent.
Scope: product
|
||
|
cproudlock
|
c8e0de4dc1 |
Let the server decide which NIC is a PC's address, and keep the rest
A CMM reported 100.0.0.1 as its address and never reported 10.48.255.210 at all. Both halves of that were the same mistake. THE BAY WAS CHOOSING. The collector schema took a single `ipaddress`, so the script picked one NIC and dropped the others - preferring an address inside a configured range, else whichever interface carried a default route. A bay cannot make that judgement: it does not know the site's addressing. On this CMM the Marvell AQtion 10Gbit link to its controller is a real hardware NIC, up, and carries a default route of its own, so it satisfied every test the script had. It won, and the corporate address was discarded before the request was even built - which is why it is not in ShopDB even as a second row. So the bay now reports EVERY connected hardware NIC with a routable IPv4, and takes no view. The server picks the primary from collector_corporate_ranges, a new per-site setting, and writes a Communication row for each of the others. Naming the ranges is what turns the answer from a guess into a decision; with none set it still falls back to the default-route flag and then to the first reported, both guesses, both better than before only because nothing is thrown away now. Rows are keyed on the MAC where the bay sends one, because an address moves between adapters and a MAC does not. A NIC that is down at check-in is left alone rather than deleted - being down is not evidence the record is wrong, and hand-entered gateway, DNS and serial settings live on those same rows. Exactly one row ends up flagged primary, and only a row this report touched can claim it, so a check-in cannot silently demote an address set by hand. -AllowedRanges and the CollectorRanges registry value are now accepted and ignored, logging a line that says where the setting moved. The parameter stays because a scheduled task still passing it would otherwise fail on an unknown argument. A bay on an older collector sends one address and no interfaces; that keeps working and is pinned by a test. Scope: product |
||
|
cproudlock
|
dd52666d6b |
Order the supplies card by how empty the cartridge is
Rows sorted on iscritical then on printer NAME. Critical is everything
at or below 5%, so a cartridge at 5% and one at 0% tied on the first key
and the alphabet decided which came first.
Wrong order on its own, and worse than it looks: the dashboard queue
keeps only the first few rows per source, so a printer that was
genuinely empty was truncated off the card while a fuller one held its
place. Reported from the floor as "starts at 5% and hides one at 0%",
which is exactly the two bugs compounding.
remaining is already normalised for waste cartridges - a full waste bin
reads as low remaining - so one ascending sort is worst-first for every
supply type, and iscritical falls out of it rather than needing its own
key. Name only breaks ties.
worstremaining is carried on the row because the supplies list is
rendered to text ("Black 0%") before it reaches the client, so the
number the sort needs is gone by then.
The toner forecast report was already worst-first (it sorts on daysleft)
and is unchanged.
Scope: product
(cherry picked from commit
|
||
|
cproudlock
|
dbc43162c5 |
Pin the approved content digest on an endpoint command
Delivery descriptors for run_script and update_app are resolved when the
agent claims a command, not when an operator queues it, and the command
row recorded no digest: the enqueue path resolved the descriptor purely
to validate it and threw the result away.
So what an operator approved and what an endpoint runs were two separate
lookups against mutable content, minutes or hours apart. Anyone able to
replace a script body or an installer package could do so in the gap.
The claim then re-resolves, the descriptor faithfully describes the new
bytes, the agent verifies correctly, and it runs as LocalSystem. The
audit trail names the operator who queued the original.
The gate on the verb and the gate on the payload were also different
grants: pushing needs endpoints.deploy, but replacing what gets pushed
needs only applications.edit.
pinnedsha256 records the digest as it stood at enqueue, and the claim
refuses a descriptor that has moved, failing the push with an audit row
instead of shipping it. Null for actions that deliver no content, and
for rows queued before this migration, which claim as before. A
descriptor that vanished entirely is not treated as tampering: that path
already sends the command through undescribed so the agent fails closed.
Separately, a result may now only finalize a command that is actually
running. That needs no agent change and closes two things: reporting on
a command that was never claimed, and overwriting a recorded failure
with a success.
The agent now sends its hostname on the result callback so the server
can check it owns the command it is finalizing. The shared collector key
does not say who is calling, so without it any agent can post a result
for any command id. The server treats the parameter as optional for
agents that predate this and enforces it when present, so there is no
flag day. This is a floor, not a fix: only a per-endpoint credential
makes the caller's identity provable, and that is tracked separately.
The .NET side is unbuilt here and needs a build plus a win11 VM run.
Scope: product
(cherry picked from commit
|
||
|
cproudlock
|
ccd8221adf |
Key the client address on the connection, not a header the caller sends
client_ip() read the first X-Forwarded-For hop unconditionally. The
login rate limit and the account lockout are keyed on that value and the
audit log records it, so rotating the header per request bought an
unbounded budget and an untraceable trail.
This walked around TRUSTED_PROXY_COUNT, which exists to decide exactly
this. The app leaves ProxyFix off by default and says why, ten lines of
comment away: ProxyFix trusts X-Forwarded-*, a client can send those
itself, and the login limiter is keyed on the address. Then client_ip
read the header anyway, so the stated protection never applied.
Resolution is now from remote_addr, which ProxyFix has already filled
from the header wherever a proxy is genuinely configured. Both cases end
at the same trustworthy source, and the port still comes off because ARR
forwards clientip:port and ProxyFix passes it through verbatim.
geenforce and tools had each hand-rolled the same mistake for their own
limiters. Rather than duplicate security-relevant logic a third time,
client_ip and strip_port join the shopdb.api surface (contract 0.28.0).
geenforce's _trusted_client_ip, which already refused the raw header for
its IP allowlist, is untouched and was the model for this.
The stability table was also missing rows for 0.26.0 and 0.27.0.
INSTALL-WINDOWS-IIS.md never mentioned TRUSTED_PROXY_COUNT, so a site
installed from it runs behind IIS with the setting at 0: every request
looks like it came from IIS, the whole site shares one rate-limit
bucket, and the audit log records the proxy for every action. It now
says so, and says not to set it when nothing is in front.
Three test modules simulated separate callers by setting
X-Forwarded-For, which is the header a real attacker rotates to escape
the limit. One of them, test_client_ip_prefers_forwarded_first_hop_
stripped, asserted the defect outright. The suite could not have caught
this. They now drive REMOTE_ADDR, with a regression guard that forging
the header buys nothing.
Scope: product
(cherry picked from commit
|
||
|
cproudlock
|
5f27f81e88 |
Clear the review blockers on the merged wilm work
A multi-agent review of the merged branch produced 29 findings that survived
adversarial verification. This clears the blockers and the four worth fixing
before it reaches main. The design held up well: path-traversal defence on all
four serving routes is clean, the agent's SHA-256 check fails closed, enqueue
authority is proportionate and audited, and no site literals or credential leaks
in the new C#.
THE TESTS NEVER RAN
pytest.ini sets testpaths = tests, so plugins/endpoints/tests/ was collected ZERO
times. Fifty-five lines of delivery assertions were dead code, and run directly
five of them failed. plugins/printedparts/tests/ had been invisible for longer.
That is the worst shape a gap can take: the suite is green, the coverage is
imaginary, and the collector auth on a download route could be deleted without
anything going red.
Both files moved under tests/. tests/test_no_hidden_tests.py now fails if a test
file appears anywhere pytest will not look; it caught a staged copy inside the
installer bundle on its first run, and was verified by planting a violation.
The five failures were stale expectations, and they split by cause. His: the
applications panel made six panels where his test said five, and 'applications'
belongs in KNOWN_PAYLOAD_KEYS now that it is persisted - his own test contradicted
his own feature. Ours: we renamed the events card to activity, moved the
fleet-wide cards to endpoints.fleet, and made the list route a bare route('')
which Flask rejects unless the blueprint is mounted under a prefix.
FOUR FIXES
The snapshot table's key was a 32-bit autoincrement on a table rewritten
wholesale every check-in, so it climbs at (endpoints x apps) per report and
exhausts on MySQL. Widened while the table is unreleased. Widening it broke every
insert on SQLite, which only autoincrements a column typed exactly INTEGER
PRIMARY KEY, so it carries a dialect variant - and nothing caught either fault,
because no test inserted into that table at all. One does now.
Ingest truncated displayname but not displayversion or publisher. On strict MySQL
an over-long registry value fails the whole check-in transaction, so one machine
with a malformed Uninstall entry goes silent entirely. SQLite does not enforce
lengths, so the suite would never have shown it.
update_app's version: branch pinned AppVersion.sha256, but that route resolves the
version to its APPLICATION and serves application.installpath - so the digest
described an artifact the route never sends, and where it was null the agent had
nothing to verify and failed open. It now pins the file actually served.
The same branch ended the delivery URL at the id, so an MSI arrived with no
extension and could not be routed through msiexec. It carries the filename now.
Agent packaging moved off require_role('admin') to a new endpoints.agentpackage
permission. The generated ZIP carries a live collector key, so this is a heavier
grant than deploying to one endpoint - and a role cannot be given to one person
for one job the way a permission can.
Full suite 2231 passed (2198 before, plus the 22 that never ran and 11 new),
frontend 314.
|
||
|
cproudlock
|
b5fc5c01d1 |
Stay quiet when a site never asked for supply monitoring
Splitting "cannot see the cartridges" from "nothing is low" was right, but it treated two unlike cases as one. isconfigured means enabled AND a URL AND a token, so its absence means the site did not ask for supply monitoring at all - and a permanent "Supply monitoring is not configured" line is a card reporting nothing wrong every day, which is how a board teaches people to stop reading it. It matters for sites adopting ShopDB without Zabbix, who would otherwise inherit a warning about a feature they declined. Not configured now returns a plain empty and the card hides. Only configured and unanswerable reports itself, because that is something meant to work that is not working. Also withdraws an unfounded inference. A dev box has no route to Zabbix by design, so "not reachable" there is the expected state and says nothing about what production can see. The value of this change is not that it revealed a production fault; it is that when production's Zabbix does go down, the board will say so instead of displaying "Low Supplies 0". Four tests pin the three cases apart: unreachable reports, unconfigured stays quiet, and a real zero from a reachable Zabbix is not flagged as blindness. The fourth holds the envelope's rows, since carrying a reason changed the response shape. Verified they are not vacuous by restoring the original branch in place - the unreachable test fails against it. Full suite 2182 passed. |
||
|
cproudlock
|
60c35728e8 |
Import USB records off an external cmmc_usb database, so a site can retire it
A site running external mode keeps its devices and check-in/out history in a separate cmmc_usb database. There was no way to bring that in, so the choice was to keep the second database forever or lose the history. This adds 'flask usb import-external' - dry run by default, --commit to write - which reads through the same connection settings the routes already use and never writes to the source. Two things it has to reconcile, and they are why it is a command rather than a few INSERTs. The schemas disagree about SHAPE. External is an event log, one row per action; self-hosted is a paired row carrying both times. Pairing walks each device in time order, and real logs after years of use are untidy. The rule is that nothing is invented and nothing vanishes: a drive still out becomes an open checkout and is NOT flagged (that is just a drive somebody has), a return with no matching issue is skipped and named, and two issues in a row leave the first open rather than inventing a return time nobody recorded. Identity is the other. usbdevices.serialnumber is unique and NOT NULL and the external devices table has no serial, so device_id becomes it. Blank or duplicated ids stop that device rather than inventing a key. external device_owner has no column here, so it is appended to the device notes rather than dropped. Also adds usbcheckouts.scannedviruses, the one field with nowhere to land. waswiped already means what external calls sanitized - selfhosted.py was mapping it - but scanned_viruses was HARDCODED None in the log output: the API promised the field and had nothing behind it, so an imported history would have rendered blank. It now reads and writes properly. Nullable, never defaulted to false. An unrecorded scan is not a failed scan, and for removable-media records those are very different claims. Full suite 2131 passed; seven new tests pin the pairing rules. |
||
|
cproudlock
|
784964c907 |
Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.
What arrives:
endpoints plugin - a ShopDB agent reporting endpoint state, telemetry,
services and interfaces, with a command channel the agent
long-polls. Ingest stays on the generic collector route
(ADR-006); the agent key is a UI-managed single-scope
token whose revocation is an instant kill switch.
computers policies - Configuration and Business Policies (RMM-style), scope
trees validated at write time, actions checked against
the endpoints command allowlist. Feature-gated: every
route 404s while the setting is off, so enabling it can
never half-expose the surface.
asset CSV import - templates, a collector script carrying the site path, and
a validate-then-commit import. A plugin can own a virtual
column plus a post-write row hook, which is how a PC's
machine number becomes a controls link without a
hand-built relationships sheet.
agent/ - the Windows agent and its installer.
Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.
MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).
Review fixes carried on top:
- six policy GETs were jwt_required(optional=True), which authenticates a
caller without checking anything; they require computers.view now, and six
redundant jwt_required decorators are gone
- the agent installer took a default BaseUrl and CollectorKey, so a mistyped
run silently pointed an endpoint at the wrong instance; both are mandatory
and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
- a committing CSV import wrote no audit row
Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.
Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
|
||
|
cproudlock
|
8953ce8210 |
Ticking a section off crosses out the section
It struck the heading and left everything underneath looking untouched, which is the opposite of what somebody ticking "this section is done" means. The strike now covers every block under that heading, down to the next heading of the same or higher rank - so a top-level section takes its sub-sections with it. That needed the rank, and the HTML importer was throwing it away: h2, h3 and h4 all arrived as a flat 'heading', so a parent heading had no children to cross out. It was the visible half of a real loss - a converted article kept its words and lost its shape. Headings now carry their level and render at it, and a guide written before this reads as flat, exactly as it did. Code blocks, tables and images inside a finished section fade but are not struck. A line through a command makes it unreadable, and re-reading the command is usually the reason to look back at a step already done. The five imported guides were re-imported so they carry their structure. |
||
|
cproudlock
|
afce24379c |
Reading anything needs a permission
A sweep of the URL map found 165 GET routes that answered anybody, with no token at all: the whole asset inventory, every machine, PC, printer and network device, the USB checkout history and who is holding which drive, the staff directory with its photographs, every core report. Guest access had been argued about one feature-flag at a time while the front door stood open. Each of those now carries the read permission it belongs to. That also makes the guest role mean something: a site can open exactly the reads it chooses, and nothing follows by accident. Thirty-three endpoints stay public, and each is named with its reason rather than left to be inferred: the app shell and its bootstrap, the settings allowlist (the handler already filters an anonymous caller down to branding and a few keys), images fetched by <img src> which carries no Authorization header, what a display kiosk renders before anybody signs in, the parts and USB kiosks, the client scripts that run as SYSTEM before a user session exists, and a secure-send link whose unguessable token IS the authorisation. tests/test_core/test_public_read_surface.py is the standing gate, the companion to the mutating-route sweep that has guarded writes for months. A new public read fails CI until somebody adds it to that list in front of a reviewer. Four further reads need a login but no permission, because the handler decides from who you are - your own user record, your own API tokens, the role catalogue the grid renders - and the same test proves each still refuses an anonymous caller. Verified not vacuous by ungating one route and watching the sweep name it. The employees plugin gained employees.view; the directory is names, SSOs, photographs and reporting lines, and it had no permission of any kind. Around ninety tests asserted payload shapes against endpoints that answered anybody. They are about what the answer says, not about who may ask, so they are signed in now rather than the gate being loosened. The ones that assert an anonymous caller is refused were left exactly as they were. |
||
|
cproudlock
|
2331c03664 |
Tick a guide's sections off as you work through it
Only checklist items could be ticked, which answers "what have I done" but not the question a long procedure actually raises: where was I. Somebody looks away to do the thing the guide asked for and looks back at a wall of text. Every step and every heading now carries a checkbox while reading. Ticking one strikes it through and fades it rather than hiding it, because somebody scanning back up the page has to still be able to read what they did. The number is struck through with it, but the step's own "Step 3" label is not - that is the address of the thing, not part of what was done. Headings count, not just steps. A guide converted from an HTML article often has fifty headings and no steps at all, and those are exactly the guides long enough to lose your place in. That also means the card's "N steps to tick" stops reading as a broken feature on a guide that is entirely tickable. Reading only, as the checklists already were: in the editor a tick would be an edit, and an author clicking around their own draft would quietly change what the next reader sees as done. Progress stays per browser and per version - shopfloor PCs share a login, so one technician's ticks must not greet the next person on that bay, and step 4 of version 3 is not step 4 of version 2. |
||
|
cproudlock
|
8531559f22 |
A guide can be carried to another site
Guides are written on one site and needed on all of them, and the sites that matter are air-gapped from wherever the writing happens. Until now a procedure was stranded in the database it was typed into: the HTML importer could rebuild one from its original article, but anything authored in the editor had nowhere to go. flask guides export writes one .guide.json per guide, with every image the blocks reference carried inside it as base64. One file on purpose - a bundle that needs its images copied alongside it is a bundle that will eventually arrive without them. flask guides import reads them back, a directory at a time, and refuses anything that is not a bundle before it writes a row rather than half-importing a stray .json sitting in the same folder. What travels is the content: title, summary, tags, blocks, images. What does not is ids, revision history, authorship and publication state, because those belong to whichever database issued them - an imported guide starts as a draft at revision 1, owned by whoever ran the import. The SLUG does travel: it is the address somebody pastes into a work order, and the same guide should answer to the same name on every site. --replace updates the guide already carrying that slug instead of creating a second one, which is how a corrected guide is re-shipped. It drops the stored images first: the incoming bundle is the whole truth for that guide, and a guide re-shipped monthly would otherwise accumulate orphans nothing points at. Bundles refuse a version they are too old to read rather than importing the parts they recognise. A newer site's guide is not a guide this one can be trusted to render. Proven by round-tripping a five-guide export back into the same site, including one carrying two PNG screenshots, and reading it in a browser. |
||
|
cproudlock
|
c8efadefba |
An HTML article can become a guide
A site adopting this app arrives holding procedures already written: styled HTML pages that get mailed around, opened off a share, and edited by whoever still has the file. Those pages are exactly what a guide is for, and retyping one into the editor is how a good procedure stays on a share forever. flask guides import-html reads the page and produces blocks. It recognises the vocabulary those pages actually use - a step box with its own number, an alert in one of three colours, a code block with a copy button, a checkbox list, a table with a header row - and maps each to the block that already renders it, lifting embedded images into guideimages so a document stops carrying its screenshots as base64. Page furniture goes: the header, the footer, and the table of contents a guide builds for itself from its own headings. The conversion is lossy on purpose. Colour, fonts, width and nested layout are dropped, because storing blocks rather than markup is what makes dark mode, printing and tickable steps work at all. So an import is always a DRAFT and the default is a dry run that reports what it would produce and writes nothing. Blocks go through the same validator the API uses: an import must not be able to store something the editor could not have produced. The parser is tolerant, because real articles are not well-formed. An unclosed <p> is normal HTML and the browser closes it at the next block element; without the same rule one unclosed tag swallows the rest of the article into a single paragraph, which is what it did before the test caught it. Proven end to end against four real articles, including one carrying two PNG screenshots, read back in a browser. |
||
|
cproudlock
|
f59aca5a1b |
USB check-in / check-out audit, as a report rather than a paged list
CMMC removable-media tracking is an audit obligation, and answering "who had this drive, and when" meant paging through the check-in/out list by hand. GET /api/usb/reports/audit returns the events - a take and a return are separate rows, because a count cannot answer the question an audit actually asks - with summary figures riding along for the people who only want the shape of it, and a CSV for the people who want it in a spreadsheet. The window is named rather than a day count: today, last month, six months, a year, all time. Nobody should have to work out that a year is 365. Both directory modes answer in one voice. The external path reads the cmmc_usb checkinoutlog; self-hosted expands one usbcheckouts row into its two ends, so a returned drive is distinguishable from one still out, and the window filter runs in SQL against either end of the loan - a drive taken before the window and returned inside it is a check-in the window has to show. Rows come back from the mode-specific half; the CSV, the summary and the response shape are built once in the route. Ordering is total in both modes. Timestamp alone ties for two events in the same second, and the same query then returns them in a different order each run, which reads as the log being unstable. |
||
|
cproudlock
|
7c47ae1f7c |
USB kiosk: take and return a drive without a ShopDB login
The people who actually take USB drives are standing at the cabinet, and most of them have no account here. Check-out and check-in were gated behind a login that was not doing much anyway: both modes already attribute the action to a scanned BADGE rather than to the signed-in user, so the login was gating the screen, not the record. /usb-kiosk is a full-screen touch page modelled on the parts kiosk, off unless a site sets usb_kiosk_enabled - every route 404s until then, rather than 403, so a disabled kiosk does not advertise that there is a door here. It works against the self-hosted tables or an existing cmmc_usb database. THE LIST IS THE TRADE. These drives carry no barcode, so unlike the parts kiosk there is nothing to present and the screen has to offer a list - which exposes more than a scan-first design, where you had to already hold a drive to learn anything. So the list is not anonymous: an SSO comes first and must resolve to a real person before any drive is named, which is the same bar taking a drive already had to clear. It is a POST so the badge stays out of proxy access logs. The main screen shows the caller's own drives, what is free, and a COUNT of everything else; holders of other people's drives appear only behind a second deliberate tap, so nobody reads the whole cabinet off a wall panel in passing. A badge that resolves to no employee is REFUSED, unlike the signed-in route which creates the user row. A checkout nobody can trace to a person defeats the tracking, and at a kiosk there is no operator standing there to vouch for it. Returning somebody else's drive asks first and then records the badge that was SCANNED, so the log says who actually handed it back. That is the honest record and it needs no change to a schema this app does not own. Refusing a mismatch outright is a setting but not the default: a drive left behind by somebody on leave would otherwise be unreturnable at the kiosk, which just moves the problem to a person with a login. The authz sweep caught these writes, twice - once for the two mutations and again for the two list endpoints. Both are exempted with the reasoning written out beside the parts-kiosk entry rather than silenced. The list endpoints are POSTs that change nothing. Verified by driving the real page: SSO on the keypad, tap Take, database moves. |
||
|
cproudlock
|
dffa2082be |
Backups: stop storing credentials, and fix a dedup chain that never matched
The NTLARS/DNC registry carries an FTP password in cleartext and it was stored verbatim. That put it in the backuprevisions table, on the revision panel which renders values, and in the download route which hands back a ready-to-import .reg - so anyone holding backups.view could read it off a web page. Redaction happens at PARSE, so the secret never reaches the database at all. Redacting on display would have left it at rest and one query away. What is deliberately NOT done is dropping the value silently. This codec already refuses to discard an unparseable value, precisely because a backup that looks complete and restores a machine with a setting missing is the worst outcome. A redacted value keeps its name, is listed in the projection, and renders as a COMMENT naming what must be set by hand. Emitting a placeholder as the password would produce a machine that looks restored and cannot connect, and the technician would go debugging the network. scripts/redact_backup_secrets.py scrubs revisions recorded before this, dry run by default. It leaves contenthash alone: the hash identifies the configuration a bay reported, so rewriting it would make every bay's next post look like a change and insert a duplicate revision across the fleet. Separately, sourcehostname is now normalized at ingest. samesource() maps '' to IS NULL for the query while the insert stored the raw '', so rows written by a client posting an empty hostname could never match their own next post: every post inserted, the table grew without bound, and the history filled with identical entries nobody could read. Whitespace did the same thing more quietly. |
||
|
cproudlock
|
cebb3e9e44 |
Guides: make images render, and stop drafts leaking to readers
Images were a broken icon for every user. The stored URL was bound straight to <img src>, and a plain image request carries no Authorization header while this app keeps its JWT in one, so every fetch was a 401. They are now fetched through the API client, which also fixes the stored '/api/...' path under a subpath mount where the API is not at /api at all. A failed image says so in words rather than rendering as a broken icon, and the editor's own thumbnails go through the same path - an author needs to see the screenshot they pasted. Revision and image routes gated on their parent guide. Both were gated on guides.view alone, and revision ids are sequential, so being able to read one guide was enough to walk the ids and read every unpublished draft on the site - the opposite of what docs/GUIDES.md promises. An image in a PUBLISHED guide still serves to any reader; one that only appears in a draft does not. The guide list, the tag chips and search now match the PUBLISHED text rather than the working copy. Matching the working copy handed readers a hit on wording that is not in the page when they open it. The slug no longer follows the title. It was regenerated on every save that carried a title and the editor autosaves every three seconds, so fixing a typo in a heading silently moved the URL and broke exactly the pasted links the stored slug exists to protect. publish() now aborts when the pre-publish save failed. It ran regardless, so the author saw "Published version N" while readers got the previous revision. Link rendering: text is escaped before formatting, so a captured URL arrives carrying entities and was escaped a second time - every multi-parameter ServiceNow and SharePoint link opened with &amp; in it. Emphasis could also reach inside an href, and '//host' passed as an in-app link when the browser reads it as off-site. Tests are now HTTP-level. The draft-invisibility rule lived entirely in routes that no test executed, which is how three of them shipped serving drafts. |
||
|
cproudlock
|
38d1e367fc |
Guides: write a procedure in ShopDB, not beside it
The knowledgebase plugin is a link registry - a description, a URL and some keywords pointing at a document held somewhere else. That leaves the documents themselves as HTML files on a share: no search, no history, no way to know whether the copy pinned to a noticeboard is current, and no way for a technician to tick off where they are. A guide is stored as an ordered list of typed blocks rather than a blob of HTML, because three of the four things this feature exists for depend on it. A tickable step needs a stable id to be remembered; dark mode needs semantic markup rather than pasted colours; and print needs to know where a step begins so it does not split one across a page. Free HTML gives none of that and lets pasted styling fight the theme. Inline text is a restricted Markdown subset - bold, italic, inline code, links, line breaks - escaped BEFORE formatting, so nothing an author pasted can reach the DOM as markup. No fonts, sizes or colours: emphasis is semantic and the theme decides how it looks. That is what makes a guide written next year match one written today, and dark mode work without an author thinking about it. DRAFTS AND VERSIONS. The guide row is the working copy; every save that changes something writes an immutable revision; and a separate pointer says which revision READERS get. Without that pointer, editing a live procedure changes it under a technician who is halfway through following it. Publishing promotes an existing revision rather than snapshotting, so a publish with unsaved edits cannot ship a version nobody reviewed. Restoring writes a new revision with old content, so the record of what people were told in between survives. Saves dedup on a content hash covering title, summary, tags and blocks, so an autosave that changed nothing writes no version - otherwise the history fills with identical entries and hides the three edits that mattered. Progress lives in the browser, not the database: shopfloor PCs share a login, so per-user ticks would show one technician's progress to the next person on that bay, which reads as "already done". Keyed by guide and version, because step 4 of version 3 is not step 4 of version 2. Five permissions, and the split between edit and publish is the point: a technician can write a procedure up while somebody else decides it is fit for the floor. create-without-edit works too - an author may edit their own guide until it is published, after which other people are following it. |
||
|
cproudlock
|
28af92aaec |
An unmeasurable backup must not hide a working one
CI caught this: judging a host by its stalest kind treated a never-confirmed revision (lastseenat NULL) as the worst case, so it won. The verdict for a NULL confirmation is tri-state None, which renders as NO BADGE - so a host with a healthy ntlars backup and one backfilled row showed nothing at all. Hiding a working backup is the one thing that badge must never do. A NULL confirmation is not stale, it is unmeasurable. The stalest kind is now chosen among kinds that can actually be judged, which is the rule staleness.py already applies: a chain becomes measurable the first time its PC posts and says nothing until then. If every kind is unmeasurable the host still reports which kind it has, with a None verdict and no badge. The test that caught it pinned the old newest-wins behaviour, which part markers made wrong; it now pins stalest-measurable-wins and keeps its NULL clause, which was right all along and is what found the bug. Also worth recording how it was missed: the change was verified against test_geenforce_reporting.py and test_backups.py, and this assertion lives in test_geenforce_publish_gate.py, which was never run. |
||
|
cproudlock
|
500f7d6ad6 |
Give the backups plugin a settings page, and show where a snapshot is
The plugin shipped five settings and no page for any of them. The core settings catalog is a hardcoded list a plugin cannot extend, so backups_shareroot and the collection interval were reachable only through the API or a flask shell on the server - which is not a configurable setting in any useful sense, and it is how today went: the setting simply was not there to set. The page is contributed the ADR-010 way rather than by editing core - a card from get_settings_cards plus a route in the plugin's own routes.js, both of which disappear when the plugin is disabled. Filed under Machines, next to Machine Types, because that is where the assets these back up live. It also states the two things the settings do not say for themselves: that health is judged on when a backup was last CONFIRMED rather than when it last changed, and that retention prunes database revisions only - files on the share stay, so enabling it on a share-backed kind leaves orphaned snapshot directories. Snapshot paths are now shown as full UNC text in both places: a Snapshot row on the Marker Contents card, and the existing path on the history page, which gains a copy button. Deliberately NOT a link - browsers refuse to navigate from an https page to file://, so a clickable path looks actionable and silently does nothing, which is worse than no link. Copy, then paste into Explorer. The path comes off the revision, never composed from the configured share root: a snapshot written before the layout changed still lives where it was put. |
||
|
cproudlock
|
ee208808f4 |
Judge a PC's backup by its stalest kind, not its newest
Until part markers, a PC reported exactly one backup kind and "newest revision wins" was the same thing as the truth. A marker bay reports two: ntlars for the machine's DNC settings, partmarker for the controller's patterns. With newest-wins, the daily ntlars post keeps the badge green while the marker backup has been failing for a month. The failure shows only on that marker's asset page, which is not where anyone looks to find out something stopped - and "is this still being checked" is the one question this badge exists to answer. So a host is green only when everything it backs up is current, and the badge names the kind that is not. Freshest revision per kind first, because lastseenat only ever moves on the newest revision of a chain; judging every row would report a healthy, frequently changing config as permanently stale. Known consequence, and the right direction to err: a device retired from a PC leaves its last revision behind, so that host stays stale until the revision is pruned. It complains about a backup that genuinely is not running rather than staying quiet about one. The dashboard card needed no change - it already lists every stale chain independently, so a stale partmarker chain was never hidden there. |
||
|
cproudlock
|
512cc9be76 |
Let a share-backed backup say what is in it
A 'share' kind keeps its bytes on the file share and gives ShopDB only a hash and a UNC path. That is right for an opaque vendor file, but it means the app cannot answer the question a tech actually arrives with - what does this part marker hold, and which pattern disappeared last month. Reading the share from the app server is not an option: the download routes refuse to mount it on purpose, because a permissions slip there becomes an unexplained empty panel. So the collector now accepts an optional manifest alongside the hash: the index, not the content. Names, sizes, per-file hashes, and the marking system each pattern was authored for. Advisory throughout - a client that sends none behaves exactly as before, and dedup still keys on contenthash. The marker asset gains a Marker Contents card built from it: counts by type, the controller family, and a diff against the previous snapshot in the same source chain. Removed files are named rather than counted, because "1 removed" only tells someone to go and look. Files edited in place are named too - the file set can be identical while the content hash moved, and reporting no change there would contradict the reason the revision exists. The manifest is deliberately NOT covered by contenthash, which attests the bytes on the share. A client that starts describing itself therefore posts an unchanged hash, so a no-op backfills a manifest the row does not yet have - otherwise the card would appear only on markers that had recently changed, and stay hidden on every healthy one. The same no-op refreshes sharepath, so a snapshot that moved on the share stops being pointed at where it used to be. Manifests are validated at ingest rather than at render: a malformed one would otherwise become the latest revision and fail every view of the asset page until the content next changed, outliving the client bug that caused it. sharedir() for the marker kind is overridden to the layout the site actually uses, so what ShopDB displays and validates matches what the collector writes. |
||
|
cproudlock
|
8750ba2e74 |
Let a site open secure send to guests, and choose how many times a link opens
Two asks, and the first is a real security decision so it is a SETTING that defaults to the safe answer rather than a code change somebody inherits. securesend_allow_guests, OFF by default. With it off the model is unchanged: a login plus tools.secrets. With it on, somebody with no account - at a shared bay terminal, a contractor - can create a link too. The cost is named in the setting's own description and pinned by a test: a guest secret has NO CREATOR, so the audit trail can say when and from where but never who. That is the whole reason it is a switch a person turns on deliberately. IT FAILS CLOSED. A missing row, a half-migrated site, an unhappy database - the answer is no. An error must never be the thing that opens an unauthenticated write endpoint. A SIGNED-IN USER STILL NEEDS THE PERMISSION even when guests are allowed, which is surprising enough that a test pins it: opening the door to guests is a decision about the door, not about revoking what a role holds. Guest creation is rate limited per address, same shape as the login limiter. Not a security boundary - a determined person has more addresses - but an open endpoint should not be an easy place to park data or fill a disk. VIEWS ARE NOW A NUMBER, 1 to 100, instead of a four-item dropdown. How many people need a password is the sender's business: a shift handover is not one colleague on a call. The floor is what matters, not the ceiling - unlimited views would just be a password on a web page. Also folded in, from the retention question: nothing removed a secret that EXPIRED WITHOUT BEING OPENED. The recipient goes on holiday, the link is never clicked, and the ciphertext sat there for good - not readable, but it made "nothing lingers" untrue and grew the table in exactly the case where the feature failed. Any create now sweeps expired rows and a late click tidies its own, so it stays bounded without a scheduler an air-gapped site would never add. Two gates caught mistakes on the way: current_user is a LocalProxy and is never None, so `current_user is None` silently took the signed-in branch for a guest; and cache was imported straight from shopdb.extensions when it is already on the shopdb.api contract surface. 1836 tests green. |
||
|
cproudlock
|
d4b49b91b4 |
Move secure send into Tech Tools, where somebody would look for it
|
||
|
cproudlock
|
98458672eb |
Hand somebody a password without leaving it in a chat window
The habit this replaces is pasting a credential into Teams, where it stays until someone deletes the conversation. Tabled on 2026-08-21 pending a decision about scope; the decision was text only, so this is the half that survives it. BACKEND ONLY. There is no page yet, so nothing here is reachable by a person. TEXT, NOT DOCUMENTS, and that shapes everything. Dropping files removed the export-control question entirely - a password is not technical data - along with blob storage, size and MIME handling, and the need to authenticate the recipient so a document fetch could be pinned to a name. TWO DOORS. Creating needs a login AND securesend.create: anonymous creation would be an untraceable way to move data out of a system that otherwise records who did what, and the creator is the only person these rows can ever name. Reading needs neither, because the recipient usually has no account here and the link is handed over on a call - the unguessable token IS the authorisation, which is why it is 128 bits of randomness and why the row is spent on first read. REDEEMING IS A POST, and that is the detail that would have broken this in practice. Teams, Outlook and Safe Links prefetch GET urls to build previews and scan for malware, so a GET that redeemed would let a preview window consume a secret before its recipient ever saw it. The GET route that remains reports only whether a link is still live, so the read page can say "opening this will destroy it" without spending the only view to find out. THE SIZE LIMIT IS THE WHOLE ENFORCEMENT. The key lives in the URL fragment, which browsers never transmit, so the server holds ciphertext it cannot read and cannot tell a password from a page of a drawing. MAX_CIPHERTEXT is therefore not a tuning knob: raising it quietly turns this into the document transfer it was scoped not to be. The refusal says what the limit is FOR, because somebody hitting it is usually trying to send a file and the answer is a different tool. Line counts, file signatures and refusing a pasted file belong in the browser, guide an honest person, and are bypassable by anyone willing to POST directly. NOT ENUMERABLE. Missing, expired and spent all answer identically - telling them apart would confirm to anyone holding a guessed token whether it ever named anything. A spent or expired row is deleted on the way past rather than waiting for a sweeper, because it has no reason to keep its ciphertext. Its own plugin rather than a Tech Tool: that plugin is browser-only by design, and this needs storage, expiry and a view counter. Not an asset extension either, so no assetid and no asset type - the scaffold assumes one and it was removed. FOUND BY ITS OWN TESTS: expiryhours=0 returned 201. `int(x or DEFAULT)` treats an explicit zero as absent, so "expire immediately" silently became "expire in a day". A caller who sends a number is now told when it is not allowed rather than quietly given a different one. BEFORE ANYONE RELIES ON THIS, test that your link rewriter preserves URL fragments: create a secret, send the link to yourself in Teams, click it from the message, and check the key is still after the #. If Safe Links strips it the secret is unrecoverable and the server cannot help, which is the point of the design. The fallback is passphrase mode, worth building only if that test says so. Three product gates caught what a new plugin has to declare, and each was right to: the route-parity gate wanted the endpoints described in api-inventory.json, the lean-build guard wanted the plugin listed in the universal profile the released installer builds from, and the authz sweep wanted the public redeem route named in EXEMPT_ENDPOINTS with a reason. That last one matters most - a mutating route that answers an unauthenticated caller is a decision somebody has to write down, not a default, and reading a secret is a mutation because it spends it. |
||
|
cproudlock
|
3703412baf |
Only zero is empty, and the last day is worth reading
Two separate floors were collapsing a live cartridge into a spent one, and removing either alone changes nothing. EMPTY_LEVEL was 5, so everything at or below 5% was assigned daysleft 0 outright. The reasoning was that a printer at 1% is out of toner as far as anyone standing at it is concerned. But the row exists to say how long is left, and 2% draining a point a day has two days in it. Flooring put that cartridge beside ones that genuinely are empty with no way back to the difference. It is 0 now: empty means empty. int(level / rate) then truncated the division, so anything under a full day arrived as 0 whatever the floor did - a cartridge with six hours in it was indistinguishable from one with nothing, and the band read it as empty. daysleft is fractional now, rounded to two places, which is about a quarter of an hour: finer than the estimate deserves, but it costs nothing and keeps the ordering of two nearly-spent cartridges meaningful. daysText reads in whatever unit carries meaning: "6 hours", "1 hour", "29 min", "4 days". Below an hour it goes to minutes with a floor of one, because rounding hours would land back on "empty" - the same bug one rung down. BEHAVIOUR CHANGE worth knowing: a cartridge at 1-5% with NO history used to get daysleft 0 from the floor and land on the order list. A rate needs two readings; with none there is nothing to divide, and the old answer was right by accident - it said "empty" about a level nobody had watched move. It now reports 'no history' and shows under "No estimate yet". This reaches only printers newly added to Zabbix; anything that got to 3% the ordinary way has the history to forecast from. Five existing tests pinned the old rule. They recorded a real decision, so they are rewritten to the new one rather than deleted. One of them was passing for the wrong reason: its series (2.0, 1.7, 1.4, 1.1) drops 0.9, under MIN_DROP_FOR_ESTIMATE, so it never had a rate at all and only passed because the floor short-circuited ahead of the rate check. It now uses a real 20-day drop at a tenth of a point a day and asserts the ten days its docstring always described. The forecast fixture's black cartridge moves from 1% to 0% so the empty band keeps its API-level coverage, and a magenta at 3% covers hours-left end to end. |
||
|
cproudlock
|
b7cfd2b198 |
A supply nobody could read is unknown, not empty
Reported as "CSF04 now reports all 0%", on the printer's own page as well as
the reports. Every supply on one host reading zero at the same instant is not
what a set of cartridges does; it is what a host that stopped answering looks
like.
getsuppliesbyip turned every unreadable item into 0:
try:
level = int(float(item.get("lastvalue", 0)))
except (ValueError, TypeError):
level = 0
An item that has never collected returns lastvalue as an EMPTY STRING, so
float('') raised and the handler substituted zero. lastclock was fetched in the
same request and read by nothing at all, so a reading from three weeks ago was
presented as current.
Zero is a legitimate level meaning spent, which is what made the substitution
dangerous rather than merely wrong. Downstream it reached `level <=
EMPTY_LEVEL`, which set daysleft to 0, which put the printer on the order list:
the report asked purchasing to buy a full set of cartridges for a printer
nobody had measured. On a waste container it was worse - waste is scored as
100 - level, so an unreadable one read as 100% full, the alarm state.
_readlevel now returns (level, lastseen) with level None for no usable reading:
empty, missing or unparseable lastvalue, or a lastclock older than 24h. A
lastclock that is absent is not treated as stale - a reading that cannot be
aged is not thereby wrong, and discarding it would lose real levels. A genuine
"0" is still 0, which is the half that keeps an actually-empty cartridge on the
order list.
classifysupply gains an 'unknown' status with remaining None. It is not
critical: critical is a claim someone acts on, and it may not be made about a
supply nobody measured.
The low-supplies report skips unknown rather than listing it as needing
replacement, and counts it in summary.unknown - a printer that has gone quiet
should be visible as that, not absent and not masquerading as empty.
_annotate_supply reads .get('level') with no 0 default, which would have put
the whole bug back.
PrinterDetail already rendered `level !== null ? ... : 'N/A'`, so it starts
telling the truth as soon as the backend sends null; the UI had been written
for a case the service never produced. It gains the unknown style - muted, not
a severity colour - and a tooltip separating "no reading at all" from "no
reading since <date>", which are different faults with different fixes.
Alerts needed no change: float(None) raises and that loop already continues,
so unknown supplies stop firing low-toner emails as a consequence.
12 of the 16 new tests fail on the old code. The 4 that pass either way pin
that real zeros still behave, which is the regression this fix could cause.
Not verified against CSF04's actual Zabbix data - dev has no reachable Zabbix.
This fixes the mechanism that turns unknown into 0%; /api/printers/<id>/supplies
now distinguishes them, with null for no reading and 0 for a measured zero.
|
||
|
cproudlock
|
d0eeaa08d5 |
Find a knowledge base article by words from different fields
An article's searchable text lives in three places: its title, its keywords,
and the name of its topic. Nobody typing a search knows or cares which word
came from which, so 'CMM Community' means "the article tagged community, under
the CMM topic". Both search paths returned nothing for it.
The plugin's own listing, which is what the KB page calls, built a single
ilike('%CMM Community%'). That needs the whole phrase contiguous in ONE field,
and no article has it: 'CMM' is only in the topic's name, 'Community' only in
the keywords. Global search already split the query into words but only ever
looked at title and keywords, so a word that only the topic could satisfy
failed there too - the same bug wearing a different face.
Every word must now be found somewhere across all three fields, in any order.
_word_match gains an `extra` hook for a word that a RELATED row satisfies
rather than a column of this table, which is how the topic joins in without
colliding with the sort=='topic' join.
The retired-topic rule is unchanged and pinned on both paths: matching more
words is not a way past it.
Verified against the 235-article dev library, where all three of these
returned nothing before: 'Fieldglass Jefferson' (title word plus topic word),
'outage notification' (two keywords, not adjacent), 'compucom Jefferson'.
The same phrase-only pattern is repeated in about ten other list endpoints
(computers, machines, printers, network, printedparts, measuringtools, usb,
notifications). Left alone here - fixing them properly means promoting the
word-match helper onto the shopdb.api contract surface rather than copying it
per plugin, which is a contract bump.
|
||
|
cproudlock
|
875fde9f48 |
Fix what the last round of device fixes broke, and two dips it missed
A review of |