Two tracked files had been edited directly in the work-PC relay clone, which
only fetches bundles and pushes them to GitHub. Nothing there reaches gitea, and
the next bundle pull would have overwritten both silently. Brought across here,
with three changes to what was written.
Kiosk links now need the route AND the plugin. The edit swapped hasRoute for
isPluginEnabled, which trades one wrong answer for another: a full build stages
every route, so a site that turns printedparts or usb OFF still saw a link to a
kiosk its backend refuses - the symptom being fixed - but isPluginEnabled fails
OPEN when it cannot reach the plugin list, and a copied registry can report a
plugin that was never staged into this build. That is what the comment above
those lines was already warning about. Both tests together answer both states.
The wizard shows the filename it is uploading. The browser puts it in chrome the
full-screen wizard hides, so an operator picking a file got no confirmation it
was the one being sent.
Image previews get a cache-buster. The stored path does not change between
uploads, so the preview showed the PREVIOUS logo or blueprint and the upload
read as having silently failed. Display only - saveStep persists form values,
never these, so no cache-busted URL is ever written to a setting.
The wizard also loads the current site logo on mount, so re-running it does not
present an empty slot as though branding had never been set. The shipped-asset
fallback stays a root-absolute path: the template applies withBase, and prefixing
it here as well produced /shopdb/shopdb/ge-aerospace-logo.svg on a subpath mount.
Frontend 391 passing.
Scope: product
A local 'dotnet build' leaves agent/*/obj and agent/*/bin full of absolute
host paths. They are gitignored, but the export rsyncs the WORKING TREE, so
.gitignore does not apply and the scrub gate stopped the whole export - which
is what it is for, but it made publication depend on whoever built the agent
last having cleaned up.
Same shape as the stray .env.bak the excludes already document. Note the trap
that follows from it: an --exclude also PROTECTS a path from --delete, so the
copies the first failed run had already placed in the publication tree had to
be removed by hand before the next run could pass.
Scope: product
Three defects that all shared a shape: the restore path looked checked and was
not.
An unreadable data revision was treated as a legacy set. backup.sh discards the
mysql error when reading alembic_version, so a set could be written with an
empty datarev; the compatibility check then fell through to the code-to-code
head comparison - the very check datarev was introduced to replace - and badged
the set restorable on no evidence at all. A manifest that CARRIES the key and
left it empty is now refused, while one with no key at all is still the genuine
legacy case and keeps the fallback. backup.sh records the failure in run history
rather than passing it over in a log line, since the set it just wrote cannot be
restored.
Dumps were bound to the schema they came from. mysqldump --databases bakes
CREATE DATABASE and USE <name> into the file, so restoring onto a site whose
MYSQL_DATABASE differs loaded every row into the dump's schema, left the live
database untouched, and reported success. New sets are dumped without it and
load into the schema named at restore time. Existing sets are still bound, so
the gate reads which schema they name and refuses a mismatch instead of
restoring beside the live data. Also dropped --no-tablespaces into the main
dump: it needs the PROCESS privilege the app user does not have on MySQL 8.
The gate's own test carried a copy of the gate. test-restore-datarev.sh
re-implemented the predicate in its own shell functions, so it would have stayed
green through any drift in restore.sh - including the plugin-chain hole fixed in
the previous commit, which it could never have caught. restore.sh now takes
RESTORE_CHECK_ONLY, which stops it immediately after the gates having touched
nothing, plus overridable paths for the migration and plugin trees. The test
drives the shipped script and asserts on its exit code, so the verdict under
test IS the verdict that ships. Confirmed non-vacuous by planting a violation:
disabling the plugin revision check turns the suite red.
Nine cases now, up from five, covering both plugin-chain directions, a plugin
the build does not have, and the schema binding.
Moving the schema check earlier was prompted by that test: it had been placed
after the safety copy, so it needed a live database to reach a verdict it can
reach for free. It sits with the other gates now and stops before anything is
written.
Round-tripped against a real stack after the dump format change: backup, mutate,
restore, data back to backup-time state, and the new dumps confirmed to carry no
schema name.
Suite 2607 passed / 9 skipped.
Scope: product
The compatibility gate read alembic_version and nothing else, but plugin schemas
move independently of core. Roll the code back to an older plugin revision and
the core head can be identical, so a dump carrying plugin tables the code has
never heard of passed the gate, loaded - destroying what was there - and only
then failed in 'flask plugin upgrade-all' with "Can't locate revision". A failed
restore that has already replaced what it was restoring over is the worst
outcome this feature can produce, and it was reachable from an ordinary rollback.
Backups now stamp the applied revision of every alembic_version_<plugin> table
alongside the core one, read from the dumped database the same way. restore.sh
checks each against that plugin's own versions directory and refuses before
touching anything, naming the plugin and the revision to deploy past. A plugin
in the dump that this build does not have at all is refused for the same reason.
The list endpoint's compatibility badge asks the same question, so the UI cannot
offer a set the restore will reject.
Sets taken before the stamp existed carry no dataplugins and keep their old
core-only verdict. Their plugin revisions are unknowable, and making every
existing set unrestorable to close this would be a worse trade than the gap.
Proven against a real stack, reproducing the exact case: core head identical,
endpoints data stamped at a revision the build does not have. The restore
refused, the live row was still there afterwards, and it stopped early enough
that the pre-restore safety copy was never needed. The same set with its true
revision restored normally.
Suite 2605 passed / 9 skipped, with the gate covered both ways plus the legacy
fallback.
Scope: product
Closes the three defects in the imported restore path, at Ben's request. Each
was proven against a real stack before and after: two backup sets with different
data and different instance files, then a restore of the OLDER one, which is the
case that was broken.
A restore no longer depends on a tab staying open. The backup sidecar has no
Flask, so it loads the dump and leaves a sentinel for the api container to run
the migrations. The only thing that ever acted on that sentinel was the poll
loop of the modal that queued the restore, so closing the tab - or letting a
token expire - during the minutes a load takes left the data at the old revision
under new code, every query on a newer column failing, and nothing retrying. A
watcher in the api container now claims and runs it. The endpoint no longer runs
the upgrade at all; it only makes sure something is pending, which is also how a
failed upgrade is retried. That takes Alembic out of the request path, where it
could outrun the gunicorn worker timeout and be killed mid-chain, leaving a
half-applied revision the next attempt trips over.
The claim is a rename, because rename(2) is atomic: several workers and a
browser poll can race the same sentinel and exactly one wins. A failed upgrade
parks itself as .upgrade-failed rather than re-arming, since putting it back
would rerun the same broken chain on every poll forever.
Instance files come from the set being restored. The restic call asked for
'latest' regardless of which set was named, so restoring anything but the newest
paired that set's DATABASE with the NEWEST snapshot's uploads and plugins.json -
a mismatch nothing downstream could detect. Now scoped by the set's own tag,
which backup.sh has always written. The tree is replaced rather than restored
over, so files created after the snapshot are gone afterwards: a restore has to
put the site back to the snapshot, not merge into it. Done as a staged swap
rather than 'restic restore --delete --target /', because that aims a delete
pass at the container root and the cost of getting its filter wrong is the whole
filesystem.
Nothing destructive happens before a copy is taken. The dump carries DROP TABLE
per table with no transaction around it, and the instance tree is replaced
wholesale, so a wrong set or a failure part-way through left nothing to go back
to. restore.sh now writes a pre-restore-<stamp>/ set - dump, instance tarball
and a README saying what it is and how to load it - and refuses to touch
anything if that copy cannot be written.
Two more found while proving the above, both of which stopped the overlay
working at all:
* Every script under scripts/backup/ was committed non-executable, while every
other .sh in the repo is 755. The compose entrypoints are
/opt/backup/watch.sh and backup.sh from a read-only bind mount, so the backup
and watcher containers could not start from a clean checkout on any machine,
and nothing could fix it at runtime.
* Dockerfile.backup could not build without the corporate CA staged. A plain
COPY of the gitignored cert fails outright; the trailing-glob form does too,
because BuildKit fails a COPY whose pattern matches nothing - which is exactly
the case being handled. Copying the directory and testing for the file inside
it is the form that works both ways. Verified by building the image with no
cert present.
Suite 2600 passed / 9 skipped. The finalizer has unit tests covering the atomic
claim, the parked failure and the endpoint no longer running the upgrade.
Scope: product
Six findings from a four-agent review of the imported work, plus the contract
bump the import owed. Nothing here changes what the imported features do.
The contract version was not bumped for a change that widens what a plugin may
declare. The asset-panel vocabulary gained two tab groups (serviceprocess,
reports), a datetime column format, a map.body key and a tabs section that
renders as a table. 0.29.0 bumped for exactly this class of change when the
optional group was added, so this is 0.31.0 with a ledger row. All four are
optional and resolve server-side when absent, so a plugin declaring none of them
is unaffected.
Site Backups now answers honestly where it has nothing behind it. The overlay is
a compose profile: a separate container writes sets onto a shared volume and the
api container only reads them. A Windows/IIS install runs no such container and
the default root resolves to a drive-letter path nothing creates, so Back Up Now
either failed on PermissionError or reported a run queued that would never
happen, under a banner reading "No verified backup yet" - which describes a
backup that failed, not a feature that was never installed. The three write
routes refuse with the same 404 shape the configuration-policy routes use for a
disabled feature, the list and status routes report whether a store exists, and
the page says so instead of offering buttons.
Two listAll callers still read a response that is not there. listAll resolves to
a plain array through fetchAllPages, so response.data.data throws: PC Types
settings rendered empty, and in Configuration Policies the throw happened inside
the shared try, so PC types, applications, the action catalog and monitors were
all left unassigned by one bad read.
The agent package upload cannot succeed on IIS as shipped. MAX_CONTENT_LENGTH
went to 256 MB to admit the ~69 MB tray binary, but IIS request filtering caps a
body at 30,000,000 bytes and answers 413 before waitress sees the request, so
web.config needs the matching requestLimits.
The backup image could not be built by anyone outside a TLS-intercepting
network. Dockerfile.backup did a plain COPY of a gitignored corporate root CA,
and a COPY of a missing file fails the build. The trailing glob makes it a no-op
when the cert is absent.
endpointstates.domain was the one new string column written without a clamp.
Under STRICT_TRANS_TABLES an over-long value raises Data too long and aborts the
whole check-in, so a single malformed report would stop a machine reporting
telemetry and events entirely.
Not fixed here, and left with upstream: the restore path finalizes its schema
upgrade only from a browser poll loop, so closing the tab mid-restore leaves the
data behind the code; it restores instance files from the newest snapshot rather
than the set that was asked for; and it takes no safety dump before the
destructive load. All three are in the Docker-only overlay, which this site does
not run, and none can be validated here.
Scope: product
Ports 37 of Benjamin Baker's commits from the wilm branch, mirrored from the
corporate GitHub at 7905917c. wilm rides the publication lineage and shares no
ancestor with wj, so this is a filtered tree patch rather than a merge:
94 files, and every path the publication export strips (docs/, tools/, mcp/,
CLAUDE.md, start-*.sh, .gitea/, the three excluded tests) is held back, because
a raw diff would delete each of them from this tree.
What comes across:
* Endpoint telemetry, phase 2. Physical disk health from two sources, NIC link
state, boot type, Defender health, processor/memory/chassis inventory, the
listening TCP set, mapped network drives and Windows Update history. Five
additive migrations, 0012 through 0016, off the 0011 head.
* Mapped drives are collected by a probe running in the interactive user's
session, because they do not exist in the agent's session 0. Two Windows
sources can disagree about one drive and the worse status wins, so a
half-broken share is never rendered as healthy.
* Windows events are reported once each, keyed on the channel's own record id,
instead of being re-sent whenever a report overlapped the last one.
* PC Detail rework: Details cards flow, the hand-written Network card gives way
to a full address history, Storage and disk health merge into one card with
two sections, Operator Reports and Corporate Tools get their own tabs.
* An opt-in restic backup overlay with a third container, restore gated on the
dump's own data revision rather than the code head, and backup/restore history.
* The agent's version is read from the binary's PE version rather than trusted
from the csproj, and the Fleet view gains an Agent Version column.
* The enqueue route now asks caller_holds instead of the owner's permissions,
which closes the scoped-token elevation on run_script, update_app and
system_restart.
Held back deliberately:
* agent/installer/install-shopdbagent.ps1 stays deleted, as upstream deleted it.
The installer that reaches a bay is generated by _install_script() in
plugins/endpoints/api/siteagent.py; the standalone script had stopped being
shipped and was collecting fixes no endpoint ever ran.
* The .gitignore hunk adding docs/*.md. Correct on the publication lineage,
where docs/ becomes the wiki and its absence is how the gates detect a
published checkout. Here it would silently ignore every new doc.
* Their deletions of docs/DEV-BUILD-TEST.md and docs/PLUGIN-HOOKS.md. Upstream
those were stubs; here PLUGIN-HOOKS.md is the 31KB document CLAUDE.md points
readers at.
* Five zero-byte files committed by an accidental `git add -A`, a run log from
the author's own machine, and docs/notes/, which is working scratch that says
so in its own header and belongs in devnotes/ on this side.
Carried with the import so it lands green: the 17 new routes are described in
docs/api-inventory.json with the spec and project map regenerated, and two new
frontend specs read fixtures through fileURLToPath(URL) which node rejects under
jsdom's own URL global.
Suite 2591 passed / 9 skipped; frontend 391 passed; naming, style and doc gates
clean.
Scope: product
Phase 3 of the device-identity proposal moves device identity to
HKLM:\SOFTWARE\GE\ShopDB\DeviceId, which survives the profile cleanup and disk
hygiene that eats C:\Enrollment and needs no new key: the collector already
opens that path for BaseUrl and ApiToken. This is the backfill that gets a
value in there without re-imaging anything.
The source depends on the pc-type, and that is the whole point. DeviceId names
the DEVICE, while a machine number answers "which bay is this" - a different
fact that coincides for exactly one family:
CMM the machine number IS the instrument id (CMM4), which is why
cmmid.txt has always doubled as identity. Copied.
part marker the machine number is the OPERATION, and several markers serve
one operation. Copying it would name the operation as the
device, which the collector refuses as the wrong kind of asset
because a machine filed under a device label reads as a marker
everywhere downstream. Never copied.
Keyence, no per-bay id has ever existed; asset-id.txt is the only
Genspect, identity these have.
wax-trace
anything else not a device bay, so nothing is written. DeviceId is not
consumed for those pc-types and a value left there is a
landmine for the day one gains a device spec.
Resolution per family mirrors Report-AssetToShopDB.ps1, including the 9999
placeholder skip and reading WOW6432Node before the native eDNC key.
Refuses to overwrite a DeviceId that disagrees, unless -Force. asset-id.txt is
written once at imaging and nothing updates it on a device swap, so a stale
file clobbering a corrected registry value would reintroduce the exact defect
this move exists to fix. GE-Enforce already reached the same conclusion for
machine-number.txt.
Verified in the win11 VM across ten cases: not-a-device bay, part marker,
CMM precedence, idempotent re-run, machine-number exclusion, eDNC fallback,
9999 skip, stale-file refusal (exit 2), -Force, and -WhatIf. The part-marker
guard was then planted - flipping it to treat the machine number as identity
writes operation 0613 into DeviceId, so the passing test is not vacuous.
Scope: product
ADR-005 split machines from measuring tools on the grounds that they are two
asset classes. It named the metrology population as CMMs, vision systems, wax
trace and Genspect; the plugin was then built against a docstring naming
calipers, micrometers and thread gages. Those are not the same population, and
the difference is the whole decision: bay-mounted capital equipment with a
controlling PC on one side, hand tools in a drawer on the other.
Sixteen months on, measuringtools holds zero rows against 204 machines. The
metrology inventory waiting to be entered is 134 instruments - 35 CMM, 32 form
tracer, 23 vision, 18 wax, 14 measuring, 8 Genspect, 2 eddy, 1 spline - every
one bay-mounted with a controlling PC, and not one hand gage among them. CMM is
already a machinetype, and Part Marker is already a machine while Form Tracer
is a measuring tool, so the line is already drawn somewhere no operator can
predict.
Retires the plugin and the measuring_tool asset type. Calibration is not lost:
it moves onto machines as four nullable columns with status still derived at
read, which is a gain rather than a port because a machine could not record
calibration at all before. ADR-005's one concrete objection - that "show me all
measuring tools" would need an enumeration of type names - is answered with
machinetypes.ismetrology, so it stays a single-column filter.
Taken now because measuringtools is empty. After the 134 rows are entered they
carry relationships, audit history, map positions and collector links, and the
migration stops being a drop.
Scope: product
The adoption guide told an adopting site that a missing EventSaver.ini makes
the binary fall back to "the reference site's file server". That was true of
EventSaver.cs until the ADR-015 fix set Folder to empty, and EVENTSAVER.md was
updated then while this page was not, so the two documents disagreed about what
a missing ini does.
Current behaviour is that it shows nothing, which is the point: failing visibly
beats displaying another site's slides. Says that, and adds the case the old
text was accidentally describing - a .scr compiled before the fix still carries
the path, so a binary somebody was handed is not the same as one they built.
Scope: product
CONTRIBUTING.md still said "main is the product every site runs and the
publication mirror exports" and framed wj/wilm as site branches feeding it.
That was true when main was the only thing anyone deployed. It is not now:
WJ production runs wj, the /ops instance runs dev, and main deploys nowhere.
Nothing in the repository mentioned dev at all, so the branch a deploy script
refuses to run without was undocumented.
Replaces that section with the branch/instance table and the dev -> wj -> main
promotion order, and says why promoting before validating is the wrong way
round. Keeps the wilm note but moves it up with the reason it cannot be
merged locally: it sits on the publication lineage and shares no ancestor.
CLAUDE.md gains the branch-to-server pointer and records that
export-github.sh now builds a dist only for wj and dev.
docs/DEPLOY.md is deliberately left alone: `git pull origin main` is correct
there. That document is the generic connected-Docker deploy another site
follows, and main is exactly what another site adopts.
Scope: product
Taken from the other site, who hit it in production today (their commit
db75ca7): agents that fail the TLS handshake retry in a loop, and with four
blocking sync workers a handful of them pinned every worker. Real UI requests
queued for 20+ seconds while the CPU sat idle - the box was not busy, it was
blocked.
gthread, 4 workers x 8 threads, and a 30-second timeout rather than 60. A stuck
connection now holds one thread out of 32 instead of one worker out of four, and
it lets go in half the time, which is the difference between absorbing a retry
storm and queueing behind it.
Their server, our agent. The client behaviour that caused it is the endpoint
agent, so it will reach any site running one - which makes this a product fix
rather than something that belongs on one site's branch.
Applied to the Dockerfile as well as docker-compose.yml. A compose file
overrides the image CMD, but docker-compose.airgap.yml, .channel.yml and
.pull.yml do not specify a command at all, so they take the image default - and
so does anybody running the image by hand. Fixing only compose would have left
every air-gapped install on sync workers.
WJ production is unaffected either way: it runs waitress under IIS, already with
--threads=8, so it was never exposed to this. That is worth knowing before
anyone treats it as a prod hardening item that was missing.
Scope: product
The rule for this site, stated plainly: only `wj` reaches /shopdb (WJ
production), only `dev` reaches /ops, and `main` is the curated release other
sites adopt and deploys nowhere. The tooling did not know that, and in two
places actively assumed the opposite.
export-github.sh refused --dist on any branch but main, because production was
assumed to install main. It does not - it installs wj. That guard fired today: a
wj export produced its bundle and then died at the dist step, so the dists on
the share came from the main export that happened to run next. The code going to
production and the frontend going to production would have come from different
branches, silently, and nothing downstream could tell.
Now one branch builds one dist: wj builds the /shopdb dist, dev builds the /ops
dist, and any other branch is refused because a dist it built could only ever be
installed by mistake. Each dist carries a .dist-source stamp naming the branch,
commit, base path and build time.
The two deploy scripts copy from the clone on WHATEVER branch it is sitting on
and only ever checked the destination (the web.config mount path), so a clone
left on the wrong branch deployed the wrong code to a correctly-identified
server without a word. Both now refuse unless the clone is on their branch, and
both read the dist's .dist-source and refuse a frontend built from anywhere
else. A directory of hashed filenames says nothing about where it came from, and
robocopy /MIR will mirror whichever one it is pointed at.
pull-shopdb-bundle.cmd and update-shopdb-github.cmd silently defaulted to main.
That was safe when main was the only thing anyone deployed; with three branches
it is a way to carry the wrong one to a server without ever typing its name. The
argument is required now, and the error lists what each branch is for.
tools/mirror-wilm.sh is the missing half of fetch-wilm-branch.cmd: that script
only puts a bundle on the share, and wilm does not appear on gitea by itself.
The new one fetches the carried bundle and pushes wilm to gitea, printing the
bundle's age first - a mirror that looks authoritative and is a week old is
worse than no mirror - and noting that wilm sits on the publication lineage, so
it shares no ancestor with main/wj/dev and cannot be merged here.
Scope: product
The last two items from the run.
The PC detail page has always rendered a Business Unit row, and the API has
always accepted the field. Only the form had no control, so it could be read and
never set from the UI - a value that appeared the moment somebody set it through
the API and could not be reached otherwise. The form has the picker now, loaded
with listAll() rather than a capped page.
Global search excluded inactive assets, so a retired record existed, was
reachable by URL, and could not be found by anybody who did not already know
where it was - including whoever wanted to restore it, which the retire dialog
promises they can. Retired assets are searchable now.
That is only safe with the ordering, so it is part of the change rather than a
follow-up: somebody searching a machine number almost always wants the machine
still on the floor, so a retired match drops 60 points of relevance, sorts below
every live one, and carries a 'retired' flag the results page renders as a
badge. Indistinguishable retired results would be worse than none.
Four tests, verified by planting the isactive filter back: three fail.
Scope: product
The last two kiosk findings.
Keypad entry collected EVERY digit in an item's code, and the shipped prefix
contains one: '3DP0004' collapsed to 30004, so an operator typing the 4 printed
on the label was told no part matches that barcode and had to work out that the
leading 3 belonged to the prefix. The number on a label is the number at the
end, so the trailing digit run is what is compared now. An ambiguous number is
still refused rather than guessed - taking from the wrong bin is worse than
being asked to scan the label.
A parts kiosk that had never been provisioned drew its normal "Scan the barcode
on the bin" screen and answered "Not found" to every scan, which reads as "we
do not stock that part" rather than "this screen was never set up". The server
cannot say which: the routes answer a bare 404 on purpose, so a caller without
the device token does not learn the door exists, and that stays. But the BROWSER
knows whether it holds a token, and saying so discloses nothing - an
unprovisioned screen now says it is not set up and how to fix it.
For the case the client genuinely cannot distinguish - kiosk switched off versus
a part that is not stocked, both a 404 by design - three misses in a row add a
line suggesting the kiosk may have been switched off. Truthful without the
server having to leak anything.
Three tests, verified by planting the all-digits match back.
Scope: product
Three findings, all about a screen or a log saying something that is not so.
Editing a PC wrote an audit entry for its named fields and nothing for the two
changes that matter most when somebody investigates later: the primary IP
address, which is how the machine is found, and the remote-access protocols and
ports, which is how somebody connects to it. Both are on the same form, in the
same request, and both were invisible afterwards - a PC that moved address had
no history of moving, and a PC that gained a VNC listener had no record of
gaining one. Both are recorded now, and re-sending the same value still records
nothing, because an audit log full of non-events is one nobody reads.
While proving that, a silent discard turned up in the same lines: both the
create and the update path did `if ip_comtype:` around the IP write and did
nothing at all when the 'IP' communication type row was missing. Somebody typed
an address, saved, got a 200, and the address was gone. A missing reference row
is a seeding gap, not a reason to drop what was typed, so it is created on
demand.
The PCs list rendered a 403 as "No computers found", which tells a person
without the permission that the fleet is empty and sends them looking for the
data rather than for the permission. It now says which permission is missing.
The Site Policies and Configuration Policies pages rendered their empty table
and an Add button while the policy feature flag was off - every route answers
404 "Policy feature is not enabled", so the button could never work. Both pages
now say the feature is off and that the empty list is a consequence of that,
not of nothing being configured.
Nine tests. The audit entries verified by planting: remove either and two fail.
Scope: product
You are right that this is the one we keep hitting. perpage is capped at 100
server-side, and the cap is applied SILENTLY: a request for 500 returns the
first 100 rows with a 200, so a picker built that way shows the first 100
options and drops the rest with nothing anywhere reporting a problem. The
missing vendor simply is not in the list.
Twelve call sites were still doing it - vendor and business-unit pickers on the
machine and measuring-tool forms, and the type lists behind PCs, machines,
printers, network devices, measuring tools and configuration policies. Each now
either pages through with listAll()/fetchAllPages() or asks for what it can
actually have. vendorsApi and businessunitsApi gained listAll(), which they
lacked, which is why those two sites reached for 500 in the first place.
The gate is the point. tests/test_core/test_perpage_cap.py reads every .vue and
.js source under frontend/src and plugins/*/frontend and fails on any perpage
over 100. It has to read the SOURCE rather than the running app, because the
response is a valid 200 either way - no runtime assertion can see this one.
Verified by planting a perpage: 500 back into MachineForm: the gate names that
exact line.
Separately, /api/auditlogs built its own meta with the pagination fields flat
and a 'pages' key, while every other list returns
meta.pagination.{page,perpage,total,totalpages,hasnext,hasprev}. A client
written against the documented contract found nothing where it looked, on the
one list whose whole purpose is being read by somebody checking what happened.
It uses paginated_response now.
Also carried here, from the previous batch's working tree: Asset.to_dict emits
ownlocationid/ownmapx/ownmapy/ownlevelid alongside the resolved values, and the
PC form binds to those. The merge of an inherited location is right for a reader
and wrong for a form, which could not tell an owned value from a derived one and
wrote the inherited one back as the record's own.
Scope: product
Reported from the floor: somebody types the employee's whole name into the
notification or recognition picker and presses Enter instead of clicking the
suggestion, and the entry does not pull from the employee database.
Two causes, one symptom.
The search matched the WHOLE query against first name or last name, so "Dana
Reyes" found nobody - no single column holds that string. Typing a full name is
what a person does before pressing Enter, and it was the one thing guaranteed to
return an empty list. A multi-word query is now treated as a full name: every
word must appear somewhere in the name, in either column, so "Reyes Dana" works
too, and "Dana Reyes" does not also return Sam Reyes. A single word still
searches broadly, and an SSO still resolves. Both directory backends changed,
the self-hosted tables and the external HR database.
With no suggestion to click, the form then turned the typed text into
`NAME:<text>` - a label linked to nothing. On screen that chip is identical to a
real selection; the difference only shows up later as no photo, no directory
link, and a recognition the person it is about cannot be found by. Both pickers
(the notifications form and the shared EmployeeSearch component) now look a
typed name up before deciding it is free text, through one helper.
An exact full-name match wins over a longer partial one, because somebody typing
"Dana Reyes" means that person and not "Dana Reyes-Smith". A genuinely ambiguous
name is NOT guessed at - the candidates are shown and the operator picks, which
is exactly when picking wrong matters most. A directory that cannot be reached
reports failure rather than silently producing a label.
Eleven tests. The search rules verified by planting the single-term query back.
One note for whoever reads the spec: the failure-branch test uses a thenable
that rejects rather than a mock returning a rejected promise. The runner
collects such a promise as an unhandled error even when the code under test
catches it, and fails the very test that proves the catch works.
Scope: product
Two operator-visible defects and three doc corrections, closing the tail of the
2026-09-03 run.
THE BADGE. The two kiosks read the same physical badges and disagreed. The
parts kiosk understood both shapes a scanner emits - the bare SSO and the PayNo
wrapper 0<digits>BZ - while the USB kiosk compared the raw string to an SSO, so
the wrapper resolved to nobody. An operator whose keyboard-wedge scanner emits
it could take a 3D-printed part and could not check out a drive, with the same
badge, at two cabinets a few metres apart.
The shape now lives in shopdb/utils/badges.py on the contract surface, because
a plugin may not import another plugin and because a badge is a property of the
site rather than of a cabinet. Both kiosks use it, and both store the DIGITS
rather than the string that arrived: one row per spelling makes a custody
history that cannot be joined to itself.
That closed a subtler hole in the same place. str.isdigit() is true for
Arabic-Indic and other decimal forms, and int() parses them, so a badge written
in those characters was accepted, matched a REAL employee, and was then stored
as an SSO string nothing else matches. ASCII digits only. An empty scan and an
unreadable one now get different answers, because they are different problems
for the person at the cabinet.
THE IMAGES. /api/applications/image/<filename> and /api/models/image/<filename>
required a permission, and a browser fetching <img src="..."> sends no
Authorization header - so the gate did not protect the file, it guaranteed the
picture never appeared. Both are ungated now, like the four sibling image routes
(branding logo, map blueprint, employee photo, part picture), and registered in
PUBLIC_ENDPOINTS with that reasoning.
/api/printers/lookup?fqdn= never matched anything: both arguments were compared
against Communication.ipaddress, so a hostname lookup 404d on printers whose
hostname was on the record. An FQDN now matches the hostname, with a fallback to
the short name.
The slide manager gained the per-slide hold time. The PATCH route and the feed
have always carried `seconds`; nothing called it, so the one documented
per-slide control was curl-only.
Docs: API-REFERENCE listed the employee lookups and the whole notifications
surface as unauthenticated when they answer 401, and called both kiosks
"deliberately open" when they are off by default and refuse anything without the
site's device token. The installer download says applications.download, which is
what the code requires. api-inventory.json said printers/lookup was
jwt-optional.
Fourteen tests. The badge rule verified by planting the old per-kiosk behaviour:
three fail, one in each kiosk.
Scope: product
The validation tail from the 2026-09-03 run. Each is small; together they are
the difference between a field that checks its input and one that looks like it
does.
Subnets accepted any string containing a slash, so 'notanip/xx' was stored -
and _devices_in_subnet then swallowed the ValueError from ip_network and
returned an empty device list, so the subnet looked real and permanently empty.
It is parsed with the same library that has to read it later, a prefix length is
required (ipaddress reads a bare 10.1.1.0 as a /32, which is legal and never
what somebody typing a subnet meant), and the stored form is normalised so
10.1.1.5/24 and 10.1.1.0/24 cannot become two rows nobody can tell apart.
VLAN numbers had no range check: 99999 and -5 were stored. Worse, a non-numeric
value reached a duplicate probe that compares against an integer column, which
MySQL coerces to 0 - so 'xyz' matched an existing VLAN 0 and the 409 named a row
that had nothing to do with it. 1-4094 per 802.1Q, checked before the probe.
Deleting a VLAN that subnets still reference now answers 409 like every sibling
in-use guard, rather than 400, which told the caller their request was malformed
when the request was fine and the state was the problem.
An appversion sha256 was stored raw, so an over-long value was truncated to the
column width and came back as 64 plausible hex characters - indistinguishable
from a real digest, and wrong. A digest is a claim the delivery path checks
against, so it is 64 hex characters or it is refused. The releasedate goes
through the strict parser for the same reason it did everywhere else.
PUT /api/settings/<key> ignored the declared valuetype. "A6-not-a-number" went
into an integer key verbatim and every consumer then read 0, because
get_typed_value swallows the ValueError - the setting looked set and the
behaviour was the default.
/api/maplevels told a caller who omitted the level name that 'buildingname and
buildingid are required'; buildingname is a field that handler never reads. It
now names the fields it actually wants.
Restoring a guide revision that is already current writes nothing, because
save() dedups on the content hash - and the response still said "Restored
version N as a new draft version", describing a revision that does not exist
while the unchanged revision list disagreed.
GET /api/backups/revisions/<id>/diff?against=<id> accepted any revision id, so
pointing it at another asset's revision rendered that asset's configuration as
the "before" side of this asset's history: a diff that reads as a change on this
machine and describes a different one. Same asset and same kind, or refused.
The default branch was always asset-scoped; only the explicit one was not.
Two frontend fixes: the per-page selector rendered blank on any list defaulting
to a size not in [10,20,50,100] (network devices uses 25), so the operator could
not see the current page size; and an emptied PC alias was dropped from the
payload instead of sent, so it could not be cleared - the form came back showing
the alias that had just been deleted.
Twenty-two tests. The backups guard verified by planting.
Scope: product
Three credential and authz findings, all the same shape: a second path to a
decision, with weaker rules than the path built for it.
PUT /api/apitokens/<id> {"isactive": true} un-revoked a token, and the ORIGINAL
secret worked again - so "revoked" meant "paused until somebody flips it back",
for a credential whose secret is assumed to be out by the act of revoking it.
Revocation is one way now. Deactivating through the same route still works;
bringing one back does not, and the refusal says to mint a new token.
PUT /api/users/<self> accepted a `password` field with no current-password
proof and no length rule, while /api/auth/change-password requires both. When
two routes reach the same state the weaker one is the real rule, so the stronger
one was decoration. Changing your own password through the users route is
refused and points at the right endpoint. An ADMIN may still set one - that is
how a temporary password is issued - with the same 8-character floor, and
mustchangepassword is now set, because a password somebody else chose is
temporary by definition.
tools.secrets was declared guest-grantable, so the role grid offered it and
stored it, and the secure-send route never consulted the guest role at all: it
decides on securesend_allow_guests, which fails closed and carries the rate
limit. Ticking the box changed nothing in either direction, so the screen and
the behaviour disagreed. The declaration is gone; tools.view stays, because it
really does gate the nav entry.
Removing it needed care: update_role refuses the WHOLE request when a submitted
permission is not grantable, and the grid resubmits what the role already holds,
so a site whose guest role held tools.secrets could no longer save that role at
all - the deadlock fixed in 723a4ff, re-created from the other end. Migration
7d41 revokes the grant from the guest role, and nothing is lost because it
granted nothing.
Eight tests. The first two verified by planting: remove either guard and three
fail.
Scope: product
The machine maintenance panel showed every date one calendar day early on this
site, and would show it a day late east of UTC.
lastmaintenancedate and nextmaintenancedate were DATETIME columns holding
midnight, and BaseModel.to_dict serialises a datetime as isoformat + 'Z', so
2026-12-15 00:00:00 went out as "2026-12-15T00:00:00Z" and the browser
converted it into local time. There is no time in a maintenance date: the form
offers a date picker, the CSV importer takes '2026-06-01', and the value is only
ever compared against other dates. Every sibling already models it correctly -
measuringtools.lastcalibrationdate, warranties.startdate and enddate are all
db.Date - so machines were the odd one out.
Migration 7d40 converts both columns; MySQL truncates a midnight time so no
data moves, and SQLite neither enforces nor needs it.
That exposed two more things. The route was assigning the raw request string to
the column, which a DATE column rejects outright - it now goes through the
strict parser from wave 2, so a mistyped maintenance date is refused with a 400
rather than stored or nulled. And BaseModel.to_dict had no branch for a plain
date, so one reached Flask's encoder and came out as an HTTP-date
("Thu, 15 Jan 2026 00:00:00 GMT") - which a browser parses back into a moment
and can still render as the day before. Dates now serialise as "2026-01-15",
with datetime checked first because it is a subclass of date.
Verified against MySQL on a live instance: the column is DATE, the API serves
2026-01-15, a mistyped date is refused with 400, and a rename on a machine that
has maintenance dates saves - the 500 from the audit-log fix in 5d81f50c, now
confirmed end to end rather than only in tests.
Scope: product
The retire dialog says "History and relationships are kept, and it can be
restored." The API always kept that promise - ?active=false lists retired rows,
PUT {"isactive": true} restores one - and no screen in the product could reach
either. A retired asset simply vanished: the lists filtered it out, global
search missed it, and its detail page still rendered Edit and Retire with
nothing anywhere saying it was retired.
Five list pages gained a "Show retired" toggle that passes active=false, and a
retired row now carries a badge. Five detail pages gained a banner saying the
record is retired and what that means, a Restore button, and the suppression of
Edit and Retire while it is retired - offering Retire on an already-retired
record was the part that made the state invisible.
Verified end to end in a browser: retire a machine, the detail page says so and
offers Restore, the list hides it until the toggle is ticked and badges it when
shown, and Restore puts it back (isactive true again on the API).
One thing this does NOT do: global search still does not reach a retired asset.
That is a separate surface with its own filter, and worth deciding on
deliberately rather than sliding in here - a search that returns retired
records by default would change every result page.
Scope: product
Three more findings from the 2026-09-03 run.
Werkzeug answered 405, 415 and a malformed JSON body itself, in HTML. A client
written against the documented {status,data,meta} contract got a page of markup
with no error code, which is indistinguishable from the server having fallen
over. The 405 had a second edge: on a route the caller cannot otherwise see, a
method-not-allowed CONFIRMS the path exists, so a gated kiosk route meant to
read as absent announced itself by refusing the wrong verb. All three now
answer in the envelope, and only under /api/ - everything else is the SPA and
keeps the framework's behaviour, deep links included.
The Toner Report rendered "0 Printers Checked / 0 Low / 0 Critical" and "No
printers match the selected filter" while the API it had just called said
supply monitoring was unreachable. Zeros read as a clean bill of health for a
fleet nobody has looked at. The page now carries the reason the API gave and
suppresses the counts, because a number nobody measured is not a result. Same
defect shape as the supplies card fixed on 2026-08-29; `unavailable` was
already in the payload and the page ignored it.
Backup revisions badged "current" per (asset, kind). NTLARS settings describe a
MACHINE but are read off the controlling PC, so a machine with two reporting
PCs has two independent chains of the same kind - and only the higher revision
id was badged, so the second PC's newest revision showed as history and lost
its last-verified stamp. The machine looked like one source re-reporting rather
than two. sourcehostname is the chain; a revision without one forms its own,
which keeps the old behaviour for everything that has only ever had one source.
Nine tests. The envelope ones verified by planting: remove the three handlers
and three fail.
Scope: product
The driver form and the API both dropped `drivername` and `vendorid`. POST
returned 201 with both null, PUT accepted them and stored nothing, and no
screen offered either field - so there was no interface in the product that
could set them.
That is not a cosmetic gap. `drivername` is the exact string the INF declares,
and Add-PrinterDriver matches on it; `name` is ours to choose and means nothing
to Windows. With it null the shipped converger has nothing to match and
installs nothing, and the drift check that compares an observed driver name
against the expected one could never fire, because the expected value was never
stored. `vendorid` is how a universal driver (HP UPD, Xerox GPD) attaches to a
whole make instead of needing a near-duplicate row per model.
Both are now read on create and update, and the driver form gained the two
fields with hints saying what they are for - the INF name box says plainly that
leaving it blank means the converger cannot install that driver. A whitespace
value clears rather than stores: an empty box means "not known", and a blank
string would look set and match nothing.
Three tests. Verified by planting: drop the two fields from the create path
again and the round-trip test fails.
Scope: product
Three map and aggregation defects. The first is a behaviour change you asked
for; the other two are the 2026-09-03 run's findings.
A dualpath secondary bay was excluded from /api/assets/map unconditionally
while the collapse setting is on, so positioning it was a silent no-op: the
form took the pin, stored mapx/mapy, and the map never drew it. One marker per
pair is right when the pair sits in one place, and that stays the default - but
a dual-bay machine whose second bay is physically elsewhere is exactly what
positioning it separately means. Now: a secondary with its own position draws
its own marker; one without is represented by the primary's, which carries the
combined '2007 / 2008' label. Nothing extra is needed for the fallback, because
the feed already requires mapx/mapy - an unpositioned bay never reaches the
loop.
The combined label is now suppressed when the partner draws its own marker.
Labelling two separate pins '2007 / 2008' would say one machine is in two
places, which is worse than either bay being unlabelled.
Verified against a live instance: unpositioned secondary gives one marker
labelled with both bays; positioned gives two markers each with its own name;
clearing the position puts it back to one. That third case matters - a
one-way door would be its own bug.
machines/dashboard/summary applied the dualpath exclusion to total and bytype
but not to bystatus, so the status bars summed to more than the headline above
them, by exactly the number of secondary bays. A breakdown that does not add up
to its own total is an afternoon of reconciling.
The measuring-tool calibrationstatus filter ran AFTER pagination. It is a
derived value so it cannot be a SQL filter, but applying it to the current
page meant asking for 'overdue' with one overdue tool in sixteen returned an
empty page 1, a total of 16, eight pages, and the row itself on page 3. When
that filter is in play the matching set is now built and filtered before
paging - the same tradeoff the old comment named, spent on correct answers
instead of a cheaper query.
Seven tests. The pagination one verified against the committed version, which
fails it.
Scope: product
Wave 6: two settings pages that told an administrator the site was configured
when it was not. That is worse than a missing feature, because nobody goes
looking for it.
Audit retention had TWO keys. shopdb/core/services/auditretention.py reads
`audit_retentiondays` (category site, default 0). A second `audit_retention_days`
was seeded under category audit with a default of 90, and the Audit & Logging
page wrote to that one - so an administrator set a period, reloaded, saw it
persisted, and nothing ever consulted it. Retention stayed off. The page now
writes the key that is read, the duplicate is out of the seed list, and
migration 7d39 carries a value an operator actually set on the dead key over to
the live one before deleting it: their intent was "keep 30 days", and the fix
should not silently discard it. The page's copy also claimed logs "will be
automatically purged", which nothing does - purging is manual, from the site
settings page or `flask audit purge --apply`.
The Authentication page offered a full SAML configuration - metadata URL, entity
id, ACS URL, auto-create, admin group - and reported "Configured - SAML login
available" once two fields held text. There is no SAML anywhere in the codebase:
no login route, no ACS, no assertion handling. SAML IS planned, so the settings
stay and the page keeps them; what changes is that it now says plainly that
nothing consumes them yet, and the status line never claims availability.
The dangerous control there was "Allow Local Login". Turning it off did nothing
at all, so an administrator could believe they had closed local password login
while it stayed wide open. It is locked on until there is another way in, and
says why. CONFIG.md marks the whole block reserved rather than describing it as
working configuration.
Scope: product
Wave 5, the asset forms. Both defects are the same mistake in different
clothes: a form deciding something it should have been told.
Four forms carried a hardcoded asset-type id - const COMPUTER_ASSETTYPEID = 2,
and the same for machines, printers and network devices - to tell
CustomFieldsInputs which type's fields to render. Those ids come from seed
order. On the instance they were written against, computer was 2; on a freshly
seeded database computer is 1 and 2 is network_device. So the PC form asked for
another type's custom fields, rendered them, and dropped everything typed into
them behind a success toast, because the values were keyed to a type the record
does not have. Verified in a browser against a correctly built dist: the form
now requests assettypeid=1 on an instance where computer is 1.
The name is the stable thing - assettype is a fixed vocabulary and the API
already publishes the mapping - so composables/assetTypeIds.js resolves it once
per page load. A failed load leaves it undefined, which renders no custom-field
panel: a missing panel is honest, a guessed id is what this replaces.
Three of the five forms that emit a map position sent mapx and mapy without
levelid. ADR-017 is explicit that a position without its level cannot be drawn
on the right floor plan, so a picked pin saved a null level and rendered
nowhere - while the form displayed "Position: x, y" back, because it kept the
value it had never sent.
The ADR-017 gate should have caught that and did not: it globbed '*.py' only,
so it policed what the server emits and never what the browser sends. It now
covers .vue and .js too, and accepts both quoting styles. Verified by planting:
drop levelid from PCForm again and the gate names that exact line.
Five composable tests, including a fixture with a DIFFERENT seed order - the
case the constants got wrong.
Still open from this area, and deliberately not in this commit: the PC form
writes an INHERITED location back as the record's own. Fixing it needs the API
to distinguish an owned locationid from a derived one, which is a contract
change rather than a form fix.
Scope: product
Three confirmed findings from the 2026-09-03 run, each independent.
GE-Enforce scope names are directory names on the imaging share, and create_scope
checked only that the name was non-empty. "../escape" wrote a manifest one level
above the share root, "../../escape2" two, and "/tmp/anywhere" wherever it liked
- proven by writing the files and then deleting them. The exposure is the
permission split rather than the traversal itself: geenforce.manage creates the
scope and geenforce.publish writes the file, so a manage-only author could aim a
publisher's write at any path that process can reach. Names are now a single
safe path segment, checked at creation AND again at the write (rows predating the
check exist, and the write is the line that acts), with the resolved target
confirmed to sit under the share root.
Agent timestamps were converted with astimezone(tz=None), which means the
SERVER's local zone, not UTC. An agent reporting 17:26:16Z was stored as
13:26:16 on this EDT box and re-served labelled 'Z', while the same row's
server-written lastreportedat stayed true UTC - one row carrying two time bases
four hours apart. The dashboard's 48-hour stability window compares a real UTC
cutoff against these values, so events left it four hours early. Invisible on a
UTC server, which is how it survived, and nothing in the Dockerfile, the compose
files or the Windows deploy pins TZ. AGENT-CONTRACT.md mandates UTC on the wire,
so a naive value is now taken as UTC rather than guessed at.
The three /api/maplevels reads went back to optional-auth. ADR-017 rule 6, the
API inventory (auth "none"), the module's own docstring and the comment in
PrinterInstallerMap.vue all say they are public, because the printer installer
map runs before anyone logs in and needs a blueprint to draw on. afce2437 swept
165 genuinely-open GET routes behind require_permission and caught these in the
net, so the anonymous installer map has drawn no floor plan since. Restored
rather than amending the ADR: blueprint paths and level names are not secrets,
and the asset positions that go on top already render on public kiosk pages.
The writes stay gated, and a test proves it.
Registered in PUBLIC_ENDPOINTS with the reason. That gate checks a public route
is DECLARED public; the new test checks it actually answers, which is the half
that was broken.
Ten tests. All three verified by planting: restore the non-empty scope check and
eight fail, restore the local-zone conversion and two fail, put the permission
back on the level list and one fails.
Scope: product
Wave 4, second half. PUT /api/machines/<id> returned 500 and rolled the whole
edit back for any machine with a maintenance date recorded - including a rename
that never touched the date. The Machine edit form could not save such a
machine at all, which is most of them once a maintenance schedule exists.
The route builds {'old': getattr(row, field)} straight off the model, so the
changes dict carries whatever the column holds: a date, a datetime, a Decimal.
AuditLog.changes serialised it with a bare json.dumps, which raises on all
three, and nothing caught it. The audit entry and the edit share a transaction,
so the failure took the edit with it.
json.dumps now has a fallback encoder: dates and datetimes become ISO strings,
Decimals become floats, sets and bytes get something readable, and anything
else becomes its repr rather than an exception. An audit entry that reads a
little awkwardly is worth more than a failed write - the alternative, and what
happened, is losing the change AND the record of it.
Fixed at the audit log rather than in the machines route: every audited route
builds its changes dict the same way, so the next one to audit a date field
would have hit the same wall.
Four tests, including the exact shape the machines route produces. Verified by
planting the bare dumps back: two fail.
Scope: product
Wave 4, first half. A personal API token scoped to endpoints.command alone
could queue system_restart, run_script and update_app against a shop-floor PC.
require_permission has always suspended the admin bypass for a scoped token,
and tests pin that. What nothing covered was a route checking a SECOND
permission by hand: the endpoints command route asked
current_user.haspermission(elevated), which reads the token's OWNER. An admin
minting a narrow token for a script therefore handed out remote restart, script
execution and app deployment - the exact bypass require_permission exists to
prevent, reintroduced one hand-rolled check at a time.
caller_holds is now scope-aware, so the answer is about the caller rather than
the person behind it: the owner must hold the permission AND, when the request
carries a scoped token, the scope list must name it. An unscoped token is the
person and is unchanged; a session is unchanged; guests still answer from the
guest role.
Both hand-rolled checks in the tree now use it - the endpoints elevated gate and
the secure-send tools.secrets check, which had the same shape and the same
consequence at a smaller scale.
Two tests, one proving the refusal and one proving an unscoped token still
passes (a fix that quietly broke every service token would be worse than the
bug). Verified by planting the owner-based check back: the first fails.
Reported by the fleet-agent lane of the 2026-09-03 test run and confirmed by an
independent verifier that minted its own token and queued its own commands.
Scope: product
Wave 3. Six findings, one cause: `int(request.args.get('typeid'))` with nothing
catching the ValueError. The caller sent a bad argument and got
500 INTERNAL_ERROR plus a traceback in the log, which says the server is broken
when the truth is that 'abc' is not a number.
shopdb/utils/queryargs.intarg reads it and raises QueryArgError, which an
app-level handler answers as a 400 naming the argument. App-level on purpose: a
route should not have to remember, and forgetting was the entire defect. The
policy differs from get_pagination_params, which falls back to a default for a
bad page number - a nonsense page is harmless and the caller still wants their
list, but silently ignoring a nonsense FILTER returns rows they believe they
excluded.
The gate matters more than the six fixes. tests/test_core/test_bad_query_args.py
sweeps all 131 registered no-parameter GET routes under /api/ with twelve
argument shapes and fails on any 5xx. It found sites the test run had not
reported - statusid and locationid on /api/assets, the second asset-list route,
limit on the employee lookup and the printer supply history - because the
pattern is how a list filter gets written rather than a property of any one
route. Verified non-vacuous by planting the old `int(...)` back into
/api/machines: four tests fail.
Four usb routes are skipped by exact path, with the reason: they reach the
EXTERNAL cmmc_usb database, so their 500 in a test environment is a missing
database rather than an argument fault. That is the complete set - every usb
GET route calling cmmc_usb_connection - not a convenient subset.
One correction worth recording: the first pass rewrote every
`if x := request.args.get(...)` in the touched files, including text arguments,
so a search for 'ZZNOTHINGMATCHES' briefly became a 400. Caught by
test_list_search_identifiers. Only id-shaped arguments are integers.
Scope: product
Wave 2 of the remediation plan. Eight findings from the feature test run were
the same defect twice: a write that reports success while storing something
other than what was sent.
The first half is a date parser that swallowed its own error. Two plugins
carried an identical private _parse_date returning None on ValueError, so an
unparseable date and a deliberately cleared one produced the same write.
PUT /api/warranty/<id> with enddate '30-06-2027' answered 200 "Warranty
updated" and stored NULL, dropping the record out of every expiry surface on
the strength of a typo; the measuring-tool calibration date did the same and
flipped the tool to 'unknown', out of the calibration report and the map
overlay. shopdb/utils/datevalues.py now holds one strict parser on the contract
surface: an explicitly empty value clears the field, anything that is not a
date is refused with a 400 naming the field, and those are different intents
with different answers. The vendor-lookup paths keep a lenient wrapper on
purpose - a provider changing its date shape must not fail the refresh of an
otherwise good record, and no person typed that value.
The second half is why the truncation findings were possible at all. MySQL was
running non-strict: sql_mode on the 5.6 server is NO_ENGINE_SUBSTITUTION with
no STRICT_TRANS_TABLES, so it accepted writes it could not store and adjusted
them. A 257-character username became its first 100 characters (and the next
distinct one then collided with it); an over-long asset number was cut at 50; a
malformed releasedate became '0000-00-00' and permanently broke every read of
that application. All with a 201. The mode is now set on the CONNECTION rather
than left to the server, so a site whose my.cnf disagrees still gets it.
That trades silent corruption for a DataError, which uncaught is a 500 - and a
500 says the server is broken when the truth is that the value does not fit. An
error handler maps it to a 400 naming the column where MySQL names one.
Verified by hand against MySQL 5.6: POST /api/users with a 257-character
username now answers 400 "That value is too long for username." and stores
nothing, where it previously stored a truncated row and answered 201.
Twelve tests. The date guard is verified by planting: restore the swallowing
parser and four of them fail. The DataError tests raise the error directly
because the suite runs on SQLite, which does not enforce VARCHAR length at all -
there is no value that would trigger it there, which is precisely why this went
unnoticed.
Scope: product
Wave 1 of the remediation plan for the 2026-09-03 feature test run. Two kiosks,
one shape of defect, one critical consequence.
Both kiosks read the state, decided in Python, and then wrote. Nothing stops two
requests interleaving between the read and the write, so both see the same
pre-state and both proceed. Reproduced on MySQL 5.6 over HTTP, five concurrent
takes of 5 from a bin holding 5: all five were accepted and quantityonhand
landed on -20, because each request wrote its own arithmetic. The ledger had
recorded every one of them, so the cached quantity and the ledger disagreed
permanently, which is the number the shop floor actually reads. The USB kiosk
had the same shape: concurrent scans of one drive all succeeded and inserted
several open custody rows, so one drive was recorded out to two people and the
documented 409 never fired.
The comparison now lives in the WHERE clause, where the database settles it at
write time. printedparts._move_quantity issues
UPDATE ... WHERE quantityonhand >= :n and treats rowcount 0 as the refusal,
raising InsufficientStock, which take and adjust turn back into the message they
already gave - carrying the quantity the database holds now, not the stale one
the request read. The low-stock alert derives its before-value from what the
write produced for the same reason. Both USB paths claim the device with
ischeckedout (self-hosted) or status (external cmmc_usb) in the WHERE clause;
the external one claims BEFORE writing the log line, so a refused claim leaves
nothing to undo. Check-in is guarded the same way.
After the fix, the same five concurrent takes: one accepted, four refused,
quantityonhand 0 and the ledger sum 0.
Check-in also closes EVERY open row for the device rather than the newest.
One device can only be out once, so a second open row is corruption, and
closing only the newest left the older ones open forever - the device then read
as returned while its custody history said otherwise.
Separately, usbcheckouts now records who handed a drive back
(checkinsso/checkinname, migration usb0004checkinidentity). Returning for a
colleague is a supported flow that the kiosk asks to confirm, but the row it
wrote attributed the check-in to the HOLDER, so the custody history and the
audit report both named somebody who was not there. Nullable and deliberately
not backfilled: copying the holder's badge into old rows would manufacture
evidence for the exact claim this stops making.
Four tests. Both guards verified by planting: strip the condition out of either
UPDATE and its test fails.
Scope: product
Five findings from the review of the WINC import, plus the rendering each fix
needed to be usable.
An undo that hit a conflict wrote status 'undone' anyway, and undo_batch returns
immediately for an undone batch. So the one entry that needed a retry was the
only one that could never get it: the human resolves the conflict, clicks Undo,
and gets a no-op. A conflicted undo is now 'partially-undone', undoneat is
stamped only when the batch is genuinely finished, and the batches table offers
"Retry undo" for that status. The column was String(16) and the new value is
exactly 16 characters, so migration 7d38 widens it to 32 rather than rely on an
exact fit.
_UNDO_REFERENCES listed only assets, so a business unit this batch created was
hard-deleted even when something else still pointed at it. Two of the three
missing referrers are real FKs (dashboarddefaults.businessunitid,
businessunits.parentid): MySQL raises and the whole undo rolls back, so the
batch cannot be undone at all. The third, notifications.businessunitid, has no
FK, so the delete succeeded and left a notification pointing at an id that no
longer exists - silently, on both engines. All four are listed and the guard is
tested through the route.
The reference guard also queried tables that need not exist. A lean site drops
the tables of plugins it does not install (ADR-014), and notifications is
plugin-owned, so on such a site the query raised 1146, rolled the undo back and
left the batch permanently un-undoable. Absent tables are skipped; a table that
does not exist cannot hold a reference.
_PACKET_GROUPS reports on location, but apply_plan only ever creates business
units - it sets locationid only when the location already resolved. A plan
needing a new location asked the admin to acknowledge it, got the tick, wrote
nothing, and reported nothing. build_packet now takes the groups the writer can
actually create; anything else needed is 'uncreatable' and blocks at plan time,
where the admin can act on it, rather than dying at commit on a message about
acknowledging something the screen had no control for. The writer refuses the
same case independently. Both the console packet and the import page name the
blocking groups: setting blocking without printing the reason would leave the
banner pointing at nothing.
The web undo route caught ValueError only, so refusing a batch belonging to
another importer surfaced as a 500. That refusal is new too: the journal is a
general primitive and undo_batch does not filter on kind, so any other
importer's batches were reversible through this endpoint, carrying WINC's
references map - which lists none of that importer's tables, making every
reference check pass and every delete unconditional.
Nine tests. The two guards were verified by planting violations: dropping
dashboarddefaults from the references map, and passing creatable=None, each fail
the test written for them.
Scope: product
Six fixes that were ours regardless of what the sister site does. Four came out
of our own reviews; two you reported from the settings pages.
MAX_CONTENT_LENGTH was set nowhere. Every upload route caps its own payload, but
each of those checks runs after the body has arrived: the first touch of
request.form or request.files makes Werkzeug parse the whole multipart and spool
the file to a temp file, so a route can reject the content and still have paid
for the transfer and the disk. Three separate reviews found that shape in three
different routes, which makes it a property of the app rather than of any
handler. 128 MB, clearing the 64 MB agent binary with room for the envelope.
The naming gate excluded devnotes with --exclude-dir, which matches a directory
of that NAME at any depth - so a force-added plugins/<plugin>/devnotes/ was
invisible to every check in the file, ADR-015 included. Replaced with a path
anchored post-filter. Verified by planting the same site literal in root
devnotes/ AND in plugins/endpoints/devnotes/: the gate now reports exactly one,
the nested one. Before, it reported none.
BaseModel.to_dict walked __table__.columns and published every one, so a model
without an override published whatever it carried - including columns added
later by someone not thinking about this method. User has no override, so
User.to_dict() returned its bcrypt hash. Nothing calls it in a response path
today, which is luck rather than design. A blanket allowlist would empty the
output of all 34 models relying on the default, so this pins a floor instead:
named secret-bearing columns are never emitted, and a model may extend the set.
Eight tests, including a sweep over every mapped model.
ActionPicker's Esc handlers had .prevent but not .stop, so the event bubbled to
the wrapper and closed the whole picker instead of returning to the monitor list.
The branding page reported "error loading image" on the site and badge logos.
Two bugs stacked. The load failure was real: the preview used the stored value
as a raw src, and that value is root-relative, so under /shopdb or /ops it
resolved against the SERVER root and 404d - siteSettings.js wraps every logo in
withBase() and this page was the one place that did not. With that fixed two of
three were still invisible, because the shipped GE mark is fill="white" for the
dark sidebar and was rendering white-on-white. The preview tile now sits on a
mid-tone checker so artwork of any colour reads against it.
The floor-map levels table ran off the right with its cells crushed - nine
columns of inputs, no scroll wrapper, and .data-table is styled nowhere. My
first attempt reached for the global .table-container and made it WORSE: that
class also sets white-space: nowrap, and a full sentence of hint text lives in
one of the cells, so nowrap forced the table wider still. It now scrolls in a
local wrapper, the hint wraps, and the name input has a width instead of
collapsing to one character.
Suite 2377, frontend 324, gate green.
Scope: product
One commit from the sister site, purely additive: a read-only `batches` command
so an operator working from the shell can find a batchid to undo, and a README
covering the four WINC commands. Two new files under scripts/site_imports/winc/,
nothing existing touched.
Verified rather than assumed: it changes none of the files fixed in 13be0ef9
(the workflow YAML, the approval check, the agentpackage description, the upload
audit), the gate passes, all five workflows still parse, and the project map is
current.
Suite 2369.
Scope: product
Eleven commits from the sister site: a reversible Maximo/WINC machine-adoption
import, a versioned agent binary store with upload and rollback, remote-BuildKit
CI, tray version reporting, and UI fixes. Applied as a patch again - the
published mirror rewrites hashes, so the branches have unrelated histories and
git refuses to merge them.
Reviewed by three agents before merging. Four things fixed here.
THE PUBLISH WORKFLOW WAS DEAD. The new remote-BuildKit step in
publish-images.yml sits at column 0 instead of indented under steps:, so the
whole file is invalid YAML and GitHub would refuse it. Their entire GHCR
distribution channel - which Start-ShopDBDev.ps1 and the compose files consume -
was broken at the source and clearly never run. Indented; all five workflows now
parse.
THE IMPORT'S APPROVAL CHECK COULD NOT FAIL. commit() built the plan and the
approval packet from the same upload and then asked the writer whether they
matched, so packet_matches_plan compared a hash against itself. The documented
contract - "a human approves a specific, frozen plan, and the writer can prove
nothing drifted" - did not exist in the web path: an admin could preview file A
and commit file B, gated only by the needs-create group NAMES they ticked. The
preview's planhash is now required at commit and checked against a freshly
rebuilt plan. Their tests committed without previewing, which is what let this
through; they go through the real flow now, and three new tests pin it - all
confirmed to fail with the check stubbed out.
endpoints.agentpackage QUIETLY GREW. It used to mean "wrap the staged exe in a
ZIP"; with the upload routes it also means "choose the executable every future
package installs and runs as SYSTEM". Before this, staging needed filesystem
access to the server. The permission description still said only "generate and
download a package", so an admin granting it could not see what they were
granting. Reworded.
NOTHING RECORDED WHAT WAS STAGED, OR BY WHOM. The manifest held version, size
and timestamp; re-uploading a version replaces the bytes in place under the same
label. A sha256 of the stored file is now recorded, and the upload is
audit-logged with it - after an incident there was previously no answer to who
put that binary on the fleet.
Also documented the nine new routes in docs/api-inventory.json (that gate cannot
run on the mirror, so it falls to us at promotion time again), regenerated the
spec and the project map, and kept our own docs where they conflicted: docs/ is
scrubbed from the mirror, so their DEV-BUILD-TEST.md still had the dev-be
channel mapping we corrected, and their PLUGIN-HOOKS.md was a new 45-line file
rather than our 737-line canonical one. Their journal section is appended to
ours.
Contract version comes across as 0.30.0 for a genuinely additive surface.
Suite 2369, frontend 324, gate green.
Scope: product
The USB kiosk had a feature flag and nothing else. A flag is a decision, not a
credential: with the kiosk on, anyone who could reach the app could move a drive
between checked-out and checked-in and sign it with a badge they guessed. Same
reasoning that closed the parts kiosk, smaller blast radius - which is why
Darkwing did not raise it and why it was worth asking about anyway.
Both kiosks now share one mechanism. The device token moved out of the
printedparts api wrapper into a request interceptor that attaches X-Kiosk-Token
by URL prefix, with one localStorage key per kiosk - they are separate screens
in separate rooms, and a shared token would mean provisioning one grants the
other. claimKioskToken() is shared too, so both views take the token from a
?token= URL once and strip it from the address bar identically.
Then the part that matters more than either fix. I told you to set the new
settings in the UI, and that was impossible: every plugin settings page carries
a hardcoded key list, and nothing had added them. printedparts_kiosk_enabled,
printedparts_kiosk_token and usb_kiosk_token were declared, seeded and read
correctly, and rendered by no page - changeable only by writing SQL.
tests/test_plugin_settings_reachable.py now fails when any declared setting
appears nowhere in its plugin's frontend. It found seven MORE that were already
in that state, all pre-existing:
collector_corporate_ranges computers_configpolicies_enabled
computers_machinelink_alerts computers_quietreporthours
geenforce_reportstaleminutes printers_dashboardpercent
warranty_expiringdays
All seven now have fields. I first parked them in a KNOWN_GAPS ratchet, then
fixed them and deleted the ratchet - a backlog list in a new check is a way of
agreeing the check does not apply yet. There is no backlog; the check is
absolute.
Two of my own mistakes on the way, both caught by tests rather than by reading:
the blanket rename of the USB guard also rewrote the call inside _kiosk_open, so
it called itself until the stack ran out; and the first version of the reachable
check skipped all 17 plugins because it looked for get_settings() when the
method is get_settings_defaults() - it passed by testing nothing, which is the
exact shape it exists to catch.
Suite 2336, frontend 316, build clean.
Scope: product
printedparts_kiosk_token declared valuetype 'secret'. The column documents
string, boolean, integer and json, and that was the only 'secret' in the
codebase - it worked only because Setting.get falls through to the raw string
when it recognises nothing.
Masking never depended on it: settings.py::_is_secret keys on the NAME
containing 'token', so the value is still hidden in API reads and in the audit
log. Changed to string, which is what it is.
Scope: product
Both from the second Darkwing scan. Every claim in it was verified against the
code before acting.
The kiosk was the product's only unauthenticated write, justified by a decision
record saying it was "physically rate-limited: it serves a touch screen on the
shop floor". Nothing in code enforced the physical half, and an HTTP route is
reachable from wherever the app is. What that left open:
- _kiosk_find_item falls back to matching bare digits against the numeric
tail of every active item, so 1, 2, 3 walk the catalogue without knowing a
barcode;
- a badge is a short numeric SSO, and the route answered 422 for an unknown
one and 200 for a real one - an oracle for sorting guesses into employees;
- the write then decremented stock and stamped the ledger, which IS the audit
trail, with whatever SSO the caller supplied;
- quantity was bounded only by what was on hand, so one request emptied a bin.
A badge is attribution, never authorization. Both kiosk routes now need the
site's kiosk flag AND a per-device token, and 404 rather than 403 without them
so a caller does not learn the door is there. The READ is gated too: gating only
the write would have left the enumeration oracle running against a kiosk a site
had switched off.
The token is per-DEVICE, not per-page, because a kiosk is a browser on a touch
screen and a token served inside the page is readable by anyone who can load it.
An admin opens the screen once as /parts-kiosk?token=..., the page moves it into
that browser's localStorage and strips it from the address bar.
Also: quantity is capped per take, JSON true no longer passes as 1
(isinstance(True, int) is True), and every badge failure on the anonymous route
returns one message. The gated restock and adjust handlers keep the specific
message - there the person is known and the detail is help, not an oracle. I
changed the wrong one of those three handlers first; caught by the test.
/api/dashboard/health formatted str(e) straight into the body of a route
PUBLIC_ENDPOINTS declares open because "monitoring has no account". A pymysql
OperationalError names the database account, the app host's address, the driver
and the failing statement. It is logged server-side now and the body carries a
coarse status. The version goes with it - it told an anonymous caller which
release to look up. Status stays 200 either way: a monitor keyed on the body's
status field is the contract here, and turning a degraded database into a
transport failure is a separate decision from not leaking the exception.
Nine tests. The three gate tests were confirmed to fail with the gate stubbed
out, rather than assumed to cover it. Suite 2325, frontend 316, build clean.
Scope: product
The previous commit said the link-resolving change had never met a real
filesystem. Run in the win11 VM against real fixtures, it turned out to reject
EVERY legitimate launch path.
ResolveDirectoryChain walked up looking for a junction and eventually asked
whether "C:\" was a link. That throws DirectoryNotFoundException, and the caller
treats a resolve failure as a rejection, so the first legitimate case came back
"cannot resolve the real path: Could not find a part of the path 'C:\'". The
agent would have failed closed on everything - an outage wearing a security
fix's clothes. It compiled with zero warnings and read correctly.
Stop at the volume root, and never let "cannot tell whether this is a link" turn
into a rejection: TryResolveLink returns null on any failure, and the assembled
path is re-checked against the roots regardless, so an unresolvable component
still cannot widen what is accepted.
One expectation in the probe was mine being wrong rather than the code. 8.3
short names used to be rejected because the lexical prefix check could not match
C:\PROGRA~1 against C:\Program Files - fail closed, but by accident of not
understanding the path rather than by intent. Resolution now expands it to the
real directory, which is inside the approved root, so accepting it is correct.
Verified on Windows, all ten cases:
accept a real exe under the approved root
accept 8.3 short name resolving into the approved root
reject exe reached through a junction pointing out of the root
reject alternate data stream on a file in the root
reject traversal back out of the root
reject UNC path
reject sibling directory sharing the root's prefix
reject wrong extension
reject file that does not exist
reject empty path
The probe is committed as agent/ShopDbEndpointAgent.PolicyProbe so this is
repeatable. It is NOT in CI and its README says why: junctions, alternate data
streams and 8.3 names do not exist on Linux, and a suite that skipped every case
it exists for would report green and mean nothing. This repo has been bitten by
that shape more than once already.
Scope: product
check to the file it actually runs
Two clusters the five-agent review found. Both are about a product that another
plant has to be able to install.
ADR-015: the shipped monitor bundle was the originating plant's fleet
convention frozen into product seed.
The business policy was seeded as "RPCA Shopfloor App Monitors". RPCA is that
plant's facility designator, the row is source='shipped', and the API refused
to rename it - so every adopting site would carry another plant's label at the
top of its policy list forever. 0007 now seeds the neutral name, and 0010
renames it on installs that already stamped 0007, guarded so it cannot collide
with a policy a site made itself. Downgrade is one-way on purpose: putting
another site's designator back is not a state worth reaching.
Which PC type the bundle attaches to was the literal 'Shopfloor PC'. A plant
whose type is "Shop Floor Workstation" silently got no shipped monitors at
all, and one using that name for something else got all of them. It is a
setting now, defaulting to the seeded name so nothing changes here, and a
blank value attaches the bundle to nothing - the right neutral answer for a
site that does not want it.
Shipped rows refused every edit, including isactive and launchpath. A plant
whose eDNC is on D:, or that runs no UDC, could not correct the path or switch
the monitor off, and then carried permanent false drift for it. ADR-015 allows
seed data BECAUSE a site can edit it afterwards. Paths and the active flag are
now editable; kind and displayname stay fixed, because that pair is the
identity 0007's bundle matches on.
Also corrected two false claims in 0007's docstring: the policy tables belong
to the computers plugin, not the core baseline, and a guarded no-op does NOT
reconcile later - Alembic stamps the revision either way, so the bundle would
simply never seed.
LaunchPathPolicy was purely lexical. GetFullPath is string normalization that
never touches the disk, so the root check constrained the NAME while File.Exists
and CreateProcess followed reparse points to somewhere else. A junction under a
writable directory in an approved root - creating one needs no privilege, only
write access to the parent, and vendor trees under Program Files are not
reliably locked down - turned WebUI monitor-edit rights into code execution as
the console operator. It now resolves the real target, leaf and directory chain
both, and re-checks THAT against the roots, rejecting if resolution fails.
Alternate data streams passed the whole policy on net8.0: an interior colon no
longer throws in GetFullPath, so "readme.txt:payload.exe" reported an .exe
extension, passed the root check, and existed. Rejected explicitly now.
Six tests added for the shipped-row and pctype behaviour. The Core project
compiles clean. The launch-path change has NOT been exercised against real
junctions, ADS or 8.3 names - that needs the win11 VM, and per the reviewer it
wants a planted-violation suite there rather than my say-so.
Still open from the review: monitor revocation does not take effect until agent
restart, ActionPicker is keyboard-inoperable, the lightbox has no focus trap,
one attached monitor grants generic any-target process control, and screenshots
are retained unencrypted in the user profile.
Scope: product
Findings the reviewers verified, fixed here. The larger cluster they found is
NOT fixed and is listed at the end.
Frontend. reportImageUrl is called from the template, so it ran on every render
with no in-flight guard: a report whose fetch was pending was re-requested, and
because each resolution reassigns the map the whole grid re-rendered and
re-fired the rest, costing up to N-squared requests for N thumbnails. Two blobs
resolving for one key orphaned the first object URL where revoke could not
reach it, and a failure was never recorded so a broken image retried forever.
Guarded, deduplicated, failures remembered.
Three CSS variables did not exist, so their hardcoded fallbacks always won:
--bg-alt (the "No screenshot" tile, near-black in light mode), and
--warning-text/-bg/-border (dark khaki on a dark modal). Replaced with real
tokens. The DEV badge's literal #c62828 is var(--danger).
Installer and launcher. The generated site installer ran docker compose with no
-p, so the project name defaulted to the package folder - which carries the
version. Installing the next version created a new project with new empty
volumes and the site came up looking wiped, old data stranded under the old
project. Pinned to shopdb-<channel>.
Start-ShopDBDev deleted any existing -TargetDir that lacked a .git, recursively
and without asking. For a script handed round as "save it anywhere and run it",
a mistyped -TargetDir pointed at Documents would have destroyed it. It now only
reclaims a directory that is empty or carries this launcher's own marker, and
refuses otherwise.
Dev secrets came from Get-Random, which is System.Random: a 32-bit seed, and all
four values off one stream, so recovering one recovers the rest. They sign JWTs
for a stack a pilot may point at real imported data. CSPRNG now, and the API
port binds to loopback like the DB rather than every interface.
Two of my own from earlier in this session. The operator-report size check sat
below request.form, and the first access to that parses and spools the whole
multipart - so it ran after the thing it prevents. Moved above. And the channel
remap missed build-channel-installer.ps1 entirely, where it set BOTH channels to
main: a "stable" package built from the trunk, the exact mislabel the remap was
for. stable now resolves the newest v* tag.
Also removed a zero-byte docs/ShopDB-Planner-Tasks-v2.csv that rode in.
NOT FIXED - needs a decision, and it is the biggest thing the review found.
Migrations 0006/0007 freeze the originating site's fleet convention into
product seed: a shipped policy literally named "RPCA Shopfloor App Monitors",
a hardcoded 'Shopfloor PC' type deciding who gets the shipped watch-list, and
shipped rows the API refuses to edit or deactivate - including their launch
paths. A site whose eDNC is on another drive cannot correct it and gets
permanent false drift. Those are stamped migrations, so this gets harder to
undo at every install. See the review notes before anyone upgrades a second
site through 0006-0007.
Suite 2314 passing, frontend 316, all three PowerShell files parse.
Scope: product
Six commits from wilm-import: the squashed import, route documentation, the
channel remap onto our tag-and-trunk model, the project map, parallel test
support, and the four smaller review findings.
Merged rather than cherry-picked: the published mirror rewrites hashes, so the
commits do not apply one at a time against our history even though the squashed
diff applies clean.
Scope: product
# Conflicts:
# docs/PROJECT-MAP.md
None blocked the merge; all four were mechanisms that did not mechanise.
_RUNNING_STATES existed twice, in api/routes.py and services/reconciler.py, with
a comment asking a human to keep them in lockstep. The panel and the reconciler
read the same agent snapshot, so a drift between the copies would paint a
service green and flag it as drift on the same page. One definition now, owned
by the reconciler and imported by routes - api -> services is the direction the
plugin layout already runs in, and the reconciler's import of routes is deferred,
so there is no cycle.
The operator-report upload checked its 10 MB cap AFTER upload.read() had already
pulled the whole body into memory, so the check could not prevent the thing it
was there to prevent. content_length is now an early out and the read itself is
bounded to cap+1, which is enough to know the cap was exceeded without taking a
body that lied about its length. The multipart envelope carries the other form
fields too, so the early out allows 64 KB of slack rather than rejecting an
image that is just under.
The same handler forced the extension to .png and the mimetype to image/png
whatever the bytes were, then served the file back under that type. The upload
headers (CSP sandbox + nosniff) make the document case inert, so this was never
the hole it looks like, but nothing is gained by storing a file we already know
is not what it claims. The PNG signature is now checked.
The lightbox bound Escape and the arrow keys to a div and never focused it, so
every one of those bindings was dead until the user happened to click the
backdrop. It takes focus on open and hands it back to the thumbnail on close.
On the two upload tests, honestly: the PNG one was confirmed to fail with the
check removed. The oversize one passes either way, because the old code did
reject an oversized image - it just did so after reading all of it. That test
pins the response, not the fix. Whether the read is bounded is not observable
through the response, so nothing here proves it; the change stands on reading
the handler.
Suite 2311 passing, frontend 316 passing, build clean.
Scope: product
The suite had grown to ~2300 tests and 5m30s. Profiling says there is no hot
spot to fix: the 25 slowest tests sum to about 60s of 331s, and the rest is
per-test fixture cost. The db fixture drops and recreates all 94 tables for
every test, measured at 0.058s a cycle, which is roughly 40% of the wall clock
on its own.
That shape parallelises almost linearly, so pytest-xdist buys more than any
rewrite would:
pytest -q 2309 passed in 331s
pytest -q -n auto 2309 passed in 103s (8 cores)
Deliberately NOT in pytest.ini addopts. Worker startup is dead weight on the
targeted single-file runs that make up most of a working day; the whole-suite
run before a push is where it pays. Opt in with -n auto.
A run distributed across 8 workers also has to be order-independent to pass at
all, so this is incidental evidence that the suite has no inter-test coupling.
Recompiling requirements-dev.txt to add xdist also moved idna, packaging and
pyjwt. That is the -c requirements.txt constraint doing its job rather than a
drift: all three ARE shipped, and the dev file had fallen BEHIND production
(idna 3.13 in dev against 3.19 in prod). CI was testing against older versions
than sites install, which is the exact failure the header comment on that file
was written about. Now matched.
Scope: product
The channel work assumed the branch model it was written against: upstream,
main is a release branch and dev-be is the trunk. Neither holds here. main IS
our trunk - 212 commits past v0.11.3 - and a release is a tag (ADR-007). dev-be
does not exist at all, so the edge half would never have fired and the launchers
would have cloned a missing branch.
Taken literally it would have published our unreleased trunk to every site as
:stable, which is the opposite of what the word promises to someone who has no
way to check.
Remapped throughout:
tag v* -> :stable the thing that was actually released
main -> :edge the trunk, DEV badge lit
publish-images.yml keys the channel on github.ref_type = tag rather than on a
branch name, so the mapping cannot silently drift back. Start-ShopDBDev.ps1,
dev-bootstrap.ps1 and build-channel-installer.ps1 resolve stable to the newest
v* tag via git ls-remote --sort=-v:refname, warning and falling back to main if
a repo has no tags yet. DEV-BUILD-TEST.md follows.
The push: trigger is commented out. Automatic publishing waits on someone
confirming the GHCR package exists under this owner and deciding who may pull
it; workflow_dispatch works meanwhile. Turning it on is one uncomment.
All three scripts parse clean under pwsh, and the tag-selection expression was
run against this repo rather than eyeballed. Per the standing rule, the runtime
paths still want a pass in the win11 VM before anyone leans on them.
Scope: product
Darkwing's reconnaissance flagged POST /api/printedparts/kiosk/take as an
unauthenticated write reaching a ledger commit and an outbound mail/webhook
trigger. Analysis dropped it, correctly: it is a declared exemption with a
decision record. Nothing to fix in the code.
The comment beside it was wrong, though. It described the USB kiosk writes as
open at the "same bar as the parts kiosk" while listing a gate the parts kiosk
does not have: usb_kiosk_enabled, off by default, every route a 404 until a site
sets it. grep -rn kiosk_enabled plugins/printedparts/ returns nothing.
So a site that adopts the product and has no parts kiosk still serves a live
unauthenticated inventory-decrement endpoint with no switch to turn it off, and
the registry that is supposed to be the record of these decisions said the
opposite. Whether printedparts should gain the same flag is a behaviour change
for a kiosk that is in daily use here, so it is left as a decision rather than
made silently; this only stops the comment misleading the next reader.
Scope: product
Found by the Darkwing scan of main (run 33654922078). Its own review dismissed
the finding as not remotely exploitable and it is right about that: nothing can
make a victim's browser attach a custom header, and deploy/windows/web.config
does no output caching. It still recommended the hardening, and the reasoning
for the dismissal is a property of OUR IIS config, not of the product - every
site runs its own instance behind its own proxy under ADR-004.
_batch_base_url() interpolated a caller-supplied host into the single-quoted
PowerShell -Uri literal of the .bat this route hands out. site_base_url ships
blank, so that fallback is the default path on a fresh install, not an edge
case, and GET /api/printers/install-batch is public.
Three rules now:
- the host is validated as a hostname before it is interpolated, so the quote
that would end the literal never reaches it,
- X-Forwarded-Host is only believed when TRUSTED_PROXY_COUNT declares a proxy
-- the rule shopdb/utils/clientip.py already applies to X-Forwarded-For.
That was centralised for this exact reason and did not cover the Host
variant,
- a request supplying neither usable header gets a relative base, which still
downloads correctly from the browser that fetched the script.
Three tests pin it, and all three were confirmed to fail with the old body
restored rather than assumed to cover it.
Scope: product
Two of the three things holding the wilm import out of main.
The monitor CRUD, the two resolver reads and the three operator-report routes
had no entry in docs/api-inventory.json, so test_api_inventory_parity failed.
The gate cannot run on the published mirror - the test ships but its data file
is scrubbed, and it skips on the file's absence - which is why the branch that
added the routes never saw it. Cost lands here at promotion time instead.
/api/endpoints is registered both with and without a trailing slash and both
forms are served, so both are described rather than guessing which one counts.
Spec regenerated: 353 paths, 504 operations.
plugins/endpoints/plugin.py had a contiguous run of continuation lines shifted
by one space. Python accepted it and git diff -w showed nothing, but it put
deletions in a diff that is otherwise purely additive. Restored; the file now
differs from the published main by additions only.
Scope: product
GE Aerospace now expects a Darkwing check on every repo. Two files rather than
one, because they answer different questions and only one of them is ours.
scan-darkwing-portable.yml is copied byte-for-byte from the ge-aero/darkwing
README so it keeps matching the sanctioned integration. Manual dispatch only.
scan-darkwing-pr.yml runs on pull requests and is ours to change. It exists
because the README workflow cannot simply be given a pull_request trigger: it
resolves its target as `inputs.target_ref || 'main'`, and a pull_request event
carries no inputs, so the expression falls through to 'main' and the job scans
the default branch instead of the code under review. That is a check that is
green because it never looked. This one resolves the PR head SHA explicitly.
Neither file gates a merge. Darkwing exits 0 whatever it finds, and its results
vary between runs on identical code, so a passing job means the scan ran and
nothing more. The SARIF-parsing step that would turn a finding into a failure is
written but commented out, to be switched on once the false-positive rate on
this codebase is known.
Scope: product
Everything on origin/wilm that is not on the published main, applied as one
change because the published history has rewritten hashes: per-commit
cherry-pick conflicts against our main even though the squashed diff applies
clean.
Contents: endpoint monitor catalog + policy attachment + warn-only drift
reconciler, operator issue reports from the tray, agent launch-path allowlist,
and the GHCR channel-image distribution path for dev workstations.
NOT READY FOR main. Open items:
- 10 new routes are absent from docs/api-inventory.json, so
test_api_inventory_parity fails. That gate SKIPS on the published mirror
(the test ships, its data file is scrubbed), which is why wilm never saw it.
- dev-bootstrap.ps1 and Start-ShopDBDev.ps1 hardcode the ge-aero GitHub org.
- plugins/endpoints/plugin.py carries a whitespace-only re-indent.
Scope: product
Eight fields across three asset types were shown on the detail page and
accepted by the API, but had no input on the form. A wrong value therefore
needed database access to correct, which is not a thing anybody should need
for a device category or a maintenance date.
The shape is the same in each case: a column was added, the detail card was
written, the API allowlist was updated, and the form was missed. Nothing was
broken - the values arrive by import or collector - so nothing complained.
PCs gain azureobjectid and azuredevicecategory. azuredeviceid was already
there, which is what made the omission easy to miss: the page showed two ids
and the form offered one. The two are not interchangeable and the pairing is
counterintuitive - the DEVICE id opens Intune, the OBJECT id opens Entra - so
each input says where it goes.
Machines gain the three maintenance fields. These matter beyond the immediate
fix: they are the columns a measuring instrument's calibration dates map onto
when metrology moves to machines, and until now there was no way to enter a
calibration date on a machine at all.
Printers gain iscolor, isduplex and isnetwork as a capability group.
isnetwork defaults to true in the form to match the API's create default; a
new printer claiming otherwise would be wrong for nearly all of them.
Checked before making any of these editable that nothing writes them
automatically - a hand edit silently reverted by the next collector poll would
be worse than no input at all. The printers collector reports queues and
supplies, not capabilities, and nothing writes machine maintenance dates,
which is exactly why they are all empty. The two genuinely collector-owned
computer fields, lastreporteddate and lastboottime, are deliberately still not
editable.
Scope: product
A CMM reported 100.0.0.1 as its address and never reported 10.48.255.210 at
all. Both halves of that were the same mistake.
THE BAY WAS CHOOSING. The collector schema took a single `ipaddress`, so the
script picked one NIC and dropped the others - preferring an address inside a
configured range, else whichever interface carried a default route. A bay
cannot make that judgement: it does not know the site's addressing. On this
CMM the Marvell AQtion 10Gbit link to its controller is a real hardware NIC,
up, and carries a default route of its own, so it satisfied every test the
script had. It won, and the corporate address was discarded before the request
was even built - which is why it is not in ShopDB even as a second row.
So the bay now reports EVERY connected hardware NIC with a routable IPv4, and
takes no view. The server picks the primary from collector_corporate_ranges, a
new per-site setting, and writes a Communication row for each of the others.
Naming the ranges is what turns the answer from a guess into a decision; with
none set it still falls back to the default-route flag and then to the first
reported, both guesses, both better than before only because nothing is thrown
away now.
Rows are keyed on the MAC where the bay sends one, because an address moves
between adapters and a MAC does not. A NIC that is down at check-in is left
alone rather than deleted - being down is not evidence the record is wrong,
and hand-entered gateway, DNS and serial settings live on those same rows.
Exactly one row ends up flagged primary, and only a row this report touched
can claim it, so a check-in cannot silently demote an address set by hand.
-AllowedRanges and the CollectorRanges registry value are now accepted and
ignored, logging a line that says where the setting moved. The parameter stays
because a scheduled task still passing it would otherwise fail on an unknown
argument.
A bay on an older collector sends one address and no interfaces; that keeps
working and is pinned by a test.
Scope: product
Twenty computer types existed and six had a machine on them. Three of the
empty ones were near-duplicates of live types - Shopfloor beside Shopfloor PC,
CMM beside CMM PC, Engineer Workstation beside Engineer - which is the kind of
list that grows because deleting from it by hand is nerve-wracking.
retype-by-role types a PC by WHAT IT CONTROLS. A metrology box has an active
relationship to a measuring_tool asset, so the fact is already recorded and
does not need inferring from a hostname or a tool's name. The imaging scope is
the authoritative source and stays so, but only 17 of roughly 292 PCs have
ever reported one, so it cannot answer for the fleet yet. Relationships can:
49 PCs drive a measuring tool, and 36 of them are currently typed as ordinary
shopfloor machines.
prune-types deletes a type only when nothing is on it AND the live pctype map
does not name it. The second guard is the one worth having: the map is what
the collector assigns from at check-in, so deleting a mapped type would leave
every future check-in logging 'mapped ComputerType not found' instead of
typing the machine.
Both report by default and write only with --apply, and prune prints why it is
keeping each type it keeps rather than only what it removes.
Also fixes `computers stats`, which counted the computer type named
'Shopfloor' - the EMPTY duplicate - rather than 'Shopfloor PC'. It has been
reporting 0 shopfloor PCs for as long as both rows existed, which is the same
duplication these commands clear up.
Scope: product
The Entra Device Info card showed two GUIDs and nothing to do with them.
Copying one into the portal by hand is the kind of errand a detail page exists
to remove.
Each link sits BESIDE the id it is built from rather than in a row at the top
of the card, because the pairing is counterintuitive: the DEVICE id opens
Intune and the OBJECT id opens Entra. A separate action row would have to
re-label which is which, and would still leave the reader matching them up.
BOTH URLS ARE SETTINGS WITH AN EMPTY DEFAULT, and that is the whole design.
The host is a tenant fact, not a product one - this site is on the US
Government cloud, portal.azure.us, where a commercial-cloud default would send
people to a page that does not exist. Microsoft also moves these blade paths,
and a site that hits that can fix it by editing a setting rather than waiting
for a release. Empty means no link and the id still renders as text, so a site
that configures nothing is unchanged. The example values live in each
setting's description, per cloud.
Same shape as the ServiceNow URL templates already in integrations, down to
the {token} placeholder, so there is one idiom for "deep link into a system we
do not own" rather than two.
Scope: product
An Azure device import died part-way with a foreign-key error, and the crash
was the lucky outcome.
The Azure adapter routes a device that already exists to its real asset by
writing that asset's assetnumber into the assetid column of the CSV it
generates, expecting the engine to look it up. The resolver instead takes any
all-digit string at face value as a literal id. This site numbers operations
0NNN, so numeric assetnumbers are ordinary here: `0704` was read as id 704,
which does not exist, hence the error. `0600` would have been read as id 600 -
a printer - and bound a PC to it silently, reporting success.
Two changes, both needed.
The resolver now tries the label first for an all-digit string and only falls
back to reading it as an id when nothing is named that. The face-value
behaviour exists for a real reason - exports round-trip and carry real ids, as
its original comment says - so it stays as the fallback rather than being
removed. Somebody writing `0600` in a column keyed on assetnumber means the
asset called 0600, and binding the row to an unrelated record is far worse than
failing.
_existing_hostname_map now hands over the assetid rather than the assetnumber.
Giving the resolver the id it is going to treat the value as makes the
passthrough correct by construction instead of correct by luck.
Verified against a restored copy of the production database: 234 devices, 221
matching an existing computer and updating in place, 13 new, no problems. The
same run previously failed on the foreign key.
Neither half was wrong alone. The passthrough dates from the original CSV
import and the hostname map from the wilm merge months later; they only meet
at a site whose natural keys are numeric.
Scope: product
"Metrics" promised numbers. The tab holds tracked services, tracked processes
and network interfaces - things being WATCHED and their current state, only
one of which is a measurement. Monitors is what the panels actually are, and
it matches the word the agent's own config already uses for them.
Moved to second, ahead of Software. What is being watched on this box is the
question asked right after "what is it", and well before its application
inventory - which is reference data somebody goes looking for deliberately.
Tab order is now Details, Monitors, Software, Activity.
Contract 0.29.0 keeps its number. It is unreleased, so the group vocabulary it
introduces is still being settled and nothing can yet depend on the old name.
Both the server and the client fall back to 'details' for a group they do not
recognise, so a site that deploys these halves out of step gets a misplaced
card rather than a missing one.
Scope: product
SOFTWARE OPENED COLLAPSED. The installed-software panel was collapsible and
started that way, because it used to be low-priority reference data near the
bottom of one long page. It is now the Software tab's whole reason to exist,
and somebody who clicked that tab has already said what they came for. Made
it neither collapsible nor collapsed.
TRACKED SERVICES VANISHED FROM METRICS. The group is decided by the SERVER, so
a new frontend against a server still serving the old vocabulary puts a panel
in a group the build does not render, and it is fetched, held, and shown
nowhere - with nothing logged. Moving services out of 'actions' is exactly that
shape. The client now falls back to 'details' for any group it does not render,
the same way the server does, so a version-skewed pair degrades to a misplaced
card instead of a missing one.
A TABLE PANEL WITH NOTHING IN IT HID ITSELF. panelVisible honoured `empty` for
list panels and ignored it for tables, for no reason anybody wrote down. A
declared empty message is an answer - "No installed software reported." - where
an absent card is a question. It matters more now that a panel can be the only
thing in a tab, because the tab disappears with it.
THE TAB STRIP HAD A VERTICAL SCROLLBAR. Per the overflow spec, when one axis is
not `visible` the other computes from `visible` to `auto`, so `overflow-x: auto`
alone made the strip scrollable vertically too and the -1px margin the tabs use
to sit on the border was enough to trip it.
Also moved .detail-full into the global stylesheet next to .content-grid, per
frontend/CLAUDE.md: it is detail-page layout, and the other four detail pages
need it when they adopt tabs.
Scope: product
Four changes from looking at the built page.
ACTIONS HELD ONE CARD. Tracked services was the only thing in it, and it was
the wrong split anyway: a service row reads as state first and a Start/Stop
second, so separating the buttons from the state they act on made a tab that
answered no question on its own. Services move to Metrics, beside the rest of
what the endpoint reports, and the group is gone from the contract.
WIDE PANELS SPAN, THEY DO NOT SCROLL. Network interfaces carries adapter,
address, MAC, speed and state; at half page width in the two-column flow it
either squeezed every cell to three lines or offered a horizontal scrollbar.
A scrollbar for data that just needs room is a layout failure. Table panels
now take column-span, which is the multi-column layout's own answer, so short
fact cards stay two-up around them. The wrapper keeps overflow-x only as a
floor for a genuinely narrow viewport.
SOFTWARE AND ACTIVITY RUN FULL WIDTH. Both are single long lists, and a list
split into two balanced columns reads as two unrelated lists with no way to
tell where the first ended. Details and Metrics keep the two-column flow
because they are short cards, which is what that flow is good at.
Contract 0.29.0 keeps its number: it is unreleased, and the group vocabulary
it introduces has simply lost one value before anybody could depend on it.
Scope: product
The page had grown to roughly ten cards in one flow, and three more surfaces
were wanted on it: installed software, live metrics, and the service controls.
Stacking those into the same column would have buried what the page is for.
Tabs were mocked and rejected on 2026-08-28, on the grounds that the strip
duplicated the sidebar and cost hero space. That objection is answered by
where the strip sits: the hero and the triage band stay above it and never
move, so "is this machine OK" is still answered without a click, and the strip
partitions the detail underneath rather than navigating anywhere.
WHY A PANEL DECLARES ITS OWN GROUP
Most cards on a detail page are not the page's. They arrive through the
ADR-010 get_asset_panels hook, so a page that sorted panels into tabs by
knowing their ids would defeat the hook it is built on. Panels now carry an
optional group instead, and the page only knows the five group names.
Resolution happens server-side, in the asset-panels route, so that an absent
or misspelled group becomes 'details' before the client ever sees it. The
client renders known groups only; an unrecognised value that survived to it
would be a panel that exists and is displayed nowhere. Contract 0.28.0 ->
0.29.0, additive - every existing plugin keeps working untouched, and the
endpoints status panel is deliberately left ungrouped so the default path is
exercised by a shipped plugin and not only by tests.
A tab is offered only when something fills it. An empty tab reads as broken
rather than as "this site does not collect that", so the strip is built from
the groups that actually returned data, and a tab that disappears on refetch
falls back to Details rather than leaving the page blank.
AssetDetailTabs is core, not a PCDetail component: machines, printers,
network devices and measuring tools all have the same shape, and the shared
asset-detail shell they need starts here.
Scope: product
A link is built by formatting that protocol's own linktemplate, and the
templates are rows in accessprotocols that a site can edit. So no scheme
is a fixed fact about the product, and writing one down as though it were
misleads whoever reads it next.
Two places did. The guest-grant docstring said the detail route resolves
"working rdp:// and vnc:// links", which is wrong for the row we actually
ship: VNC is com.realvnc.vncviewer.connect://{host}:{port}, the RealVNC
handler, not vnc://. And the protocol admin form offered vnc://{host}:{port}
as its placeholder, in the one screen where somebody authors a new template,
so the example contradicted the seeded data it sits next to.
Neither changes behaviour. _computer_access_links already reads the
template from the database and the frontend only ever renders the resolved
link, so nothing in the code assumed a scheme - only the prose did.
Scope: product
The sidebar box was drawn for anyone signed in, on the reasoning that
offering it to a signed-out reader gives them a control that can only
answer 401. That stopped being the right test when the guest role became
able to hold assets.view: the box was hidden from exactly the visitor the
grant was reopened for, and a guest could reach search only by typing
/search?q= by hand.
The client cannot decide this. Its hasPermission getter reads the signed-in
user's permission list and a guest has no user at all, so asking it about a
guest answers no for everyone. get_navigation already had the problem and
already solved it by letting the server decide, so capabilities does the
same thing and reports what the CALLER holds.
It reports booleans about the caller and never what the guest role holds in
general, so it tells an anonymous reader nothing the rendered page does not.
Public because it is the pre-login call that decides whether the box is
drawn at all.
Members who lack assets.view now lose a box that only ever 401'd for them.
On a failed call the box stays hidden rather than falling back to the old
signed-in test, which would put it back in front of those same members.
Scope: product
Inverting the grantable rule from the .view suffix to explicit declaration
dropped every permission that had qualified by how it was spelled. Two of
them were shipped guest features with grants already in the field.
guides.view: the nav item and list_guides both reason in comments about a
signed-out reader holding it, and the route is written around one - drafts
are filtered out, search matches the published revision so draft wording
cannot leak through a hit, and the author branches cannot open for a caller
with no user id.
tools.view: without it a guest holding tools.secrets has permission to use
Secure Send and no way to reach the page, because tools.view is what puts
the entry in the nav. The tools themselves run in the browser and touch no
data.
The failure was worse than a missing checkbox. update_role refuses the
WHOLE request when any submitted permission is undeclared, and the role
grid resubmits what the role already holds, so a site that granted either
of these under the old rule could no longer save the guest role at all -
including to tick assets.view, which is the grant the shop floor needs to
look up an asset number without signing in.
The test asserts both stay grantable, so narrowing the set again means
looking at what is already granted in the field rather than only at what
the catalogue declares.
Scope: product
Global search is gated on assets.view, and that was the whole check.
Behind it the route queried twelve domains: applications, the knowledge
base, guides, the staff directory, assets, measuring tools, USB devices,
printed parts, custom fields, notifications, hostnames, and vendor and
model reference data, plus addresses and subnets for an IP-shaped query.
None of them consulted their own permission.
So anyone who could look up an asset number could also read the staff
directory out of the HR database, the USB checkout history and who holds
which drive, and the network inventory. The SEARCH_DOMAINS settings look
like they bound this but they are site feature toggles, not
authorization.
Each domain now answers only to a caller who holds its permission.
Assets, custom fields and vendor/model stay on assets.view because all
three emit asset rows routed to asset pages.
caller_holds rather than require_permission: refusing the whole request
because one domain is off-limits would make search useless to anyone
holding less than everything. A domain the caller cannot see contributes
nothing and the rest still answers.
This is what makes assets.view safe to grant the guest role again. It
left the grantable set in 3474028 because of the leak above, and that
took the shop floor's asset lookup with it - a signed-out visitor at a
machine could no longer search for the asset number in front of them.
Now the grant reaches assets and nothing else.
The first version of the leak test passed with the gates removed. The
staff directory is an external HR database and is unreachable under
test, so the domain silently returned nothing and the assertion proved
nothing. It patches _employee_rows now, and fails when the gate goes.
Scope: product
(cherry picked from commit 0a01ddcf25)
Rows sorted on iscritical then on printer NAME. Critical is everything
at or below 5%, so a cartridge at 5% and one at 0% tied on the first key
and the alphabet decided which came first.
Wrong order on its own, and worse than it looks: the dashboard queue
keeps only the first few rows per source, so a printer that was
genuinely empty was truncated off the card while a fuller one held its
place. Reported from the floor as "starts at 5% and hides one at 0%",
which is exactly the two bugs compounding.
remaining is already normalised for waste cartridges - a full waste bin
reads as low remaining - so one ascending sort is worst-first for every
supply type, and iscritical falls out of it rather than needing its own
key. Name only breaks ties.
worstremaining is carried on the row because the supplies list is
rendered to text ("Black 0%") before it reaches the client, so the
number the sort needs is gone by then.
The toner forecast report was already worst-first (it sorts on daysleft)
and is unchanged.
Scope: product
(cherry picked from commit 6644a07ae1)
The export rsyncs the WORKING TREE, so .gitignore does not apply to it:
anything ignored-but-present reaches the publication repo unless it is
named as an exclude. devnotes/ is declared never-committed and holds
review writeups and chat handoffs, and it went straight through.
The scrub gate stopped it, which is what that gate is for, and nothing
was committed or bundled. But a gate failure is a stop, not a filter,
and the fix a hurried person reaches for is deleting their notes to get
a bundle out.
Same shape as the stray .env.bak that reached this tree once before, so
the rsync now carries a note saying why excludes live there rather than
in .gitignore.
Scope: product
(cherry picked from commit f1136d920d)
Delivery descriptors for run_script and update_app are resolved when the
agent claims a command, not when an operator queues it, and the command
row recorded no digest: the enqueue path resolved the descriptor purely
to validate it and threw the result away.
So what an operator approved and what an endpoint runs were two separate
lookups against mutable content, minutes or hours apart. Anyone able to
replace a script body or an installer package could do so in the gap.
The claim then re-resolves, the descriptor faithfully describes the new
bytes, the agent verifies correctly, and it runs as LocalSystem. The
audit trail names the operator who queued the original.
The gate on the verb and the gate on the payload were also different
grants: pushing needs endpoints.deploy, but replacing what gets pushed
needs only applications.edit.
pinnedsha256 records the digest as it stood at enqueue, and the claim
refuses a descriptor that has moved, failing the push with an audit row
instead of shipping it. Null for actions that deliver no content, and
for rows queued before this migration, which claim as before. A
descriptor that vanished entirely is not treated as tampering: that path
already sends the command through undescribed so the agent fails closed.
Separately, a result may now only finalize a command that is actually
running. That needs no agent change and closes two things: reporting on
a command that was never claimed, and overwriting a recorded failure
with a success.
The agent now sends its hostname on the result callback so the server
can check it owns the command it is finalizing. The shared collector key
does not say who is calling, so without it any agent can post a result
for any command id. The server treats the parameter as optional for
agents that predate this and enforces it when present, so there is no
flag day. This is a floor, not a fix: only a per-endpoint credential
makes the caller's identity provable, and that is tracked separately.
The .NET side is unbuilt here and needs a build plus a win11 VM run.
Scope: product
(cherry picked from commit d64b7122ab)
client_ip() read the first X-Forwarded-For hop unconditionally. The
login rate limit and the account lockout are keyed on that value and the
audit log records it, so rotating the header per request bought an
unbounded budget and an untraceable trail.
This walked around TRUSTED_PROXY_COUNT, which exists to decide exactly
this. The app leaves ProxyFix off by default and says why, ten lines of
comment away: ProxyFix trusts X-Forwarded-*, a client can send those
itself, and the login limiter is keyed on the address. Then client_ip
read the header anyway, so the stated protection never applied.
Resolution is now from remote_addr, which ProxyFix has already filled
from the header wherever a proxy is genuinely configured. Both cases end
at the same trustworthy source, and the port still comes off because ARR
forwards clientip:port and ProxyFix passes it through verbatim.
geenforce and tools had each hand-rolled the same mistake for their own
limiters. Rather than duplicate security-relevant logic a third time,
client_ip and strip_port join the shopdb.api surface (contract 0.28.0).
geenforce's _trusted_client_ip, which already refused the raw header for
its IP allowlist, is untouched and was the model for this.
The stability table was also missing rows for 0.26.0 and 0.27.0.
INSTALL-WINDOWS-IIS.md never mentioned TRUSTED_PROXY_COUNT, so a site
installed from it runs behind IIS with the setting at 0: every request
looks like it came from IIS, the whole site shares one rate-limit
bucket, and the audit log records the proxy for every action. It now
says so, and says not to set it when nothing is in front.
Three test modules simulated separate callers by setting
X-Forwarded-For, which is the header a real attacker rotates to escape
the limit. One of them, test_client_ip_prefers_forwarded_first_hop_
stripped, asserted the defect outright. The suite could not have caught
this. They now drive REMOTE_ADDR, with a regression guard that forging
the header buys nothing.
Scope: product
(cherry picked from commit 7260244b7e)
There is no token denylist in this product, so the JWT user loader is
the only place a deactivated account can be stopped, and it did a bare
lookup by id. Somebody deactivated at 09:00 kept a working access token
until it expired, and a refresh token that minted fresh ones for thirty
days.
/api/auth/refresh and the API-token path already checked isactive. This
makes the loader consistent with them rather than inventing policy.
Scope: product
(cherry picked from commit 9c1848fc61)
guest_grantable_permissions() opened any permission whose name ended
'.view' or '.read' to the guest role, on the reasoning that reading is
the safe default. That is true of a permission gating a list of asset
numbers and false of one gating bytes, and a suffix cannot tell them
apart. Twenty-four of eighty-five qualified and none had been decided:
applications.view reached the SYSTEM-context PowerShell bodies and
the installer binaries pushed to bays
backups.view returned the full payload that the separate
backups.download permission exists to protect
assets.view gated a global search that queries the staff
directory, USB history and network inventory with
no per-domain check of its own
audit.view usernames, addresses, the before/after of every
settings.view change, and every setting row - both admin
category, opened by naming alone
users.view gates nothing at all; list_users is
require_role('admin')
Every entry is now declared, by CORE_GUEST_GRANTABLE or by the plugin
that owns the permission, having read what its routes actually return.
Adding one is a code change, which is the point: it puts the decision
in a diff rather than in a suffix.
The two routes serving script bodies and installer packages move to a
new applications.download permission, which is not grantable, so a site
can still open the catalogue metadata to its floor without the bytes
going with it.
The declared set preserves the grants this site had already chosen:
applications.view, computers.view, endpoints.view, reports.view,
tools.secrets, warranty.view.
Two known gaps are recorded on the hooks that expose them rather than
silently carried: the computer serializer publishes every column plus
resolved rdp:// and vnc:// links, and the per-asset endpoint panels are
keyed on a small integer so walking ids reconstructs much of the roster
that endpoints.fleet exists to protect. Neither is widened here.
A shipped test was granting guests audit.view and passing.
Scope: product
(cherry picked from commit 347402896c)
A multi-agent review of the merged branch produced 29 findings that survived
adversarial verification. This clears the blockers and the four worth fixing
before it reaches main. The design held up well: path-traversal defence on all
four serving routes is clean, the agent's SHA-256 check fails closed, enqueue
authority is proportionate and audited, and no site literals or credential leaks
in the new C#.
THE TESTS NEVER RAN
pytest.ini sets testpaths = tests, so plugins/endpoints/tests/ was collected ZERO
times. Fifty-five lines of delivery assertions were dead code, and run directly
five of them failed. plugins/printedparts/tests/ had been invisible for longer.
That is the worst shape a gap can take: the suite is green, the coverage is
imaginary, and the collector auth on a download route could be deleted without
anything going red.
Both files moved under tests/. tests/test_no_hidden_tests.py now fails if a test
file appears anywhere pytest will not look; it caught a staged copy inside the
installer bundle on its first run, and was verified by planting a violation.
The five failures were stale expectations, and they split by cause. His: the
applications panel made six panels where his test said five, and 'applications'
belongs in KNOWN_PAYLOAD_KEYS now that it is persisted - his own test contradicted
his own feature. Ours: we renamed the events card to activity, moved the
fleet-wide cards to endpoints.fleet, and made the list route a bare route('')
which Flask rejects unless the blueprint is mounted under a prefix.
FOUR FIXES
The snapshot table's key was a 32-bit autoincrement on a table rewritten
wholesale every check-in, so it climbs at (endpoints x apps) per report and
exhausts on MySQL. Widened while the table is unreleased. Widening it broke every
insert on SQLite, which only autoincrements a column typed exactly INTEGER
PRIMARY KEY, so it carries a dialect variant - and nothing caught either fault,
because no test inserted into that table at all. One does now.
Ingest truncated displayname but not displayversion or publisher. On strict MySQL
an over-long registry value fails the whole check-in transaction, so one machine
with a malformed Uninstall entry goes silent entirely. SQLite does not enforce
lengths, so the suite would never have shown it.
update_app's version: branch pinned AppVersion.sha256, but that route resolves the
version to its APPLICATION and serves application.installpath - so the digest
described an artifact the route never sends, and where it was null the agent had
nothing to verify and failed open. It now pins the file actually served.
The same branch ended the delivery URL at the id, so an MSI arrived with no
extension and could not be routed through msiexec. It carries the filename now.
Agent packaging moved off require_role('admin') to a new endpoints.agentpackage
permission. The generated ZIP carries a live collector key, so this is a heavier
grant than deploying to one endpoint - and a role cannot be given to one person
for one job the way a permission can.
Full suite 2231 passed (2198 before, plus the 22 that never ran and 11 new),
frontend 314.
Commit 0c2b816 added Script to shopdb.api so a plugin can read the ShopDB-hosted script library that backs run_script delivery, but left __contract_version__ at 0.26.0. This bumps it to 0.27.0 so a plugin can pin core_version against the widened surface. Additive name, minor bump, matching the existing convention. The docs/PLUGIN-HOOKS wiki page (not tracked in-repo) still needs the Script entry.
Reverses everything the installer and site-agent generator create: stops and deletes the service, kills agent/tray processes, removes the tray Run key and the HKLM\SOFTWARE\GE\ShopDB credential hive in both registry views, drops the Defender path exclusion, and deletes the install directory and ProgramData tree. Every step is best-effort and idempotent so it is safe to re-run and safe on a partially installed host.
Terminate TLS in gunicorn using a host-generated cert mounted read-only under ./certs, so the browser sees a secure context. Secure-context-only web APIs (navigator.clipboard and others) refuse to run over plain http on a non-localhost origin, which the FQDN-accessed test box hits. Healthcheck now probes https with -k to match the TLS listener. A site swaps in its own cert or fronts this with its own IIS/nginx proxy.
Bumps the agent to 1.4.1. run_script fetches a ShopDB-hosted script by digest and runs it in the script's configured context (LocalSystem or the interactive logged-on user), replacing the local scriptpaths allowlist; user-context runs wait for exit and report the real exit code instead of fire-and-forget. update_app resolves an app-level or version-level install source, fetches the pinned bytes, verifies sha256, and runs silently. Both stage into a fixed Defender-excludable folder (C:\ProgramData\ShopDB\EndpointAgent) instead of the LocalSystem temp path. Adds the DeliveryClaim contract for the claim payload.
Adds the two delivery actions to the endpoints command allowlist, each gated on a new endpoints.deploy permission ON TOP OF endpoints.command so pushing software is a heavier grant than a routine toggle. Enqueue validates update_app appversionid ownership and installer presence before queueing. New collector-authenticated download routes let the agent fetch installer packages and script bodies by digest rather than pre-staging them or sending bodies over the command wire. Wires the scriptsApi client, the siteagent generator routes, and a temporary dev-only Delivery test bench (view/route/nav) for exercising the path while the production UI is built.
AppVersion gains sha256 and silentargs so the endpoint update_app push can pin exact installer bytes and carry per-version silent switches; Application gains installersha256 for the app-level install source. New ShopDB-hosted script library (Script model plus scripts API) backs run_script delivery, with the body fetched by digest on push rather than pre-staged or sent over the command wire. Two idempotent migrations with real downgrades. Script added to the shopdb.api contract surface for plugin use.
The command queue is at-most-one-in-flight, but nothing ever expired a stuck command. When an agent claimed a command (RUNNING) then died before reporting a result - killed, uninstalled, or offline mid-flight - the row stayed in-flight forever and every operator action for that endpoint returned 409 'already in flight'. Reap in-flight rows past a staleness ceiling (15m for RUNNING, 60m for PENDING) to a new STATUS_EXPIRED at enqueue time, inside the same transaction, so the queue self-heals lazily the next time someone queues work. No background job and no schema migration - status is free-text.
A misconfigured BaseUrl (plaintext http:// against the TLS port) made the command long-poll fail at connect in milliseconds. The worker misread that fast failure as a normal idle hold-window close and re-polled with no delay, spinning into a request storm. Gate the idle-close path on the poll having stayed open at least MinIdlePollDuration, so a sub-second transport failure takes the transient backoff instead; add a MinPollInterval floor so no path can exceed a safe request rate. Also reject a non-loopback http:// BaseUrl at startup with a single clear error, turning a silent runtime flood into a fast fail. Loopback http stays valid for local dev.
Several days of endpoint-agent work left DB dumps, sample datasets, captured command output, generated maps, vuln scan output, merge-kit exports, and staged agent publish trees in the tree. None are product source. Ignore them so status shows only real work and they cannot be committed by accident. exports/ stays on disk for reference. agent/.gitignore also reserves dev-bin/ and _stage/ for delivery-build scratch.
Tracking docs/openapi.json (so the app can serve the spec at runtime) makes docs/ exist on the published GitHub mirror, which defeated the skipif(not DOCS.is_dir()) guard in the docs publish/version tests: they then ran against a prose-less checkout and failed on CI. Key the skip off docs/*.md prose instead, matching the canonical published-checkout definition in scripts/check-gates-ran.sh.
Also: the OpenAPI generator emitted a trailing-slash path (/api/docs/), which a strict proxy like IIS treats as a redirecting resource a subpath-mounted client cannot follow; normalise trailing slashes in _norm_path and correct the committed spec. And allow docx (python-docx) as an undeclared import: it is used only by scripts/docs/generate_docx.py, a developer tool that never runs on a deployed site.
The endpoint agent already reports an installed-software list (SBOM), but ingress dropped it: the collector schema had no entry for it and there was nowhere to store it, so operators could not see what is installed on a PC.
Persist it as a per-asset snapshot in a new endpointapplications table, replaced on every report (current inventory is what operators ask for, not install history). Add applications to the collector schema and KNOWN_PAYLOAD_KEYS so ingress stops discarding it, a read route for the raw collection, and a collapsed-by-default Installed software panel to surface it. appid is nullable groundwork for a future Managed Applications link; no matcher is wired yet.
Standalone one-shot enrollment tool that creates a ShopDB PC + Endpoint entry using only the endpoints collector key, reusing the existing collector adopt path. Collects identity, WMI hardware, and installed-software SBOM; requires admin. Server-side packaging route attaches to the existing endpoints blueprint; no core contract change.
The agent now reports installed applications from both Uninstall registry hives (64-bit and WOW6432Node) as {displayname, displayversion, publisher}, mirroring the Assess-Orchestrator applications collection. Registry-only, no Win32_Product/WMIC self-repair and no AppX. The server logs it as an unpersisted key until the SBOM store is built.
The endpoint agent now sends disk, bios, memory, processor, osinfo, and domain telemetry. The server does not persist these yet, so ingest logs any unpersisted payload keys with a compact preview. This makes incoming telemetry observable in the logs before any UI or schema work, so we can confirm the agent is sending it and enhance the UI later.
wj held nothing main lacked - git cherry marks every one of its commits as
already present - but promotion cherry-picks, so the histories diverged and a
fast-forward is not possible. Merging rather than resetting keeps the branch
recoverable and needs no force-push.
Levelling first so wilm's patches apply against current product code.
# Conflicts:
# CONTRIBUTING.md
# frontend/src/views/Dashboard.vue
# tools/wj-status.sh
Asked twice what "All assets" covered, which is the signal a label is
underspecified rather than a reader being slow.
The endpoint already counted every status to build its rollup and then returned
two of them, so a reader could see In Use and had no way to learn where the rest
of the total went. It now returns them all, and the panel lists them under the
total - indented and quieter, because they break the total down rather than
adding to it.
Three things this makes visible that were not guessable:
Retired assets are IN the total. Retired is a status on an active record, not
the same as deactivated, and seven of them sit inside this site's 610.
There is an "Inventory" status. I would have hardcoded In Use, In Repair and
Retired and silently dropped it. The rows are built from whatever the API
returns, sorted by count, so a status a site adds appears rather than
disappearing - which matters for sites adopting with their own seed data.
Rows held back by isactive are in NO status bucket, so they cannot be found by
adding the buckets up. That gap is exactly what makes a dashboard total
disagree with a list page, so the count is reported and captioned.
The caption states the counting policy - active records only, dual-bay pairs
counted once - and the dual-path clause renders only when that site toggle is
on, since asserting it unconditionally would be false at a site that has it off.
Full suite 2182, frontend 314.
The panel had no group label before it was one panel, so grouping it needed a
heading and it got 'Estate' - RMM idiom, and the word the leadership documents
already used for the whole of what a site owns.
Wrong register for this surface. The sidebar already groups exactly these things
under a section named assets, so the dashboard had invented a second word for a
group the app already names, and the panel's own total row says 'All assets' -
the heading and the total disagreed inside one panel. 'Estate' is also
enterprise-IT and largely British idiom; it earns its place in an executive
summary and not on a screen read at 6am.
The scoped classes follow the label, so the markup does not keep a word the UI
stopped using. The documents keep 'estate', where it is doing real work for a
leadership reader.
Scope: product
The bundle became one file per branch, but the frontend dists did not: they are
still frontend-dist-subpath-{ops,shopdb}, fixed paths, and update-prod-server.cmd
robocopies the shopdb one onto the production server with /MIR - a full mirror -
with no idea which branch built it.
So --dist from a site branch would have replaced production's frontend with
unreleased work at the next deploy, silently, and nothing downstream could have
told. Worse than the bundle collision it was sitting beside, because the dist is
what actually lands on the server.
Branch-qualifying the directories would only move the trap to whoever pointed a
deploy at the wrong one. Production runs the product, so the export now refuses
and says which commits to promote instead.
Scope: product
Promotion cherry-picks, which gives the commit a new sha on main. wj-status
compared by sha via main..wj, so after promoting ten commits it still listed all
ten as unpromoted - and would have invited promoting the same work twice.
git cherry compares patch content instead and marks an already-present commit
with a leading '-'. When nothing is genuinely outstanding it now says so, counts
what has already landed, and shows how far the branch is BEHIND main with the
rebase to run. Behind was the state that mattered here and the tool could not
express it at all.
Scope: product
Recorded that wilm no longer existed, on the evidence of the local gitea clone
where it was merged and its ref deleted. That was wrong. It lives on the
corporate GitHub and the other site is developing the .NET endpoint agent on it.
The dev box cannot see that branch - no account there - so its absence locally
was never evidence of anything, and fetch-wilm-branch.cmd on the share exists
precisely to bring it back for review. Says so now, so the next reader does not
draw the same conclusion from the same missing ref.
Scope: product
The work-PC instructions began with 'git checkout <branch>', which fails the
first time a branch goes over because it does not exist in that clone yet -
exactly the moment somebody follows them most carefully. Nothing to check out,
and nothing to merge into either.
Now prints both: fetch then checkout -b then push -u the first time, and the
checkout/fetch/merge/push cycle afterwards.
Scope: product
(cherry picked from commit 1fe788a3ed)
promote-to-main.sh checks out main. The tooling lives only on the site branch,
so that checkout removes tools/promote-to-main.sh while the script is still
running - and bash reads a script lazily, by file offset, so it would carry on
executing whatever bytes happen to land there. While manipulating branches.
Not theoretical: main does not have tools/ right now, so the first real use
would have hit it.
Both versions now copy themselves to a temp file and re-exec from there. The
bash copy unlinks itself immediately, since the open handle keeps working. The
Windows copy has the same hazard for the same reason - cmd.exe re-seeks the
batch file after every command - and two extra traps worth naming: the copy runs
from %TEMP%, so the repository root has to travel with it rather than being
derived from %~dp0, and %errorlevel% inside a parenthesised block is expanded
when the block is PARSED, before the call it is meant to report on.
Scope: product
(cherry picked from commit 6d42110a91)
A guest got a heading and nothing else.
THE STRUCTURAL FAULT WAS MINE
Moving the Estate panel into DashboardCards' rail slot put it behind that
component's root v-if, which was `liveCards.length > 0`. A guest is permitted
zero cards, so the whole component disappeared and took the slot with it. Before
that move the tiles were top-level and always rendered. The wrapper no longer
vanishes with the cards.
TWO LIES THE EMPTY STATE WAS TELLING
"Nothing needs attention" is now shown only to somebody who can see the checks.
A reader permitted none of them is not looking at a healthy fleet, they are
looking at nothing, and saying otherwise is the same fault as reporting Low
Supplies 0 while the monitoring is down. The count of PERMITTED cards is now
tracked separately from the count that currently hold rows.
The Estate panel no longer renders zeros for a caller who cannot read totals.
/api/dashboard/summary needs assets.view, and the failure was swallowed into an
empty object that drew a tidy column of zeros - an estate of nothing, stated
with confidence.
WHAT A GUEST GETS INSTEAD
Where they can go: the same permission-filtered navigation the sidebar uses,
which the server already filters to the guest role for a signed-out visitor. So
it cannot offer a link that dead-ends at the router guard, and this page names no
plugin it had to know about. Verified against the running instance - a guest is
offered Guides and Tech Tools, matching guides.view and tools.view.
ALSO: the measuring tools row
totalassets counts five types - machines, PCs, network, printers AND measuring
tools - but only four had rows, so the total stopped matching the addends above
it wherever gages exist: 610 of visible rows under a stated 729. That is exactly
the fault this panel was built to remove, and it looked right here only because
this database has no measuring tools. Every counted type now has a row.
Frontend 314 passed.
(cherry picked from commit 0f2da9beaf)
The export wrote a single shopdb-flask-pub.bundle whatever it published. That
was fine while only main was ever published, and became a trap the moment it was
not: exporting a site branch silently overwrote the bundle holding main's work,
and the work PC's documented "git fetch <bundle> main" then failed because that
ref was no longer in the file.
It had already happened here - two wj exports overwrote a main bundle from the
same morning. The failure is at least loud on the work PC rather than pushing
the wrong branch, but the main bundle has to be regenerated.
Now shopdb-flask-pub-<branch>.bundle. Two branches cannot share one filename.
Scope: product
(cherry picked from commit 1311dea113)
The branch checkout landed after the rsync, where git checkout --force reverts
the tree to the branch's committed state and discards everything rsync had just
written. The export then found no changes and published nothing, while still
printing a bundle path and looking like it had worked. A silent no-op is the
worst shape for this: the next person believes the export ran.
Scope: product
(cherry picked from commit 06c584240e)
export-github.sh syncs the WORKING TREE, not a branch, and then committed
whatever it found onto the publication repo's main and printed
"git push origin main".
So running it while checked out on a site branch published that site's work AS
THE PRODUCT, to the repository every other site consumes, and said nothing. The
branch model made that reachable in one wrong command, from a checkout that
looks completely normal.
It now reads the current branch, mirrors it in the publication repo (creating it
from main the first time, as in the dev repo), bundles that branch, and prints
work-PC instructions naming it. Publishing anything other than main also prints
a reminder that other sites read main and these commits are not there yet.
The publication checkout uses --force, which is safe only because that tree is a
mirror: the rsync has already replaced it, so there is nothing local to lose. It
would not be safe anywhere else.
Also cleans up the publication repo: origin pointed at a bundle in a scratchpad
from a session that ended long ago, and two orphaned remote-tracking refs
(github/main, wilm/review-fixes) survived the removal of their remotes. None of
it broke the export, which never pushes, but a `git fetch` in there failed with
a confusing path.
Scope: product
(cherry picked from commit 81df9607e7)
The convention existed only in the scripts' comments and in one conversation,
which is not where somebody joining looks. CONTRIBUTING is.
Records the two branches, the Scope trailer, the promote flow, and the point
that most work is product: ADR-015 puts site differences in settings, seed data
or scripts/site_imports/<site>/, so a site branch is about timing rather than
locality.
Also names the failure worth watching. Promoting something site-specific by
accident is caught by the ADR-015 gate on the way into main. The real risk is
product work stranded on a site branch because nobody decided, which is what
turned wilm from a merge into a port.
Scope: product
(cherry picked from commit 1d4ac11572)
The scripts could move a commit to main but nothing said which commits should
go, and that decision does not survive being deferred: months later it is a list
of subject lines and a guess.
So it is recorded at commit time, as a trailer:
Scope: product the rest of the world should have it
Scope: site this site alone, keep it here
tools/wj-status.sh reads them and prints main..wj in three groups, with the
promote command for the ones marked product.
The path fallback is deliberately weak. Under ADR-015 the only site-specific
home in the tree is scripts/site_imports/<site>/ - everything else is meant to
be product, with settings and seed data carrying what differs between sites. So
"touches only site_imports" is the one confident site verdict paths can give,
and everything else is reported UNMARKED rather than guessed into a bucket. A
tool that quietly sorted commits by directory would be wrong often enough to
stop being read.
The unmarked message says which way to lean, because the failure mode here is
not promoting something site-specific by accident - the ADR-015 gate catches
that on the way in. It is product work sitting on wj forever because nobody
decided, which is how the wilm branch became a port rather than a merge.
Scope: product
(cherry picked from commit 0400a58786)
Same two operations as the bash pair, for driving git from Windows rather than
from the development box.
CREDENTIALS ARE HANDLED DIFFERENTLY, ON PURPOSE
The bash version reads a token out of secrets.md and feeds it to git through
GIT_ASKPASS, because that box has no credential store. Windows has one: Git for
Windows installs Git Credential Manager and enables it by default, so `git push`
prompts once and Windows Credential Manager keeps it.
So the token handling is deliberately NOT ported. Copying a token into a script,
an environment variable or a remote URL on a machine that already has a
credential store is strictly worse than the thing it would replace, and the
comment in the file says so, because "make it match the other script" is exactly
the tidying somebody would otherwise do later.
THE PROMOTE SCRIPT REFUSES TO RUN WITHOUT BASH
scripts/check-naming-and-style.sh is a shell script, and it carries the ADR-015
site-literal check - the one gate that makes a site branch survivable rather
than a slow leak into the product. Skipping it on a machine without bash would
defeat the entire point of the script, so it stops and says how to fix it
instead of promoting without it.
CRLF
.gitattributes already pinned *.sh to LF so Git Bash can run them; *.cmd is now
pinned to CRLF for the same class of reason in the other direction. cmd.exe
parses a lone LF inconsistently around labels and goto, so an LF-only batch file
can run its happy path and then jump nowhere on the error path - which in a
promote script is the path that matters.
(cherry picked from commit 49a898fc5a)
wj holds this site's work; main is the product every other site sees and the
publication mirror exports. Two scripts, one for each direction.
tools/git-push.sh pushes a branch without leaving the token behind. The way this
was being done by hand - rewrite remote.origin.url to embed the token, push,
rewrite it back - writes the token into .git/config in plaintext on every push,
and leaves it there if the push fails or the terminal closes mid-way, where
`git remote -v` will happily print it. Putting it in the URL on the command line
is no better: argv is visible in `ps` to every user on the box. The token now
goes in through GIT_ASKPASS, so it never reaches the config or the process list
and there is nothing to clean up when the script dies.
tools/promote-to-main.sh cherry-picks NAMED commits onto main, rather than
merging the branch. Merging takes the site-specific work along with the product
work, which is how a site branch quietly becomes the product; naming the commits
keeps that decision explicit and per change.
It runs main's gates after the pick and before any push, and does not push at
all - it prints what landed and leaves that to a human. The ADR-015 site-literal
check is the one that earns its place: it catches a hostname or share path
riding along in a commit that looked generic, which is the failure that makes a
site branch dangerous rather than merely awkward. It caught this very commit,
which named the site in a comment.
Both refuse a dirty tree. Pushing one pushes the last commit rather than what is
in front of you, and cherry-picking from one mixes uncommitted work into the
promotion. On failure the promote script aborts the pick and returns you to the
branch you started on, with main untouched.
(cherry picked from commit 6d9954c6f9)
The work-PC instructions began with 'git checkout <branch>', which fails the
first time a branch goes over because it does not exist in that clone yet -
exactly the moment somebody follows them most carefully. Nothing to check out,
and nothing to merge into either.
Now prints both: fetch then checkout -b then push -u the first time, and the
checkout/fetch/merge/push cycle afterwards.
Scope: product
promote-to-main.sh checks out main. The tooling lives only on the site branch,
so that checkout removes tools/promote-to-main.sh while the script is still
running - and bash reads a script lazily, by file offset, so it would carry on
executing whatever bytes happen to land there. While manipulating branches.
Not theoretical: main does not have tools/ right now, so the first real use
would have hit it.
Both versions now copy themselves to a temp file and re-exec from there. The
bash copy unlinks itself immediately, since the open handle keeps working. The
Windows copy has the same hazard for the same reason - cmd.exe re-seeks the
batch file after every command - and two extra traps worth naming: the copy runs
from %TEMP%, so the repository root has to travel with it rather than being
derived from %~dp0, and %errorlevel% inside a parenthesised block is expanded
when the block is PARSED, before the call it is meant to report on.
Scope: product
A guest got a heading and nothing else.
THE STRUCTURAL FAULT WAS MINE
Moving the Estate panel into DashboardCards' rail slot put it behind that
component's root v-if, which was `liveCards.length > 0`. A guest is permitted
zero cards, so the whole component disappeared and took the slot with it. Before
that move the tiles were top-level and always rendered. The wrapper no longer
vanishes with the cards.
TWO LIES THE EMPTY STATE WAS TELLING
"Nothing needs attention" is now shown only to somebody who can see the checks.
A reader permitted none of them is not looking at a healthy fleet, they are
looking at nothing, and saying otherwise is the same fault as reporting Low
Supplies 0 while the monitoring is down. The count of PERMITTED cards is now
tracked separately from the count that currently hold rows.
The Estate panel no longer renders zeros for a caller who cannot read totals.
/api/dashboard/summary needs assets.view, and the failure was swallowed into an
empty object that drew a tidy column of zeros - an estate of nothing, stated
with confidence.
WHAT A GUEST GETS INSTEAD
Where they can go: the same permission-filtered navigation the sidebar uses,
which the server already filters to the guest role for a signed-out visitor. So
it cannot offer a link that dead-ends at the router guard, and this page names no
plugin it had to know about. Verified against the running instance - a guest is
offered Guides and Tech Tools, matching guides.view and tools.view.
ALSO: the measuring tools row
totalassets counts five types - machines, PCs, network, printers AND measuring
tools - but only four had rows, so the total stopped matching the addends above
it wherever gages exist: 610 of visible rows under a stated 729. That is exactly
the fault this panel was built to remove, and it looked right here only because
this database has no measuring tools. Every counted type now has a row.
Frontend 314 passed.
The export wrote a single shopdb-flask-pub.bundle whatever it published. That
was fine while only main was ever published, and became a trap the moment it was
not: exporting a site branch silently overwrote the bundle holding main's work,
and the work PC's documented "git fetch <bundle> main" then failed because that
ref was no longer in the file.
It had already happened here - two wj exports overwrote a main bundle from the
same morning. The failure is at least loud on the work PC rather than pushing
the wrong branch, but the main bundle has to be regenerated.
Now shopdb-flask-pub-<branch>.bundle. Two branches cannot share one filename.
Scope: product
The branch checkout landed after the rsync, where git checkout --force reverts
the tree to the branch's committed state and discards everything rsync had just
written. The export then found no changes and published nothing, while still
printing a bundle path and looking like it had worked. A silent no-op is the
worst shape for this: the next person believes the export ran.
Scope: product
export-github.sh syncs the WORKING TREE, not a branch, and then committed
whatever it found onto the publication repo's main and printed
"git push origin main".
So running it while checked out on a site branch published that site's work AS
THE PRODUCT, to the repository every other site consumes, and said nothing. The
branch model made that reachable in one wrong command, from a checkout that
looks completely normal.
It now reads the current branch, mirrors it in the publication repo (creating it
from main the first time, as in the dev repo), bundles that branch, and prints
work-PC instructions naming it. Publishing anything other than main also prints
a reminder that other sites read main and these commits are not there yet.
The publication checkout uses --force, which is safe only because that tree is a
mirror: the rsync has already replaced it, so there is nothing local to lose. It
would not be safe anywhere else.
Also cleans up the publication repo: origin pointed at a bundle in a scratchpad
from a session that ended long ago, and two orphaned remote-tracking refs
(github/main, wilm/review-fixes) survived the removal of their remotes. None of
it broke the export, which never pushes, but a `git fetch` in there failed with
a confusing path.
Scope: product
The convention existed only in the scripts' comments and in one conversation,
which is not where somebody joining looks. CONTRIBUTING is.
Records the two branches, the Scope trailer, the promote flow, and the point
that most work is product: ADR-015 puts site differences in settings, seed data
or scripts/site_imports/<site>/, so a site branch is about timing rather than
locality.
Also names the failure worth watching. Promoting something site-specific by
accident is caught by the ADR-015 gate on the way into main. The real risk is
product work stranded on a site branch because nobody decided, which is what
turned wilm from a merge into a port.
Scope: product
The scripts could move a commit to main but nothing said which commits should
go, and that decision does not survive being deferred: months later it is a list
of subject lines and a guess.
So it is recorded at commit time, as a trailer:
Scope: product the rest of the world should have it
Scope: site this site alone, keep it here
tools/wj-status.sh reads them and prints main..wj in three groups, with the
promote command for the ones marked product.
The path fallback is deliberately weak. Under ADR-015 the only site-specific
home in the tree is scripts/site_imports/<site>/ - everything else is meant to
be product, with settings and seed data carrying what differs between sites. So
"touches only site_imports" is the one confident site verdict paths can give,
and everything else is reported UNMARKED rather than guessed into a bucket. A
tool that quietly sorted commits by directory would be wrong often enough to
stop being read.
The unmarked message says which way to lean, because the failure mode here is
not promoting something site-specific by accident - the ADR-015 gate catches
that on the way in. It is product work sitting on wj forever because nobody
decided, which is how the wilm branch became a port rather than a merge.
Scope: product
Same two operations as the bash pair, for driving git from Windows rather than
from the development box.
CREDENTIALS ARE HANDLED DIFFERENTLY, ON PURPOSE
The bash version reads a token out of secrets.md and feeds it to git through
GIT_ASKPASS, because that box has no credential store. Windows has one: Git for
Windows installs Git Credential Manager and enables it by default, so `git push`
prompts once and Windows Credential Manager keeps it.
So the token handling is deliberately NOT ported. Copying a token into a script,
an environment variable or a remote URL on a machine that already has a
credential store is strictly worse than the thing it would replace, and the
comment in the file says so, because "make it match the other script" is exactly
the tidying somebody would otherwise do later.
THE PROMOTE SCRIPT REFUSES TO RUN WITHOUT BASH
scripts/check-naming-and-style.sh is a shell script, and it carries the ADR-015
site-literal check - the one gate that makes a site branch survivable rather
than a slow leak into the product. Skipping it on a machine without bash would
defeat the entire point of the script, so it stops and says how to fix it
instead of promoting without it.
CRLF
.gitattributes already pinned *.sh to LF so Git Bash can run them; *.cmd is now
pinned to CRLF for the same class of reason in the other direction. cmd.exe
parses a lone LF inconsistently around labels and goto, so an LF-only batch file
can run its happy path and then jump nowhere on the error path - which in a
promote script is the path that matters.
wj holds this site's work; main is the product every other site sees and the
publication mirror exports. Two scripts, one for each direction.
tools/git-push.sh pushes a branch without leaving the token behind. The way this
was being done by hand - rewrite remote.origin.url to embed the token, push,
rewrite it back - writes the token into .git/config in plaintext on every push,
and leaves it there if the push fails or the terminal closes mid-way, where
`git remote -v` will happily print it. Putting it in the URL on the command line
is no better: argv is visible in `ps` to every user on the box. The token now
goes in through GIT_ASKPASS, so it never reaches the config or the process list
and there is nothing to clean up when the script dies.
tools/promote-to-main.sh cherry-picks NAMED commits onto main, rather than
merging the branch. Merging takes the site-specific work along with the product
work, which is how a site branch quietly becomes the product; naming the commits
keeps that decision explicit and per change.
It runs main's gates after the pick and before any push, and does not push at
all - it prints what landed and leaves that to a human. The ADR-015 site-literal
check is the one that earns its place: it catches a hostname or share path
riding along in a commit that looked generic, which is the failure that makes a
site branch dangerous rather than merely awkward. It caught this very commit,
which named the site in a comment.
Both refuse a dirty tree. Pushing one pushes the last commit rather than what is
in front of you, and cherry-picking from one mixes uncommitted work into the
promotion. On failure the promote script aborts the pick and returns you to the
branch you started on, with main untouched.
Splitting "cannot see the cartridges" from "nothing is low" was right, but it
treated two unlike cases as one. isconfigured means enabled AND a URL AND a
token, so its absence means the site did not ask for supply monitoring at all -
and a permanent "Supply monitoring is not configured" line is a card reporting
nothing wrong every day, which is how a board teaches people to stop reading it.
It matters for sites adopting ShopDB without Zabbix, who would otherwise inherit
a warning about a feature they declined.
Not configured now returns a plain empty and the card hides. Only configured and
unanswerable reports itself, because that is something meant to work that is not
working.
Also withdraws an unfounded inference. A dev box has no route to Zabbix by
design, so "not reachable" there is the expected state and says nothing about
what production can see. The value of this change is not that it revealed a
production fault; it is that when production's Zabbix does go down, the board
will say so instead of displaying "Low Supplies 0".
Four tests pin the three cases apart: unreachable reports, unconfigured stays
quiet, and a real zero from a reachable Zabbix is not flagged as blindness. The
fourth holds the envelope's rows, since carrying a reason changed the response
shape. Verified they are not vacuous by restoring the original branch in place -
the unreachable test fails against it.
Full suite 2182 passed.
TWO CONTROLS, ONE BOARD
Severity answers "how bad". It could not answer "what kind", and on a fleet
where eight endpoints go quiet at once, being able to set those aside and look
at the four crashes is the difference between a board and a list.
Chips per contributing check, coloured by its severity, composing with the
severity rollup above them. Both narrow what is already loaded rather than
refetching, so no combination can disagree with the counts. Picking a severity
that excludes the active chip clears the chip: two controls lit above an empty
queue, neither obviously wrong, is worse than losing the narrower one.
A chip counts every row, not the rows the queue had room for. A chip reading 5
above a "3 more" line would be its own small lie.
"LOW SUPPLIES 0" WAS NOT TRUE
Asked what happened to toner replacement, and the answer was worse than the
layout question that prompted it.
_get_low_supplies_data returned the same empty result for three different
situations: Zabbix not configured, Zabbix unreachable, and every cartridge
full. The dashboard strip therefore displayed "Low Supplies 0 / Critical 0"
while the monitoring was dead - and this instance is exactly that case, with
zabbix_enabled true against a URL nothing answers on.
So the strip removed in the previous commit was not information. It was
reassurance, and it read identically whether the printers were fine or nobody
could see them. That is the failure this codebase already names elsewhere: an
empty card and a broken card must not look the same.
The endpoint now distinguishes the three, and returns an envelope rather than a
bare list so a card can say WHY it has no rows. cardVisible shows such a card
whatever its empty rule, because "I cannot tell" is not "nothing to report".
It renders as a notice line, not a queue row: there is no asset to act on, and
burying "the monitor is dead" among the things it failed to find would be the
same mistake in a new place.
Note what this does NOT fix: the Zabbix URL on this instance is a dev
placeholder. The board will now say so instead of implying the cartridges are
full, but somebody still has to point it at a real server - and it is worth
checking what production points at, since nothing would have shown there either.
Frontend 314 passed (13 new), full suite 2178. The response shape changed from
a list to an envelope; no test depended on the old shape.
The board was one card per source in a fixed grid, and both halves of that hurt.
Every source got the same small box whatever it had to say. Warranties showed 5
of 50 behind "and 45 more"; the crash card truncated at 4; a one-row card sat
beside a four-row card and left a hole. Nothing had room, which is the complaint
that started this: not enough space to relay the data.
And a box per source flattened priority. An enforcement failure and a warranty
notice looked equally urgent because each owned a card, while the largest type
on the page belonged to counts that are true every day and need nobody.
WHAT THE PAGE IS NOW
A severity rollup, one worst-first queue of everything that needs a person, and
a row of context beneath it.
The queue merges every critical and warning row and sorts them, so reading order
is work order. It is full width, which is where the room came from. Rows are
capped per source, because fifty expiring warranties must not bury two crashed
PCs, and what did not fit is named per source with a link to it - a bare "and 45
more" that leads nowhere is worse than saying nothing.
The rollup counts ROWS, not cards, and filters the board in place rather than
navigating away, so the counts and the rows cannot disagree.
Severity, position and permission were already declared per card and already
sorted on. The old layout threw that away; this only stops it.
WHAT WENT, AND WHY IT SHOULD NOT COME BACK
Four asset tiles plus an "All assets" tile: machines + PCs + printers + network
IS all assets, so the page showed a total beside its own addends at the same
weight. It is now a line under a rule, where a total belongs.
The printer strip. It carried the printer count a third time - the tile row and
"Total Printers" were already the same number - and "Online" equalled total, so
three tiles held one fact. A genuinely low cartridge already reaches the queue
through the printers-supplies card, which is the moment anyone wants it.
Three cards whose whole job was to display 0.
Recent Devices: the largest block on the page, listing five machines that were
all "In Use" with business unit "TBD". Recency is not a reason to look at
something.
CONTRACT 0.26.0: 'shortname'
Cards may declare a short label for their rows in the merged queue. Their
'title' is written as a heading - "Endpoints with crashes or shutdowns" - and as
a row badge it ran wider than the hostname it qualified, putting the badge
rather than the asset at the start of every line. CSS-truncating the title was
tried first and was worse: four consecutive rows reading "ENDPOINTS WITH CR..."
say nothing at all. Optional and defaults to the title, so a plugin that does
not declare one is unaffected. Additive field, minor bump.
ALSO: "silent 1 days"
A meta suffix is written for the plural case and the card cannot know the value.
One row saying it was a shrug; the merged queue puts every quiet endpoint on one
screen, so it said it five times in a column and read as broken. Now singularised
at exactly 1, on the first word, keeping what follows: "1 event (48h)", "silent 1
day". The test asserting ' ms' survives is what caught the first version turning
it into ' m'.
DashboardRow.vue is extracted because the queue and the context panels render
the same row, and the map tooltip, link fallbacks and chips are too easy to get
subtly different in a second copy.
Verified on screen against live data. Full suite 2178 passed, frontend 301.
The triage band now answers the common case above the fold - what an asset
serves, or what drives it - so the full edge list below is reference rather than
the first thing to read. On a machine showing two relationships, both already
appeared in the band and the banner, and the card was a third copy.
Collapsed by default, using the same caret, roles and keyboard handling as the
collapsible panels on the same page rather than a second, slightly different
one.
The card is NOT redundant with the band and is not removed. It is the only
surface with Add Relationship and the per-row delete, and it keeps each edge's
exact type where the band deduplicates by asset. The printer page is the proof:
four rows there - usesprinter and defaultprinter, for each of two machines -
collapse to two band chips, and which machine DEFAULTS to that printer is a
distinction the band deliberately drops. Fine for triage, wrong for management.
THE COUNT IS OF WHAT IS RENDERED, NOT OF EDGES
A collapsed card that just says "Relationships" cannot be told from an empty
one, so the header carries a count. The first version counted raw edges and was
wrong on the first machine it was shown: a symmetric Dualpath link is stored in
BOTH directions in the legacy shape, so it is two rows in outgoing+incoming and
one "Connected" entry on screen. The badge read 3 above a card that shows 2 when
expanded - exactly the mismatch the badge exists to prevent. It now counts
rendered items.
Add Relationship expands the card before opening the modal, so a new row cannot
land behind a closed caret and read as the add having failed.
Verified on screen: machine reads 2 and matches, printer reads 4 and matches.
Frontend 280 passed.
TWO STRIPS OF CHROME SAYING THE SAME KIND OF THING
The machine page opened with a banner - "Dual-bay machine - sibling bay: 2022" -
directly above a hero whose band said "Controlled by". Both answer "what am I
looking at", and splitting them across a banner and a band cost a strip of
vertical space to say so.
The sibling is now a chip in the band beside the controlling PC. The band takes
an `extraitems` prop rather than learning what dualpath is: the pairing is
resolved by the machines plugin, not from the relationships endpoint, so core
cannot derive it, and a dualpath-shaped prop on a core component would be the
machines plugin leaking upward. A page passes [{label, links:[{text, to}]}] and
the band renders it with its own markup, which also keeps the chips visually
identical to the ones it derives itself.
The label stays "Dual-bay sibling" rather than "Dualpath". The banner spelled
the meaning out because a dualpath pair is ONE physical machine in two bays; a
chip reading "Dualpath" would have dropped that.
THE GAGE PAGE FILTERED FOR A COUNTERPART THAT CANNOT EXIST
MeasuringToolDetail was wired counterparttype="machine" labelled "Used on".
Wrong on both counts. The import synthesizes one measuring-tool asset per
metrology PC and a 'controls' edge PC -> tool (stage_metrology), so a gage's
counterpart is a COMPUTER, exactly as on the machine page. Filtering for a
machine peer meant the band rendered nothing on every gage that has a PC.
It looked merely empty rather than wrong because this dev database holds one
measuring tool with no relationships at all, so there was nothing to be missing.
A hand-held gage genuinely has no such edge and correctly shows nothing; a
metrology station now shows the PC that drives it.
The relationships card stays. It is the only surface with Add Relationship and
the per-row delete, and it lists types the band filters out - the band is
triage, the card is management. It does now repeat what the band shows, which
is worth revisiting separately.
Verified on screen. Frontend 280 passed.
Three defects broke the loader's own documented resume guarantee. All three
were silent: nothing raised, the run finished cheerfully, and fewer rows
arrived than anyone counted.
THE PAGE SIZE WAS CLAMPED, SO THE 409-RESOLVE SEARCHED THE FIRST 100 ROWS
_upsert asked for ?per_page=10000 and read one response. The server clamps any
page size to MAX_PAGE_SIZE, which is 100 - not an error, not a warning, just
100 rows. Anything past that resolved to None, so the id never reached the
crosswalk and the run quietly dropped a model's link on every machine that used
it, or an application's installs.
The target already holds more than 100 models and more than 100 applications,
so this fired on any re-run or resume - exactly the case the loader documents
as safe.
_resolve_existing now tries the exact natural-key filter the API has exposed
all along (?modelnumber= on models, ?appname= on applications, ?vendor= on
vendors) and then pages properly, bounded. An endpoint without such a filter
ignores the unknown parameter and the paged walk finds the row.
stage_metrology repeated the same mistake in its own way: it skipped 409s
entirely, so a resumed run left the tool out of the crosswalk and the Controls
edge unmade - metrology PCs driving nothing. stage_printers already had the
resolve branch; that stage was written later and missed it.
THE CROSSWALK WAS NOT BOUND TO THE DATABASE IT DESCRIBED
idmap.json maps this site's legacy ids to assetids in ONE database, and those
assetids mean nothing anywhere else. Keeping it across a drop-and-rebuild left
warranties and printer assignments attaching to whatever asset now holds that
id: wrong data, no error. The runbook's two "drop and rebuild" steps never
mentioned the file.
The loader now writes a uuid marker into the TARGET's own settings table, so it
dies with the data, and stamps it into the crosswalk. A crosswalk built against
a different database is refused before a row is written. The database name
alone would not do - a rebuild keeps the name. A file written before this guard
is adopted rather than refused, so no existing crosswalk is stranded.
Also adds --idmap, so one checkout can drive more than one target.
THE SOURCE DATABASES ARE NAMED, NOT HARDCODED
'shopdb_src' appeared at 29 call sites, so aiming the loader at the prodscratch
restore meant editing run.py. Those names are now LOGICAL and resolved in
Source.db() from SOURCE_DB_MAIN / SOURCE_DB_USB / SOURCE_DB_EMPLOYEES, with
host, port, user and password overridable too. The call sites are untouched -
resolving centrally avoids missing one, and the literals now read as
documentation of which source a stage draws from. The run prints every source
it redirected, so "it imported nothing" can be told apart from "it read the
wrong database".
This is what stood between the loader and the prodscratch run.
TWO SMALLER ONES
_source_password() raised IndexError rather than its intended message whenever
the DATABASE_URL user is not root, which it will not be on a real server.
main() returned 1 on failure into a caller that discarded it. A run that logged
200 endpoint errors exited 0 and read as a success in a deploy log; it now
exits non-zero.
TESTS AND DOCS
The loader had no tests at all. Sixteen now cover the resolve, the crosswalk
binding and the source aliasing - the logic that can run without a MySQL
source, which is the part that was wrong. Verified they are not vacuous by
running the old algorithm against the same fixture: it returns None where the
new one returns the row.
The README claimed 15 stages "complete" with counts from 2026-07-13 that two
later routing changes invalidated, and the runbook told operators to verify a
fresh import against them. Both corrected, with the three gaps NOT fixed here
recorded plainly: warranties, notifications and knowledgebase POST without a
duplicate check so resuming across them doubles rows; there is no --dry-run or
target guard; and the verify stage counts target assets against source machines,
which is apples to oranges.
Full suite 2178 passed.
THE RECIPROCAL LINKS BUG
A PC's page showed "Serves machine 7803" at the top. The machine's page showed
nothing equivalent, though the same edge was right there in the relationships
card at the bottom. That asymmetry read as missing data and was reported as
such.
It was neither missing data nor an API fault. The endpoint returns both
directions and always did; AssetTriageBand already walked outgoing AND incoming
and deduped, with a comment explaining that production holds reciprocal rows.
It then discarded every peer whose asset type was not 'machine', and hardcoded
the label "Serves machine". So the component could only ever work from the PC
side, and it was mounted only on the PC page.
The counterpart type and label are now props, defaulting to the old values so
the PC page is unchanged. The machine page mounts the band with
counterparttype="computer" and the label "Controlled by".
Eliminated with evidence rather than assumed, because this bug had already been
misdiagnosed twice: the endpoint runs two queries and returns both arrays; the
generic card renders both; that route has no pagination to clamp; and a single
permission gate cannot admit one direction and not the other.
THE RELATIONSHIP TYPE WAS ALSO HARDCODED
Not every asset pair is linked by 'controls'. A machine reaches a printer
through usesprinter and defaultprinter, so the band was blank on the printer
page - the one page where the link is not obvious from the asset itself. The
types are now a prop, and the printer page passes its own.
THE CHIPS WERE DEAD LINKS
Found while adopting the band more widely. It linked to /assets/<assetid>.
There is no /assets/:id route and no catch-all, so every chip in the band was
dead - and this change would have propagated that to four more pages.
AssetRelationships.vue already resolved this correctly and privately. That
function is now frontend/src/utils/assetRoute.js and both components use it, so
a third caller cannot get it wrong again. Two things it gets right that are easy
to miss: the detail routes key on pluginid, not assetid, so passing assetid
opens the wrong record; and the lookup is now case-insensitive, because
csvimport.py seeds 'Computer' and 'Machine' capitalised while this site's
database holds them lowercase - a case-sensitive match produced dead links at
whichever site seeded the other way.
The band now carries pluginid and assettypename on each chip, which the
resolver needs.
TESTS
assetRoute had none and neither did the band. Seven cover the resolver,
including the assetid-vs-pluginid confusion and the seeding-case difference.
On the backend, nothing asserted that the TARGET of a directional edge receives
it in `incoming` - only that rows carry isdirectional. That missing assertion is
this bug's exact shape, so it is now pinned: the endpoint was right all along,
and the next such report should not cost anyone a re-diagnosis of the API.
Verified on screen: the machine page shows "Controlled by G6K76CW3ESF" in the
hero, the PC page is unchanged, and a page with no such link renders no empty
band. Full suite 2162 passed, frontend 280 passed.
docs/PLUGINS.md listed sixteen of the seventeen bundled plugins. The missing one
was endpoints - the newest, and the one another site is most likely to ask
about, since it is the Fleet console and the live panels on a PC detail page.
The row records the two things that are not guessable from the name: it depends
on computers because an endpoint is telemetry ABOUT a PC asset rather than a
second record of it, and its four permissions are split deliberately. Read is
grantable to guests by the .view suffix rule; fleet, command and restart are
not, and the plugin declares no override, so the command channel cannot be
opened to anonymous callers by ticking a box.
Ingest is called out as NOT being a route here - agent reports arrive on the
generic collector route under a service token scoped collector.ingest.endpoints.
Somebody looking for the ingest endpoint in this plugin will not find one.
applications/ stays absent on purpose: it has no manifest and no plugin.py, so
per the manifest-less rule it is core frontend surface that always ships, not a
plugin a site can choose.
requirements.txt carried a hand-written header explaining why the lockfile is
universal and hashed, what each flag buys, and a warning not to hand-edit it
because waitress and tzdata were once added by hand and vanished on the next
compile.
Running `uv pip compile` to upgrade two packages wiped all of it, including the
warning about hand-edits vanishing. The header had been documenting its own
failure mode and still lost.
The explanation now lives in requirements.in, which is the file people edit and
which a compile never rewrites. The header is restored in requirements.txt for
anyone reading that file directly, but it is no longer the only copy.
Also records the --upgrade-package form for moving one package without moving
everything else, and the scan_sbom.py check to run before a release.
scan_sbom.py reported 50 advisory matches against shipped components, 16 of them
at or above HIGH. This clears all of them: the scan now reports 0 shipped
findings and the gate passes.
pyjwt 2.12.1 -> 2.13.0
idna 3.13 -> 3.19
axios 1.13.2 -> 1.20.0 (29 advisories)
postcss 8.5.6 -> 8.5.26
dompurify 3.4.11 -> 3.4.14
nanoid 3.3.11 -> 3.3.18
follow-redirects 1.15.11 -> 1.16.0
form-data 4.0.5 -> 4.0.6
Every one is an in-major bump inside a range the manifests already allowed, so
this is a lockfile refresh rather than a migration. The Python side was compiled
with --upgrade-package for each name so nothing else moved; the diff is two
version lines and their hashes.
nanoid is worth recording: it is transitive under postcss at ^3.3.11, and its
advisories are fixed on both the 3.x and 5.x branches. 3.3.18 is the fix that is
actually reachable.
The declared floors for axios and dompurify were raised to the versions in the
lockfile. They were ^1.6.0 and ^3.4.11, which still nominally admitted the
vulnerable range - npm resolves to the highest match so an install was never
going to pick one, but the manifest should state the minimum safe version rather
than rely on that.
axios 1.20 adds optional proxy-agent support, which pulls https-proxy-agent and
agent-base into the tree. Both are Node-only and neither reaches the browser
bundle; the build was checked for them. The main chunk grew 256kB to 271kB
(85.7kB to 90.5kB gzipped) from the rest of the axios delta.
The PyJWT entry in the allowlist was removed rather than kept. It stopped
matching the moment the upgrade landed, and an allowlist that accumulates
entries for advisories that no longer apply is one nobody rereads. The reasoning
is preserved as the worked example in docs/DEPENDENCY-SECURITY.md.
Verified: full suite 2161 passed, frontend 273 passed, build clean,
npm audit --omit=dev reports 0 vulnerabilities.
The rollup took the highest fix version across a component's advisories. That
is wrong whenever an advisory is patched on more than one release branch.
nanoid is the case that exposed it. Both advisories against it are fixed on two
branches - 3.3.16 and 5.1.16, 3.3.18 and 5.1.6 - and the rollup recommended
5.1.16 with a MAJOR warning. But nanoid is transitive, pulled in by postcss at
`^3.3.11`, a range no 5.x release can satisfy. The recommendation was not merely
a bigger job than necessary, it was unreachable: the actual target is 3.3.18, a
patch inside the existing range.
Per advisory the target is now the lowest published fix above the version in
use; per component it is the highest of those, since one upgrade has to satisfy
every advisory against it. Fixes on older branches are skipped rather than
offered as a downgrade - axios advisories list 0.30.x fixes that would have been
a downgrade from 1.13.2.
This also gives the MAJOR label meaning. It now fires only when no fix exists on
the current branch, rather than whenever some branch happened to be higher.
START-HERE routes by the question somebody arrived with, and 'are we exposed to
this CVE' had no entry - the SBOM was only mentioned in the Windows update doc,
which is not where anybody asking that would look.
The SBOM was generated and installed onto every server, and nothing read it.
"Are we exposed to this advisory" was still a person going and looking, which
meant it happened when somebody remembered to.
scripts/scan_sbom.py queries OSV.dev by purl - no account, no API key, no agent,
no vulnerability database to maintain on a box that cannot reach the internet.
The SBOM is already the input OSV wants, so this is a lookup against what
shipped rather than a second scan of the tree that could disagree with it.
Three decisions worth recording:
Shipped and build-only findings are both reported, but only shipped ones can
fail a build. Two thirds of the components here are build tooling the SBOM
already marks scope 'excluded', and the highest-severity match in the whole tree
today is a CRITICAL against the test runner. A gate that fails a release over
that trains everybody to bypass it, and then it catches nothing.
Severity is taken from the advisory, never derived. Where there is only a CVSS
vector the report says UNKNOWN, which is shown and never gates. A guessed score
is worse than an honest gap because somebody would act on it.
The allowlist is a record rather than a mute button: each entry carries the
reason, the accepter, the date, and what would invalidate the reasoning. The
first entry is a worked example - a PyJWT algorithm-confusion advisory that
needs both mixed algorithm families and a raw JSON JWK as the key, and this
application configures neither.
The report ends with a per-component upgrade rollup, since that is the part
somebody acts on. A target on a different major version is labelled rather than
listed beside patch bumps as though the two cost the same.
Current state: 8 shipped components have fixes available, none of them
exploitable through a path this application opens today.
Three colour faults on the PC page, each hiding the next.
ACTIVITY BADGES WERE ALL GREY. The list renderer reads a badge colour from a
ROW FIELD, and the activity panel declared none, so every severity rendered
identically. The feed's whole job is to make a critical event visible and it
was painting it the same as a logon. The route now emits severitycolor - info
grey, warning yellow, critical red - and the panel declares it.
PAUSED AND THE TRANSITIONAL STATES WERE RED. statecolor was green-or-red-or-
grey, so a paused service looked like a stopped one. They are different: a
paused service was paused deliberately, and one stuck in Start Pending is
failing to start on a retry cycle - which is exactly what raises event 7011,
the thing behind the critical counts on the Fleet page. Both are amber now.
NAMED COLOURS ALWAYS GOT WHITE TEXT. readableText only parsed hex and returned
white for anything else, but panels emit NAMES. Green, red and grey are dark
enough that white worked by luck; amber is not, so the fix above shipped as
white-on-yellow and could not be read. Names resolve to hex before the contrast
test, so this holds for any named colour anywhere in the app rather than these
two panels.
colorStyle had no test file at all. It has one now.
Every one of these was found by looking at the rendered page. None of them
would fail a test that checked markup.
Full suite 2131 passed, vitest 273 passed.
Reported from the Activity card: the header dot was red while every visible row
was info, because the critical event was older and sat further down the list.
The dot and the list appeared to contradict each other, and the only way to
find the critical was to scroll.
The dot is computed from all rows the panel received, so a red dot always means
a critical IS in the data. The chips are computed from that same array, which
means the count sits directly under the dot and the two cannot disagree by
construction: the card now reads 'Critical 2 Warning 1 Info 12', and one
click surfaces them wherever they sit in time.
Derived from the panel's EXISTING status declaration rather than a new one per
panel. A card that shows a health dot has already said what it dots on, so
Tracked Services and Tracked Processes get chips with no plugin change, and any
future list panel with a status rollup does too. Two declarations of the same
thing would eventually disagree.
Details that matter: a colour-keyed panel is labelled by meaning, so services
read Critical and OK rather than Red and Green; chips are ordered worst first
rather than alphabetically; and no selection means show everything, not hide
everything - a card that emptied when you deselected the last chip would read
as broken. Filtering to nothing says so, and gives the hidden count.
vitest 267 passed, including the exact reported case.
A site running external mode keeps its devices and check-in/out history in a
separate cmmc_usb database. There was no way to bring that in, so the choice
was to keep the second database forever or lose the history. This adds
'flask usb import-external' - dry run by default, --commit to write - which
reads through the same connection settings the routes already use and never
writes to the source.
Two things it has to reconcile, and they are why it is a command rather than a
few INSERTs.
The schemas disagree about SHAPE. External is an event log, one row per action;
self-hosted is a paired row carrying both times. Pairing walks each device in
time order, and real logs after years of use are untidy. The rule is that
nothing is invented and nothing vanishes: a drive still out becomes an open
checkout and is NOT flagged (that is just a drive somebody has), a return with
no matching issue is skipped and named, and two issues in a row leave the first
open rather than inventing a return time nobody recorded.
Identity is the other. usbdevices.serialnumber is unique and NOT NULL and the
external devices table has no serial, so device_id becomes it. Blank or
duplicated ids stop that device rather than inventing a key. external
device_owner has no column here, so it is appended to the device notes rather
than dropped.
Also adds usbcheckouts.scannedviruses, the one field with nowhere to land.
waswiped already means what external calls sanitized - selfhosted.py was
mapping it - but scanned_viruses was HARDCODED None in the log output: the API
promised the field and had nothing behind it, so an imported history would have
rendered blank. It now reads and writes properly.
Nullable, never defaulted to false. An unrecorded scan is not a failed scan,
and for removable-media records those are very different claims.
Full suite 2131 passed; seven new tests pin the pairing rules.
Reported from production: health showed '418 critical' while the ten most
recent events were all info. Both were true, which is the problem.
The health rollup counts critical events in a 48h window; the activity feed
shows the fifteen most recent events of ANY severity. A box that logged 418
service timeouts overnight and ordinary logons since produces exactly that
pair - a page contradicting itself with no way to tell why.
Two fixes.
The rollup now names the DOMINANT event instead of totalling: 'Service timeout
x418' rather than '418 critical'. Counting by event id costs nothing (one more
GROUP BY column) and turns a number into a lead. This is the same reasoning
that made health a phrase rather than a score; it stopped one step short.
The activity feed now fetches the newest criticals separately and merges them,
and keeps up to five of them when trimming to the cap. They hold their place in
time but survive the cut, so what health is reporting always appears in the
feed that exists to explain it.
Verified against the reported shape: 20 service timeouts two hours old under 12
newer logons. Before, the feed was fifteen logons; now it carries five critical
rows and health reads 'Service timeout x20'.
Worth noting the underlying signal is real - several hundred criticals on one
endpoint in 48h is a box crash-looping or a service failing on a retry cycle.
The display was hiding it, not inventing it.
Full suite 2124 passed, naming and parity gates green.
Select endpoints, then Collect now or Restart across the selection. Restart is
gated on endpoints.restart, both are re-checked server-side.
The confirmation names the MACHINES, not just the hostnames:
Restart 2 computer(s)?
GJX9B2Z3ESF, HOST-7
This STOPS the machines these PCs serve:
DT Office
A dialog listing eight hostnames tells an operator nothing about what they are
about to halt, and that is the whole consequence of the action on a shop floor.
Where no machine is recorded it says so, because 'none listed' and 'we did not
check' are different things at 6am. Only possible now the Machine column exists.
A 409 is reported as a SKIP, not a failure. The queue permits one command in
flight per endpoint, so a bulk action across eight PCs where two are already
busy says '6 queued' and '2 skipped - a command is already in flight'. Folding
those into an error count would make a working button look broken.
Selections clear on every reload: the rows they referred to may not be on the
new page, and acting on an invisible selection is how bulk tools surprise
people.
No per-row buttons. They were built first, and a tenth column on a nowrap table
pushed itself off the edge - selecting one row costs a single click and leaves
one command path instead of two.
Full suite 2124 passed, vitest 260 passed, naming gate green.
The tiles were 6px while .card is 0.25rem, which left Fleet marginally rounder
than every other page. The global card treatment is a separate decision and has
been deferred; until it is made, this page should look like the rest of the app
rather than half-way to a mockup.
Four things the mockup showed and the first cut did not have.
FLEET
- A 'with critical events' tile, from the stability route that existed and
nothing called. Clicking it narrows to rows whose health is not ok - client
side, because the health rollup is assembled per row after pagination, so
it narrows the PAGE rather than the fleet. Worth knowing before the count
is read as fleet-wide.
- PC type and Location filters. The list route already accepted both; there
was simply no control. Both use listAll, not list: perpage is clamped to
100 server-side, so a long location list would lose its tail in a dropdown
while still looking complete.
- The machine cell shows number and name together where they differ, so
'3105 Okuma' rather than a bare number.
PC DETAIL
- A Policy card, contributed by the computers plugin. It matters because the
effective policy already narrows the command allowlist, so without it the
reason a control is unavailable on a given PC is written down nowhere.
Provenance, so it sits in the reference column, not the triage band.
Its route returns an EMPTY field list rather than the feature-gate 404 when
policies are disabled. The panel contract hides a card with no fields, so
it disappears on sites that have not enabled the feature - a 404 would have
rendered a broken card on most of them. That was flagged in the design
review as a snag; this is the fix.
- Refresh Status and Restart Computer moved out of the page header into the
triage band. They act on the MACHINE; Print Label, Edit, Retire and Back to
List act on the RECORD, and mixing them put the most consequential button
on the page - it halts whatever machine the PC serves - between printing a
sticker and editing a form. In the band it sits beside the liveness chip
and the machine it would take down.
The band takes them through an 'actions' slot rather than knowing about PC
verbs, so a measuring tool can put 'Calibrate' there without the core
component learning anything about computers.
Full suite 2124 passed, vitest 260 passed, gates green, spec at 333 paths.
Two changes to the PC page, both about what a responder meets first.
LIVE STATE FIRST. The contributed panels moved to the head of the card flow.
The grid is a two-column FLOW (column-count, with display:contents on the
column wrappers), so a card cannot be pinned to a side and only document order
decides where it lands - putting the panels first fills the left column with
endpoint status, services, processes, interfaces and activity, and leaves
identity, hardware, network and location to the right. Panels still self-order
among themselves by declared position and still self-hide when no plugin
answers for the asset type.
ONE ACTIVITY FEED replaces the Recent events card. Events and commands answer
the same question - what happened to this box - and reading them as two lists
meant interleaving by eye. GET /api/endpoints/asset/<id>/activity merges them
newest-first, capped at 15. Both are keyed by assetid, so it is two indexed
reads and a merge sort rather than a join, and it renders through the existing
ADR-010 list renderer with no bespoke frontend work. A failed command is marked
critical, because that is the row somebody needs to notice. Not collapsed by
default, unlike the card it replaces.
The core AuditLog is deliberately NOT a third source: its rows carry the action
as the entity name and the asset only inside a details string, so including it
would mean parsing prose to find out which machine a row belongs to.
Requester names are withheld from callers who cannot themselves issue commands,
via caller_holds. A read-only viewer - a guest, on a site that grants this
panel - gets the history without a roster of who restarted what, and the lookup
is skipped entirely rather than fetched and discarded. requestedby holds a user
ID, so names resolve through one batched query, not one per row.
Verified on dev telemetry: a queued command shows as 'Collect now / Pending by
admin' above the Windows events, ordered by time.
Full suite 2124 passed, vitest 260 passed, naming/contract/parity gates green,
spec regenerated at 332 paths.
Adds a shared triage band to the PC detail hero: is it alive, what stops
without it, how do I reach it.
'Serves machine' is the reason it exists. On a shop floor a PC matters because
a machine stops without it, and that fact was buried in the generic
relationships card at the bottom of the page. Both edge directions are read,
deduplicated, and filtered to assets typed 'machine' - production holds
reciprocal rows, so reading one direction shows the link on half the pages and
not the other half.
The band is a CORE component, not a PCDetail block. Five asset detail pages
already rebuild the same hero and grid independently, which is how they drifted
apart; machines, printers and network devices can drop this in rather than
growing a sixth copy. Liveness is passed in rather than fetched, because only a
plugin that collects telemetry can answer it and a site may not have one.
The status chip's verdict comes from the SERVER: /api/endpoints/<id> now
returns the same reporting/quiet/never it computes for the fleet list. The
quiet window is a site setting, so a threshold guessed in the browser would
disagree with the Fleet list about the same endpoint the moment a site changed
it.
Two cards removed from the body:
- Remote Access duplicated the band's chips. It is what a responder clicks
first, so it belongs above the fold, once.
- Status was worse than duplication. It reads the COMPUTER RECORD, while the
endpoint panel beside it reads live agent telemetry, and on an
agent-managed PC the record fields sit empty - so the page rendered 'Last
Reported: Never' a few inches from live telemetry saying otherwise. It now
renders only when that record actually holds something, which is the case
on sites running the asset-report script instead of the agent.
Verified against dev telemetry: band shows 'Reporting, 1 h ago', the machine it
serves, and its access chips; both duplicate cards are gone.
vitest 260 passed, naming/contract/parity gates green.
Two colour bugs on the same page, both invisible in light mode.
A button does not inherit color the way it inherits font, so the plain
Endpoints total rendered in the browser default black on the dark theme's card.
The coloured tiles set their own color, which is why only that one showed it.
The coloured ones were wrong too, just less obviously: dark mode redefines
--success and --warning to bright variants but leaves --success-dark and
--warning-dark tuned for a white background, so those numbers and the health
pills came out dark green and dark amber on a dark blue card.
Scoped to the stamped [data-theme="dark"] rather than a prefers-color-scheme
query, matching the note in style.css: the theme store always stamps the
attribute, and a bare media query leaks dark styling into light mode on a
dark-OS machine.
Twelve endpoints routes shared one permission, which made a reasonable policy
impossible to express. A site that wants signed-out visitors to see live status
on a PC page had to hand them the entire fleet - including every logged-in
username on the floor - because the per-asset panels and the fleet-wide list
were gated on the same string.
endpoints.fleet now gates the list and the three dashboard routes.
endpoints.view keeps the per-asset reads: status, services, processes,
interfaces, events, command history.
The name is load-bearing. guest_grantable_permissions() opens any '.view' or
'.read' permission to the guest role, so calling this one 'endpoints.fleet'
rather than 'endpoints.fleetview' is what keeps the roster off the public
surface while per-asset telemetry stays grantable. Verified both ways.
Also: the services panel emitted Start/Stop row actions to every reader,
because row_actions_for() never consulted the caller. Clicking was always
refused server-side, so nothing was exploitable, but showing a read-only viewer
controls over a shop-floor PC is the wrong signal - and it stops being cosmetic
the moment a site grants that panel to guests.
That needed a contract addition rather than a local check: caller_holds() on
shopdb.api answers whether THIS caller holds a permission without refusing the
request, which is the complement to require_permission for a surface that
serves everyone but should only offer what the caller can act on. Guests are
included, since a signed-out visitor holds whatever the guest role was granted.
Reaching into shopdb.utils.authz instead would repeat the ADR-002 violation the
contract gate caught earlier in this branch. Contract 0.24.0 -> 0.25.0.
DEPLOYMENT: endpoints.fleet is a new permission. Run 'flask seed permissions'
and grant it to the roles that should keep seeing the Fleet page. Until then
that page is admin-only, since admins bypass every check - a safe direction,
but not an automatic one.
Full suite 2124 passed, naming and contract gates green, map and spec
regenerated.
The fleet list showed hostname, user, uptime, CPU, memory and last-reported,
sorted by last-reported descending. Two of those columns have never held a
value, and that sort floats the healthiest endpoints to the top and buries the
ones somebody needs to act on.
What a shop-floor fleet view is actually for is which MACHINE stops if a PC is
dead, so that is now a column. Both edge directions are checked, deduplicated,
and constrained to assets typed 'machine' - without that last filter the column
filled up with office locations that happen to carry a controls edge.
Also here:
- Sorted WORST FIRST: never reported, then quiet, then reporting. Expressed
as a sortable SQL rank rather than a label applied after the fact, so the
order survives pagination.
- Health rollup as a PHRASE, not a score: 'eDNC stopped', '2 critical'. A
percentage does not tell an operator what to do.
- Filters for status, PC type, location and business unit; a sort parameter
(the route accepted none); and search widened from hostname alone to asset
number and the machine number a PC serves, which is how people actually
look a bay up.
- CPU and Memory columns dropped. The agent declares those fields and has
never assigned them, so they rendered '-' on every row.
- A summary strip wired to the dashboard routes, which existed and which
nothing called.
Two bugs found while building it:
The health rollup first read a 'running' boolean, but service rows carry a
STATE STRING and the vocabulary varies by source, so every service would have
reported as stopped. The panel and the rollup now share one _service_running()
helper; two copies of that rule would drift.
The summary counted quiet as total - reporting, which folded never-reported
endpoints in with quiet ones. Those are different failures - an enrolment or
key problem versus a machine that went down - so 'never' is counted separately.
Per-page batching for machines and critical-event counts: three queries per
page rather than two per row.
Full suite 2124 passed, vitest 260 passed, naming and contract gates green.
get_pagination_params reads 'perpage' and only falls back to 'per_page', and
this client has always sent 'perpage', so the interceptor's duplicate never did
anything except make every list URL carry a parameter twice.
Removing it also clears a live production problem. The endpoints list route used
to answer a 308, which is a PERMANENT redirect: the browser caches it against
the exact URL including its query string. Clients that hit the old route keep
that rewrite after the server is fixed, because the request no longer leaves the
machine - nothing deployed can reach it, and it fails as mixed content when the
cached target is http. Confirmed in the field: the page failed on load with the
default query string while a search, whose query string differs, worked in the
same session.
Changing the query string changes the cache key, so an affected client misses
the poisoned entry and reaches the corrected route without the user having to
clear site data.
Plugin views that pass per_page explicitly still work - the backend fallback is
untouched. vitest 260 passed.
An endpoint reported telemetry correctly for hours while the Fleet page showed
"No endpoints found". The PC detail page displayed its panels the whole time, so
ingest was plainly working; only the list was empty.
The list route was the one route in 331 paths registered with a trailing slash:
@endpoints_bp.route('/', methods=['GET']) -> /api/endpoints/
The browser client calls it without one, so Flask answered a 308 redirect to an
ABSOLUTE url. On dev that resolves and nobody notices. Behind the IIS subpath it
cannot: with no ProxyFix, Werkzeug builds that Location from the scheme and host
it believes it has, so it emitted http:// and dropped the /shopdb prefix. A
browser will not follow an https page to http, the request failed, and the
page's catch block rendered the empty state.
Every other route in the plugin worked throughout, because none of them carries
a trailing slash - the detail panels call /endpoints/asset/<id>/status and were
never redirected. Detail works, list empty is the signature of this bug.
Registered as '' now, so the rule is /api/endpoints. That is what
CONTRIBUTING.md's API section spells and what every other list route in the
product already did. The inventory and the generated spec are updated to match.
Also here:
- tests/test_core/test_no_redirecting_routes.py fails the build if any /api
rule is registered with a trailing slash and strict slashes. Verified by
planting the violation: it named the offending route, then passed once
reverted. It takes the shared app fixture rather than building its own
application - doing that rebinds SQLAlchemy and leaves the NEXT module
without tables, which showed up as three unrelated "no such table: settings"
failures visible only in a full run.
- The Fleet page now renders "Could not load endpoints (HTTP nnn). This is a
failed request, not an empty fleet." A list that reports a 403 and an empty
fleet identically is what made this take as long as it did.
- TRUSTED_PROXY_COUNT (default 0) applies ProxyFix when set. Off by default on
purpose: it makes X-Forwarded-* trusted, and a client can send those itself,
while the login rate limit and account lockout are keyed on the client
address. Set it to 1 on an instance behind IIS.
Full suite 2124 passed, vitest 260 passed, naming and docs gates green.
The export died with no output and no dists, exit 1, having committed nothing.
grep exits 1 when it matches NOTHING, and under set -euo pipefail that failing
pipeline takes the script with it. Once every internal reference had already
been scrubbed, the gitea grep matched nothing, and the SUCCESS case became the
abort. It printed not one line explaining itself, which is what made it cost an
afternoon: the run looked like it had worked.
All three scrub greps are guarded now. This is the third failure of this shape
in this file, after the two fetch/push tag omissions.
Also excludes a stray .env copy from the sync. A hand-made .env.bak-<date>
reached the publication tree, because rsync walks the WORKING TREE and
.gitignore does not apply. The scrub gate caught it, which is what that gate is
for, and it never reached a commit or a bundle - but a file of live credentials
should not get that far. --include '.env.example' comes first so the example
still publishes.
Both verified by planting a violation: a no-match grep under pipefail, and a
planted .env.bak file that the sync now leaves behind.
The endpoints plugin declared a section of its own, so the sidebar rendered a
one-item 'Fleet' group at the bottom, below Information, far from the PCs the
view is about. Its own docstring said it belonged just after PCs, and its
position (16, against PCs at 15) already said so too; only the section
disagreed.
Filed under assets, which puts it directly under PCs. The now-empty fleet
section is removed from NAV_SECTIONS - an unused section is an invitation to
file something else there.
The printers observedqueues migration failed with error 1071 on a 5.6 dev
server whose file format was still Antelope. The unique index it needs is
(hostname, queuename), two utf8mb4 VARCHAR(255) columns, which is 2040 bytes
against a 767-byte cap.
The compiler hook in shopdb/utils/mysql_charset.py was already emitting
ROW_FORMAT=DYNAMIC, which raises that cap to 3072. What is easy to miss is that
Antelope makes 5.6 IGNORE the row format and build a Compact table with a
warning rather than an error, so the tables look created and keep the old limit.
docs/DEPLOY.md has required innodb_large_prefix plus Barracuda for pre-5.7
servers all along; nothing pointed a developer hitting 1071 at it.
So the troubleshooting row that said 'use MySQL 8 for dev' now gives the fix,
names the silent Compact downgrade, and says such tables stay capped until they
are rebuilt with ALTER TABLE ... ROW_FORMAT=DYNAMIC.
DATABASE_URL also gains ?charset=utf8mb4, which DEPLOY.md already requires and
this environment did not have.
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.
What arrives:
endpoints plugin - a ShopDB agent reporting endpoint state, telemetry,
services and interfaces, with a command channel the agent
long-polls. Ingest stays on the generic collector route
(ADR-006); the agent key is a UI-managed single-scope
token whose revocation is an instant kill switch.
computers policies - Configuration and Business Policies (RMM-style), scope
trees validated at write time, actions checked against
the endpoints command allowlist. Feature-gated: every
route 404s while the setting is off, so enabling it can
never half-expose the surface.
asset CSV import - templates, a collector script carrying the site path, and
a validate-then-commit import. A plugin can own a virtual
column plus a post-write row hook, which is how a PC's
machine number becomes a controls link without a
hand-built relationships sheet.
agent/ - the Windows agent and its installer.
Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.
MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).
Review fixes carried on top:
- six policy GETs were jwt_required(optional=True), which authenticates a
caller without checking anything; they require computers.view now, and six
redundant jwt_required decorators are gone
- the agent installer took a default BaseUrl and CollectorKey, so a mistyped
run silently pointed an endpoint at the wrong instance; both are mandatory
and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
- a committing CSV import wrote no audit row
Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.
Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
Every job on GitHub is warning:
Node.js 20 is deprecated. The following actions target Node.js 20 but are
being forced to run on Node.js 24: actions/checkout@v4, actions/setup-node@v4
Working today because the runners shim it, and a hard failure the day that shim
goes. checkout@v5, setup-node@v5 and setup-python@v6 are the releases built for
Node 24, so the warning goes away rather than being suppressed.
Nine references across five jobs. The comment above them says why, so nobody
pins them back to the v4 generation on the assumption it is the safer number.
The org's actions policy allows GitHub-owned actions with no SHA pinning
required, so the tag change needs no allow-list update.
Two faults with one cause: docs/ is excluded from publication because the prose
becomes the wiki, and two things under docs/ are not prose.
shopdb/core/api/docs.py reads docs/openapi.json AT RUNTIME to serve /api/docs.
Excluding it meant every site installed from the published repo shipped the
ReDoc page, loaded it, and got {"error":"openapi.json not generated"}. The spec
now publishes - it is a product file that happens to live under docs/. The prose
still goes to the wiki and nowhere else, and docs/api-inventory.json stays out,
because nothing reads it at runtime.
The second fault is the reason this was worth chasing. Three gate modules skip
themselves when the file they read is absent, which is correct in a published
checkout and silently wrong on GitHub, which runs against exactly that checkout.
The modules skipped, CI reported green, and an outside branch added nine
undocumented routes and passed. A gate that stops running is worse than no gate,
because nobody looks at it again.
scripts/check-gates-ran.sh asserts each gate ran, with two lists: the ones that
must run everywhere, and the ones needing files publication deliberately leaves
behind - required here, allowed to be absent there. A `gates` job runs it in CI.
Proven in both directions: green in each checkout, and red when the file a gate
reads is removed.
Also purge instance/plugins.json from the publication tree. It was never
published - it is untracked there - but an old copy had sat in that tree for
months, protected from --delete by its own exclude, and it made the published
repo fail its own suite locally: 13 plugins enabled where the code expects 16.
A publication tree that cannot pass its own tests cannot be verified before
shipping. That is the exact trap the comment at the top of the script warns
about, which had claimed a second victim.
Also: concurrency on the workflow, so a second push supersedes the first rather
than queueing behind it holding a self-hosted runner.
A link on its own invites being pasted onto text that is already in the chat
box, and every chat client turns the selection into the link's caption. The
secret then travels as the visible message with the secure link wrapped around
it, protecting nothing.
This is not hypothetical. Six LAPS passwords went out that way in production -
machine names and passwords as the caption, the correct /s/<token>#<key> URL
underneath, doing nothing at all.
Copy message puts a whole sentence on the clipboard: what the link is, how many
opens it has, when it expires, and then the URL. A sentence has nowhere to
attach itself - pasted into an empty box it is a message, pasted over a
selection it replaces it rather than decorating it. Plain text on purpose; a
rich-text flavour is exactly what would let a secret ride along as markup.
The success card now says to delete the password from the chat box before
pasting. The existing hint explains that the key cannot be recovered, which is
true and does not stop anybody pasting the password itself.
Nothing about the link was wrong and nothing about it changed: the URL carries
the token, the key rides in the fragment the browser never sends, the plaintext
is dropped the moment it is sealed, and redeeming stays a POST so a preview
cannot spend it.
Sending somebody a label meant sending a screenshot and a list of settings to
retype, and they would get one of them wrong. Copy share link hands them a URL
that reopens the generator with the content, the label, the copy count and
every layout value that was actually changed.
The whole payload rides in the fragment. A fragment is never transmitted, so it
does not reach Flask, it is not in the IIS log, and the proxy between them never
sees it - which matters because the content is usually an asset number or a
serial, and those have no business scattered through logs. It is NOT secrecy:
the link sits in a chat message and in browser history, and anyone holding it
can read the content. Secure Send exists for the case where that matters, and
the note under the box says so.
Only values that differ from a first-time visitor's defaults are carried, so
the link stays short enough to paste into a chat box; a shared link then wins
over the recipient's own saved settings, because somebody following one is
asking to see THAT code rather than their own last one. An unknown key is
ignored rather than refused, so a link made by a newer version still opens on
an older one. The picture is never carried - a data URL runs to megabytes.
Values are coerced back to the type the field holds. Everything arrives from a
URL as a string, and a nudge of 0 plus '2' is '02'.
Proven end to end: a link made in one browser, opened in a clean context with
no saved settings, rendering the same code.
It struck the heading and left everything underneath looking untouched, which
is the opposite of what somebody ticking "this section is done" means. The
strike now covers every block under that heading, down to the next heading of
the same or higher rank - so a top-level section takes its sub-sections with
it.
That needed the rank, and the HTML importer was throwing it away: h2, h3 and h4
all arrived as a flat 'heading', so a parent heading had no children to cross
out. It was the visible half of a real loss - a converted article kept its
words and lost its shape. Headings now carry their level and render at it, and
a guide written before this reads as flat, exactly as it did.
Code blocks, tables and images inside a finished section fade but are not
struck. A line through a command makes it unreadable, and re-reading the
command is usually the reason to look back at a step already done.
The five imported guides were re-imported so they carry their structure.
Every tool on that page is usable without an account, on purpose - the routes
carry no requiresAuth and say so - and Secure Send is the one write this
product offers a guest. The sidebar entry still never appeared for them: it
named no permission, so the filter that decides what a signed-out visitor may
see had nothing to go on and dropped it.
The plugin had no read permission to name, only tools.secrets, so it gains
tools.view: reaching the page at all. The tools themselves run in the browser
and touch no data, which is why this is about whether the entry appears rather
than about what anybody can read.
The item names either right. Somebody granted only Secure Send needs the entry
that leads to it, and a manager who holds neither still does not see it.
The sidebar hides what the caller cannot reach - that is what makes it useful
to a guest, and what stops a member with a sparse role being offered links that
can only answer 403. It reads the permission off the nav item to decide.
Only four items ever declared one. Machines, PCs, Printers, Network, USB,
Knowledge Base, Notifications, Calendar, Dashboard, Map, Applications and
Reports declared nothing, so the filter had nothing to go on: granting the
guest role machines.view opened the API and left the sidebar empty, which reads
as the grant not working.
Every item now names its read permission - in the plugins, in the core
navigation endpoint, and in the frontend's fallback list, which is what renders
when the endpoint is unreachable and would otherwise disagree with it.
A sweep of the URL map found 165 GET routes that answered anybody, with no
token at all: the whole asset inventory, every machine, PC, printer and network
device, the USB checkout history and who is holding which drive, the staff
directory with its photographs, every core report. Guest access had been argued
about one feature-flag at a time while the front door stood open.
Each of those now carries the read permission it belongs to. That also makes
the guest role mean something: a site can open exactly the reads it chooses,
and nothing follows by accident.
Thirty-three endpoints stay public, and each is named with its reason rather
than left to be inferred: the app shell and its bootstrap, the settings
allowlist (the handler already filters an anonymous caller down to branding and
a few keys), images fetched by <img src> which carries no Authorization header,
what a display kiosk renders before anybody signs in, the parts and USB kiosks,
the client scripts that run as SYSTEM before a user session exists, and a
secure-send link whose unguessable token IS the authorisation.
tests/test_core/test_public_read_surface.py is the standing gate, the companion
to the mutating-route sweep that has guarded writes for months. A new public
read fails CI until somebody adds it to that list in front of a reviewer. Four
further reads need a login but no permission, because the handler decides from
who you are - your own user record, your own API tokens, the role catalogue the
grid renders - and the same test proves each still refuses an anonymous caller.
Verified not vacuous by ungating one route and watching the sweep name it.
The employees plugin gained employees.view; the directory is names, SSOs,
photographs and reporting lines, and it had no permission of any kind.
Around ninety tests asserted payload shapes against endpoints that answered
anybody. They are about what the answer says, not about who may ask, so they
are signed in now rather than the gate being loosened. The ones that assert an
anonymous caller is refused were left exactly as they were.
Only checklist items could be ticked, which answers "what have I done" but not
the question a long procedure actually raises: where was I. Somebody looks away
to do the thing the guide asked for and looks back at a wall of text.
Every step and every heading now carries a checkbox while reading. Ticking one
strikes it through and fades it rather than hiding it, because somebody
scanning back up the page has to still be able to read what they did. The
number is struck through with it, but the step's own "Step 3" label is not -
that is the address of the thing, not part of what was done.
Headings count, not just steps. A guide converted from an HTML article often
has fifty headings and no steps at all, and those are exactly the guides long
enough to lose your place in. That also means the card's "N steps to tick"
stops reading as a broken feature on a guide that is entirely tickable.
Reading only, as the checklists already were: in the editor a tick would be an
edit, and an author clicking around their own draft would quietly change what
the next reader sees as done. Progress stays per browser and per version -
shopfloor PCs share a login, so one technician's ticks must not greet the next
person on that bay, and step 4 of version 3 is not step 4 of version 2.
Whether somebody with no account may see something was being answered one
feature at a time: a securesend_allow_guests setting, an enable flag per kiosk,
and very nearly a third for guides - each with its own settings page, its own
vocabulary and its own hand-written branch inside a route. Every one of them
answered a permissions question, and this app already administers permissions
per role.
So a caller with no token is not nobody: they hold the seeded `guest` role.
require_permission consults it when there is no identity, which is the entire
mechanism. The role is seeded holding NOTHING, so an upgrade opens no door; a
site opens one by ticking a box in the same grid it uses for every other role,
and closes it the same way. Revocation is read on every request rather than
cached, because taking guest access away is what somebody does the moment they
realise what they opened, and it has to bite immediately.
The redundant @jwt_required() above require_permission is gone from 184 routes.
It was belt-and-braces before; now it is actively wrong, because it refuses a
request with no token before the permission check can have an opinion, so a
grant would appear to do nothing.
What a guest may hold is bounded. Any .view or .read permission is grantable.
A WRITE has to be blessed by whoever owns it, through the new
get_guest_grantable_permissions hook (contract 0.21.0), and Secure Send is the
only case that exists: sending a password to a vendor is the one write this
product means to offer somebody who never signs in. Everything else is refused
by the API and greyed out in the grid rather than warned about - a checkbox
that quietly makes deleting an asset anonymous is not a warning anybody reads
twice. A guest write records no creator: the trail says when and from where,
never who, and that is the reason for the bar.
GE-Enforce and Slides gained read permissions so they can be opened to guests
without handing out the manage right. Their read routes accept either through
require_any_permission, so nothing an existing manager could reach has moved.
The navigation endpoint now returns only what the caller can actually reach.
Without that a signed-out visitor gets a sidebar of the whole app where every
link bounces off the login page, which reads as a broken site rather than as a
closed door. A nav item may name several permissions; holding any one is
enough, which is what GE-Enforce and Slides need now that a viewer and a
manager hold different strings.
Two things this found on the way: current_user is a LocalProxy, so
`current_user is None` is False for a request with nobody behind it - harmless
under a required JWT, an AttributeError the moment verification became
optional. And an image pasted into a guide before its first save was never
attached to it, so it was visible to editors and invisible to every reader;
saving now attaches it, and `flask guides fix-images` repairs what is stored.
The guides guest setting written earlier today is gone. Granting guides.view to
the guest role replaces it, which is the point.
Guides are written on one site and needed on all of them, and the sites that
matter are air-gapped from wherever the writing happens. Until now a procedure
was stranded in the database it was typed into: the HTML importer could rebuild
one from its original article, but anything authored in the editor had nowhere
to go.
flask guides export writes one .guide.json per guide, with every image the
blocks reference carried inside it as base64. One file on purpose - a bundle
that needs its images copied alongside it is a bundle that will eventually
arrive without them. flask guides import reads them back, a directory at a
time, and refuses anything that is not a bundle before it writes a row rather
than half-importing a stray .json sitting in the same folder.
What travels is the content: title, summary, tags, blocks, images. What does
not is ids, revision history, authorship and publication state, because those
belong to whichever database issued them - an imported guide starts as a draft
at revision 1, owned by whoever ran the import. The SLUG does travel: it is the
address somebody pastes into a work order, and the same guide should answer to
the same name on every site.
--replace updates the guide already carrying that slug instead of creating a
second one, which is how a corrected guide is re-shipped. It drops the stored
images first: the incoming bundle is the whole truth for that guide, and a
guide re-shipped monthly would otherwise accumulate orphans nothing points at.
Bundles refuse a version they are too old to read rather than importing the
parts they recognise. A newer site's guide is not a guide this one can be
trusted to render.
Proven by round-tripping a five-guide export back into the same site, including
one carrying two PNG screenshots, and reading it in a browser.
A site adopting this app arrives holding procedures already written: styled
HTML pages that get mailed around, opened off a share, and edited by whoever
still has the file. Those pages are exactly what a guide is for, and retyping
one into the editor is how a good procedure stays on a share forever.
flask guides import-html reads the page and produces blocks. It recognises the
vocabulary those pages actually use - a step box with its own number, an alert
in one of three colours, a code block with a copy button, a checkbox list, a
table with a header row - and maps each to the block that already renders it,
lifting embedded images into guideimages so a document stops carrying its
screenshots as base64. Page furniture goes: the header, the footer, and the
table of contents a guide builds for itself from its own headings.
The conversion is lossy on purpose. Colour, fonts, width and nested layout are
dropped, because storing blocks rather than markup is what makes dark mode,
printing and tickable steps work at all. So an import is always a DRAFT and the
default is a dry run that reports what it would produce and writes nothing.
Blocks go through the same validator the API uses: an import must not be able
to store something the editor could not have produced.
The parser is tolerant, because real articles are not well-formed. An unclosed
<p> is normal HTML and the browser closes it at the next block element; without
the same rule one unclosed tag swallows the rest of the article into a single
paragraph, which is what it did before the test caught it.
Proven end to end against four real articles, including one carrying two PNG
screenshots, read back in a browser.
POST /api/reports/email has always taken any {columns, rows} from any caller,
but the Email report button existed on the toner and warranty pages only - not
because those reports are special, but because they happen to be their own
routed pages and the reports page had no button at all. It is now beside Export
CSV for every report the page can build a table from, which is all of them:
plugin reports through their declared columns, core reports through the flat
list they return, and the asset inventory summary by flattening its three
tables into one with the grouping named in a column. Presentation-only fields -
a status colour, a link id - are left out; a column of hex codes is worse than
no column. The shapes live in a module with tests rather than in the view.
The button is now hidden without reports.export on all three pages. Offering
somebody a button that can only answer 403 is worse than not offering it.
Separately, on the USB kiosk a long drive name pushed Take or Return out
through the side of its card, leaving a drive nobody could press. The name is a
flex item and so defaulted to min-width:auto, keeping its full intrinsic width;
it now yields, wraps mid-word for the serial-style names that carry no spaces,
and clamps its description to two lines so one long name cannot make every card
in the row tall. The locker line is never clipped - it is how somebody finds
the drive.
A cabinet of thirty drives was one long column to read through. The free list
now filters from a search box and lays out in as many columns as the panel is
wide - four on a wall display, two on a tablet, one on a phone - while the
steps that hold a single control stay narrow, because stretching one input
across a 24 inch screen reads as a broken page.
A drive is found by its id, by what is written on it, or by where it lives. The
words can arrive in any order, and "locker" is part of what is searched, so
"locker engineer" finds the engineering locker - that is how people say it out
loud. Filtering is client-side: every free drive is already on the screen, and a
round trip per keystroke on a wall panel would be worse than no filter at all.
The matcher lives in its own module with tests rather than inside the view.
The box takes focus as the picker opens so somebody can simply start typing,
and it is cleared when the kiosk resets - the next person at the cabinet must
not inherit a filter and conclude there are no drives left.
Also: both this view and the guides blocks coloured muted text with a
`--textmuted` variable this app has never defined. The browser dropped the
declaration, so every hint, subtitle and locker line was rendering at full
strength. The variable is `--text-light`.
The reports page ran a hardcoded switch over five core report ids and fell
through to `default: console.error('Unknown report')`, and its results panel was
a matching chain of per-id templates ending in a `JSON.stringify`. So a plugin
could register a report, watch it appear in the catalogue, and get a wall of
JSON when it was opened - which is where the three printedparts reports have
been since they were added.
Any report that declares an endpoint is now fetched from it, and any report
returning {columns, rows} renders as a table. That shape is already the
convention on the server side, so a plugin needs no core code at all to have a
working report. The per-id templates stay for the core reports whose payloads
are not tabular.
Export CSV goes through the api client rather than window.open. A new window
carries no Authorization header, so every report behind a real jwt_required -
which is every plugin one - answered the export button with a 401. The core
reports are jwt-optional, which is the only reason the old way ever appeared to
work. It also followed /api/reports/<id>, which does not exist for a plugin
report, instead of the endpoint the report itself came from.
A report that fails now says so instead of leaving an empty panel, and the page
offers the time window the endpoints already accepted - printedparts-consumption
was permanently stuck on its 30-day default because nothing could send one.
Verified in a browser against both plugin report families and the core reports.
CMMC removable-media tracking is an audit obligation, and answering "who had
this drive, and when" meant paging through the check-in/out list by hand.
GET /api/usb/reports/audit returns the events - a take and a return are separate
rows, because a count cannot answer the question an audit actually asks - with
summary figures riding along for the people who only want the shape of it, and
a CSV for the people who want it in a spreadsheet.
The window is named rather than a day count: today, last month, six months, a
year, all time. Nobody should have to work out that a year is 365.
Both directory modes answer in one voice. The external path reads the
cmmc_usb checkinoutlog; self-hosted expands one usbcheckouts row into its two
ends, so a returned drive is distinguishable from one still out, and the window
filter runs in SQL against either end of the loan - a drive taken before the
window and returned inside it is a check-in the window has to show. Rows come
back from the mode-specific half; the CSV, the summary and the response shape
are built once in the route.
Ordering is total in both modes. Timestamp alone ties for two events in the same
second, and the same query then returns them in a different order each run,
which reads as the log being unstable.
The kiosk was a single column of full-width rows top-aligned on the page, so a
cabinet of a dozen drives meant scrolling a wall panel, and the page itself
scrolled - which takes the "Your drives" heading off screen and leaves somebody
looking at a list with no idea whose drives they are. The lists now sit in a
grid that fills the width and scroll INSIDE themselves against a bounded
height, with the panel centred as one block the way the parts kiosk beside it
already is. Two kiosks on the same floor that sit differently on their screens
read as two different products.
Drive ids sorted as text, which puts USB-10 ahead of USB-3. Compared with a
numeric collator so the list runs in the order the labels on the drives read.
fetchUser() existed and was called from nowhere. The signed-in user - including
the permission list every v-if in the app reads - was written once at login and
cached in localStorage, and nothing ever refreshed it. So an admin granted
somebody guides.create, the person was told they had it, and the New guide
button stayed hidden until they happened to log out and back in. Nothing on
screen explained why, and the natural conclusion is that the permission system
is broken.
Called on app boot, so a permission change lands on the next page load.
Reproduced before and after: button hidden, permission granted mid-session,
plain reload, button there.
Only a 401 logs out now. The old handler logged out on ANY failure, so an API
restart or a network blip would have thrown people out of sessions they still
held - a cached user is stale, not invalid.
The kiosk page tells an administrator to enable it in Settings under USB. That
place did not exist - the setting was reachable only through the API or a shell,
which is the same trap the backups plugin shipped with. A settings page and a
catalogue card, so the instruction leads somewhere.
Machines, PCs, printers and measuring tools all carry Print Label / Edit /
Retire / Back to List in a page header. A network device had no header at all:
Edit was tucked inside the hero title row, and Print Label and Delete sat in a
bar at the very bottom of the page, past every card. Managing one meant looking
somewhere different from every other asset type.
They move to the standard header. The bottom bar is removed rather than left as
a second copy, and the two delete handlers collapse into one - the route is a
soft delete, so it is called Retire here as it is everywhere else, and the
confirm says what is kept.
Four things people kept tripping over.
The move controls were a literal ^ and v character. They are arrows now.
Sections can be dragged. The handle is a grip in the block bar, not the block
itself: making the whole container draggable swallows text selection inside its
own textarea, so you could no longer select a word to bold it.
The type palette lived once, at the bottom of the entire guide. Adding a step in
the middle meant adding it at the end and walking it up one press at a time.
There is now an insert point in every gap - and at the end, so there is one way
to add a section rather than two.
Linking to another guide or a KB article meant knowing its slug or id by heart.
A picker searches both by title and writes the markdown link at the cursor. It
offers published guides only - never send a reader to a draft - and it stores an
in-app path rather than an absolute URL, so the link survives the site being
reached by another hostname or mounted under a subpath. The knowledge base is a
separate plugin that a site may not have installed; its absence costs nothing
but its results.
The section headings were derived from position ranges: 10-29 became Assets,
30-49 became Information. A number is for ORDER; it was never a statement about
meaning, and using it as one meant a plugin landed in whichever bucket its
number happened to fall in.
That is how USB Devices - an asset - came to sit under Information beside
GE-Enforce and Tech Tools, and 3D Parts with it. Nobody chose that. It also
meant two plugins picking the same number silently reordered each other:
Warranties and Guides both held 8, the tie broke on plugin load order, and
Guides rendered under Warranties rather than under Slides where it was asked to
go. Warranties moves to 9.
Each nav item now names its section and the renderer groups by what was
declared. An item with no section stays in the unlabelled group at the top,
which is the safe default: it appears, it is simply not filed.
Assets gains USB Devices and 3D Parts. GE-Enforce and Tech Tools move to a new
Fleet section - they are neither assets nor reference material, and calling them
Information was only ever an artefact of their position numbers.
Also: sorting used `position || 99`, and 0 is falsy in JavaScript, so Dashboard -
the only item that uses 0 - sorted to the end of its own group.
The kiosk route lives outside AppLayout, so nothing linked to it - a full-screen
page with no way in from the app is a page nobody finds. It joins the Displays
section beside Parts Kiosk, TV Slideshow and the Shopfloor Dashboard, gated on
the route existing so a build without the USB plugin does not advertise it.
The people who actually take USB drives are standing at the cabinet, and most
of them have no account here. Check-out and check-in were gated behind a login
that was not doing much anyway: both modes already attribute the action to a
scanned BADGE rather than to the signed-in user, so the login was gating the
screen, not the record.
/usb-kiosk is a full-screen touch page modelled on the parts kiosk, off unless
a site sets usb_kiosk_enabled - every route 404s until then, rather than 403,
so a disabled kiosk does not advertise that there is a door here. It works
against the self-hosted tables or an existing cmmc_usb database.
THE LIST IS THE TRADE. These drives carry no barcode, so unlike the parts kiosk
there is nothing to present and the screen has to offer a list - which exposes
more than a scan-first design, where you had to already hold a drive to learn
anything. So the list is not anonymous: an SSO comes first and must resolve to
a real person before any drive is named, which is the same bar taking a drive
already had to clear. It is a POST so the badge stays out of proxy access logs.
The main screen shows the caller's own drives, what is free, and a COUNT of
everything else; holders of other people's drives appear only behind a second
deliberate tap, so nobody reads the whole cabinet off a wall panel in passing.
A badge that resolves to no employee is REFUSED, unlike the signed-in route
which creates the user row. A checkout nobody can trace to a person defeats the
tracking, and at a kiosk there is no operator standing there to vouch for it.
Returning somebody else's drive asks first and then records the badge that was
SCANNED, so the log says who actually handed it back. That is the honest record
and it needs no change to a schema this app does not own. Refusing a mismatch
outright is a setting but not the default: a drive left behind by somebody on
leave would otherwise be unreturnable at the kiosk, which just moves the problem
to a person with a login.
The authz sweep caught these writes, twice - once for the two mutations and
again for the two list endpoints. Both are exempted with the reasoning written
out beside the parts-kiosk entry rather than silenced. The list endpoints are
POSTs that change nothing.
Verified by driving the real page: SSO on the keypad, tap Take, database moves.
.env is committed git-crypt encrypted. Without the key the checked out
file is the raw encrypted blob, and python-dotenv fails reading it as
UTF-8 with "invalid start byte". The migrations job already provides
DATABASE_URL, SECRET_KEY and JWT_SECRET_KEY through env:, so removing
the file is enough. Previously masked by the job dying earlier on a
missing mysql client.
A service container is reachable on 127.0.0.1 only when the job runs
directly on the runner host, which is how GitHub-hosted runners work and
how the old host-executor runner worked. Under a container-mode runner
the job is itself a container on the services network, published ports
land on the runner host rather than the job's own localhost, and the
service answers to its name.
Fixes: pymysql.err.OperationalError (2003, "Can't connect to MySQL
server on '127.0.0.1' ([Errno 111] Connection refused)").
Three failures, all from the workflow inheriting whatever the runner
happened to have rather than declaring what it needs.
- backend ran pytest without installing it. pytest is in
requirements-dev.txt, which nothing installed, so the step died with
"No module named pytest".
- Both Python jobs used the runner's system interpreter. The old host
executor had a matching Python by accident; a container image has
3.10, where alembic additionally requires tomli. requirements.txt is
hash-pinned and compiled on 3.14, so tomli has no hash and pip refuses
the whole install under --require-hashes. 3.14 matches the Dockerfile,
the README and the .github workflow.
- The utf8mb4 step shelled out to the mysql client, which is not in
every runner image. pymysql is already a runtime dependency.
The comment explaining the concatenation fix spelled the example hostname out
in full, which tripped the publication scrub - the same trap the surrounding
comment warns about, since the scrub greps this file for the very strings the
patterns are assembled from. Reworded with a neutral example.
Re-verified after the reword by planting a violation and watching the gate fail.
Guides was shipped without an entry in PLUGINS.md or START-HERE, so a site
adopting the product had no way to find out it exists.
BACKUP-KINDS.md described a host's badge as coming from the most recently
confirmed kind. It has not worked that way since the fleet report was reworked:
the freshest revision per (host, kind) is taken, and then the STALEST of those
kinds decides, because reporting the freshest would let a healthy NTLARS backup
mask a part-marker capture that stopped a month ago. Stalest among kinds that
can be JUDGED - a kind never confirmed is unmeasurable, not stale, and treating
it as the worst value made a host with a working backup show no badge at all.
The backups row also still advertised UDC and CMM kinds that do not exist.
api-inventory, openapi.json and PROJECT-MAP regenerated.
Unlike its twenty siblings it ran a bare add_column plus create_index, so
`flask plugin upgrade-all` failed with "duplicate column name: gagelabtag" on
any database whose tables were built by create_all and stamped nowhere - and a
chain stops at its first error, so 0004 never applied either.
The index is guarded by COLUMN SET rather than by name: create_all names indexes
by SQLAlchemy's convention and the migration by this repo's, so a name-keyed
guard would have added a second index over the same column instead of skipping.
The test that should have caught it was restricted to plugins whose whole chain
is a single baseline, on the reasoning that a longer chain meets a second
problem - an ALTER whose column is already present - that guarding a create_table
does not fix. True, but the conclusion was wrong: the answer is to guard the
ALTERs too, not to stop testing the chains that have them. It now covers every
post-cutover chain, and was confirmed to fail against the unguarded migration
before the guard went in.
The NTLARS/DNC registry carries an FTP password in cleartext and it was stored
verbatim. That put it in the backuprevisions table, on the revision panel which
renders values, and in the download route which hands back a ready-to-import
.reg - so anyone holding backups.view could read it off a web page.
Redaction happens at PARSE, so the secret never reaches the database at all.
Redacting on display would have left it at rest and one query away.
What is deliberately NOT done is dropping the value silently. This codec already
refuses to discard an unparseable value, precisely because a backup that looks
complete and restores a machine with a setting missing is the worst outcome. A
redacted value keeps its name, is listed in the projection, and renders as a
COMMENT naming what must be set by hand. Emitting a placeholder as the password
would produce a machine that looks restored and cannot connect, and the
technician would go debugging the network.
scripts/redact_backup_secrets.py scrubs revisions recorded before this, dry run
by default. It leaves contenthash alone: the hash identifies the configuration a
bay reported, so rewriting it would make every bay's next post look like a
change and insert a duplicate revision across the fleet.
Separately, sourcehostname is now normalized at ingest. samesource() maps '' to
IS NULL for the query while the insert stored the raw '', so rows written by a
client posting an empty hostname could never match their own next post: every
post inserted, the table grew without bound, and the history filled with
identical entries nobody could read. Whitespace did the same thing more quietly.
Every domain catches broadly and returns no rows so one broken table cannot take
the whole search box down. That part is right. On its own, though, it makes a
DOWN domain indistinguishable from an EMPTY one: the technician sees no results,
is offered a spelling correction, and concludes the record does not exist. It
does - the search just could not look.
Failed domains are now named in the response and stated above the results.
The suggestion is deliberately still offered on a degraded run. Suppressing it
was tried first and is worse: a chronically unreachable domain, an external HR
directory say, would silently disable spelling help for every search for ever
with nothing saying why. Naming the failure is the honest fix; withholding a
second useful signal is not.
Guides search also moves to the published revision. It correctly limited itself
to published GUIDES, but matched and displayed title/summary/tags/blocksjson
from the Guide row - which is the WORKING copy. A published guide with
unpublished edits matched on wording that only exists in the draft, and the
reader opened it to find the word absent.
The audit log is append-only and nothing ever removed a row, so it grows for as
long as a site runs. This site reached tens of thousands of pages, which is past
the point of being useful: nobody pages through that to find anything.
Retention is OPT-IN and the default keeps everything. An upgrade that silently
started deleting a site's audit history would be indefensible, so
audit_retentiondays defaults to 0 and nothing is removed until somebody sets a
number. The Site & Facility card shows the log's size and exactly how many
entries a purge would delete BEFORE anything is pressed.
Deleting happens in committed batches. That is not an optimisation: one DELETE
over a million rows builds a single enormous transaction and holds locks across
the table while it runs, which on this server means blocking the collectors
posting inventory. The API caps its batch count so the request cannot outlive a
proxy timeout, and a run that stops early reports how much is left rather than
claiming it finished - a purge reporting success over 900k remaining rows would
be the same silent half-failure this codebase keeps getting bitten by.
`flask audit stats` and `flask audit purge` for the server, the latter reporting
by default and deleting only with --apply, with no batch cap because it is not
behind a proxy and a purge that stops halfway leaves the operator guessing.
audit.purge is a separate permission from audit.view. Reading the log and
destroying it are different acts, and plenty of people need only the first.
Chunking selects a page of ids and deletes those rather than using
DELETE ... LIMIT, which is MySQL-only while the suite runs on SQLite.
The patterns are assembled from fragments so the published script does not
contain the strings it greps for. They used the apostrophe-escape form
('a'\''b'), which EMBEDS a literal apostrophe, instead of the concatenation form
('a''b'), which joins them. Every pattern therefore carried a stray quote in the
middle and could never match a line of real code, so the check passed everything
from the day it was written.
Verified the only way that means anything: plant a violation, watch it fail,
revert. Fixed, it immediately found 25 real hits.
Among them, in seven places, the default ServiceNow URLs - one organisation's QA
tenant, shipped with the integration ENABLED. Any site that had not configured
ServiceNow still rendered ticket links, pointing its technicians at a tenant
they cannot reach; an adopting site would have been pointed into somebody else's
instance entirely. There is no sensible cross-site default for "which ServiceNow
is yours", so the URLs ship blank and the integration ships off, matching the
Zabbix block that got this right.
A named SaaS tenant is as site-specific as a hostname, so it joins the patterns.
Colocated .spec.js files are excluded the same way tests/ already is: sample data
naming a facility is not a shipped literal.
Images were a broken icon for every user. The stored URL was bound straight
to <img src>, and a plain image request carries no Authorization header while
this app keeps its JWT in one, so every fetch was a 401. They are now fetched
through the API client, which also fixes the stored '/api/...' path under a
subpath mount where the API is not at /api at all. A failed image says so in
words rather than rendering as a broken icon, and the editor's own thumbnails
go through the same path - an author needs to see the screenshot they pasted.
Revision and image routes gated on their parent guide. Both were gated on
guides.view alone, and revision ids are sequential, so being able to read one
guide was enough to walk the ids and read every unpublished draft on the site -
the opposite of what docs/GUIDES.md promises. An image in a PUBLISHED guide
still serves to any reader; one that only appears in a draft does not.
The guide list, the tag chips and search now match the PUBLISHED text rather
than the working copy. Matching the working copy handed readers a hit on
wording that is not in the page when they open it.
The slug no longer follows the title. It was regenerated on every save that
carried a title and the editor autosaves every three seconds, so fixing a typo
in a heading silently moved the URL and broke exactly the pasted links the
stored slug exists to protect.
publish() now aborts when the pre-publish save failed. It ran regardless, so
the author saw "Published version N" while readers got the previous revision.
Link rendering: text is escaped before formatting, so a captured URL arrives
carrying entities and was escaped a second time - every multi-parameter
ServiceNow and SharePoint link opened with &amp; in it. Emphasis could also
reach inside an href, and '//host' passed as an in-app link when the browser
reads it as off-site.
Tests are now HTTP-level. The draft-invisibility rule lived entirely in routes
that no test executed, which is how three of them shipped serving drafts.
Five of the seven asset types had no way to take a record out of ShopDB from
the interface. Network devices and 3D printed parts had a button; machines,
PCs, printers and measuring tools did not, so a scrapped tool stayed in every
list and report until somebody went to the API.
The endpoints were already there and already correct. Every one of these
DELETEs sets asset.isactive = False and nothing more, so the record leaves the
lists while its calibration history, backups and relationships stay exactly
where they were.
Called "Retire" rather than "Delete" for that reason: the two existing buttons
say Delete, which reads as destruction of a record that is in fact kept and can
be restored by setting isactive back. The confirm says so too, because the
question somebody needs answered before clicking is whether the history goes
with it.
The button is guarded on the record having loaded, on the two pages whose
action row renders before the fetch resolves - measuring tools showed Retire
beside a hidden Edit while the tool was still loading, which would have called
delete with an undefined id.
printersApi had no delete method at all; added.
USB is deliberately left out. It retires through a status column in the
external CMMC database rather than isactive, has its own /retire endpoint, and
needs its own decision about what retiring means there.
Every detail page read "West Jefferson ShopDB - Machine". Two tabs on two
different machines were indistinguishable, a bookmark said nothing about what
it pointed at, and a link shared into Teams previewed as the section rather
than the thing.
The router cannot fix this alone: afterEach runs before the view has fetched
anything, so it only ever knows the KIND of page. So a detail view names its
record once the record arrives, and both writers go through
composables/pageTitle.js.
THE ORDERING IS THE WHOLE PROBLEM. The router awaits the site settings to build
the "<facility> ShopDB" prefix, so on a cold load it finishes AFTER a view that
fetched quickly and would overwrite the good title with the generic one. The
route's fullPath is claimed before that await, and the generic write stands
down when a record title already belongs to the route being landed on. The
reverse case is handled too: a view that names its record BEFORE the facility
name arrives composed against the placeholder prefix, so the late resolve
rewrites it rather than leaving "ShopDB - CSF16" on a site called West
Jefferson. Both are pinned by tests, because both fail only under timing.
Covers machines, PCs, printers, network devices, subnets, measuring tools, USB
devices, printed parts, applications, employees, knowledge base articles and
guides. Printers reuse the page's own displayTitle, which already discards the
literal name "NONE" a discovered queue can carry, so the tab and the heading
agree.
A record with no usable name, or one that failed to load, keeps the section
name: "Machine" beats "Machine 4192" for somebody scanning a row of tabs.
Long names are cut at a word boundary - knowledge base articles are named with
a whole sentence of description, which a browser otherwise truncates mid-word.
Reports gained meta.title, which the router already supported and nothing used.
That also fixes "Pc Relationships", which the name-derived spelling produced.
Two faults the guides plugin shipped with, both caught by gates that were
already there.
deploy/site-profile-universal.json never listed `guides`. That file is the
profile the released Windows installer is built from, so the wizard would not
have offered Guides at all - and worse, `flask plugin prune-schema` drops the
tables of any plugin absent from the profile at provisioning (ADR-014). A site
installing the next release would have had its guides table dropped.
plugins/guides/models/guide.py imported shopdb.core.models.base directly.
Plugins reach core only through shopdb.api or shopdb.plugins.base (ADR-001),
and shopdb.api already re-exports BaseModel, so this is the same symbol by the
supported path.
The suite was red on main because of these two; it passes again.
Global search has per-domain toggles (search_<type>_enabled), enforced
server-side and edited from Settings. Guides answered the search box but had no
toggle, so it was the one domain nobody could turn off - and the omission would
only have surfaced when somebody went looking for the switch and could not find
it.
Missing key still means enabled, so nothing changes until somebody decides
otherwise.
The repo already enforces that every served route appears in
docs/api-inventory.json, and the guides endpoints did not - so the parity test
was failing. They are now listed with auth, params and a working example each,
beside knowledgebase because the two get read for the same reason.
docs/GUIDES.md covers what the inventory cannot: the block schema and why it is
blocks rather than HTML, the id rule that makes a tick rememberable, what the
server refuses and why validation happens at the door, the working-copy /
revision / published-pointer split that stops a live procedure changing under
somebody following it, and a worked example of creating and publishing a guide
from a script.
It also states plainly what does not exist yet - no importer, orphaned images
never pruned - and what converting existing documents will actually be like:
structured HTML and Markdown map onto block types nearly one for one, while a
plain-text runbook lands as paragraphs and needs marking up by hand afterwards.
Generated ids have to stay stable across re-imports, or every re-import resets
readers' progress.
Guides moves to position 8 in the sidebar, beside Slides, rather than down among
the reference lists in Information. A guide is something somebody goes and does
something with, not a catalogue they look things up in.
A guide was invisible to global search. Core search has a hardcoded block per
domain rather than a hook, so a new plugin stays unreachable until someone adds
one - which is the same shape as the gap this test file already documents for
USB devices and printed parts: records with a list page, labels in the world,
and no way to find them from the search box.
Guides are searched by title, summary, tags AND body. The body matters more here
than elsewhere: people look for a procedure by something written halfway down it,
not by a title they never read. PUBLISHED ONLY - a draft is deliberately
invisible to readers, and search is exactly where that rule leaks if nobody
writes it down. The test proves it with a draft sharing a tag and a body word
with a published guide, so a missing filter would show up as a ride-along.
PROOFING catches what a spellchecker cannot, because none of these are spelling:
a smart quote in a command so it will not run, a checklist item with no text so
a step is silently missing, a heading left with nothing under it after a
reorganise, an image with no alt text, a TODO published by accident. Warnings,
never blocks - somebody publishing at 14:40 with one empty bullet should be
told, not stopped, and a procedure that exists imperfectly beats one nobody
wrote.
The smart-character issue reports once per block with a one-click fix, rather
than once per offending character: an opening and a closing quote are one
problem, and listing both makes the panel look like it is nagging.
The character table itself is written as escapes. The style gate forbids
non-ASCII in source, and a file whose job is detecting smart quotes cannot
contain any - which the gate caught, correctly.
Seven changes, all aimed at the same thing: think about the procedure, not
about blocks.
Live preview beside the editor, rendered with the SAME component readers get,
so what an author sees is what will be published - a separate preview
implementation would drift within a month.
Formatting buttons wrap the selection, so nobody needs to know the markup is
Markdown. New blocks and pasted images land AFTER the block being worked on
rather than at the bottom of the document, which removes the scroll-back that
made adding a screenshot mid-guide annoying.
Enter in a list makes the next item and focuses it, the way every other list in
every other tool behaves; backspace on an already-empty item removes it and
steps back up. Typing a ten-item checklist no longer needs the mouse.
Templates, because the blank page is the real barrier to somebody writing a
procedure down at all. The shapes are lifted from the KB articles the team
already writes, so guides written here read like the ones written before them.
Autosave on a three-second debounce with an honest status line, safe precisely
because the server dedups on content - an autosave that changed nothing writes
no version. Leaving with unsaved work now asks first.
CLIPBOARD: the code block no longer writes to the clipboard. A page that
programmatically copies a shell command is the exact shape of a ClickFix attack,
where malware plants a command in your clipboard and talks you into pasting it
into Run - and uBlock Origin blocked ours, correctly, because it was writing
PowerShell on click. The button now SELECTS the code and says to press Ctrl+C:
same convenience, decision back with the person, and nothing for security
tooling to object to.
AUTOCORRECT is on for prose and off for code and table cells. A code block that
autocorrects produces a guide telling somebody to run a command that does not
work - smart quotes replacing a straight one, an autocapitalised cmdlet - and it
still LOOKS right to the author. The person at the bay gets the syntax error.
Printing a dark-mode page lays a full page of toner down for the background and
knocks the text out of it in white. On a shopfloor printer that is a cartridge
for a handful of guides, and the result is harder to read than the plain
version. Somebody would have found this the expensive way.
The theme tokens are redefined for print only, in an unscoped block because
overriding them has to reach :root. Every guide element already renders through
those variables rather than literal colours, so this is a dozen lines instead of
a rewrite - the same discipline that made dark mode work without dark-mode code.
Backgrounds are dropped rather than printed grey: a callout is identified by its
rule and its heading, which cost nothing. Checkboxes print EMPTY even where they
are ticked on screen, because a printed guide is a worksheet somebody carries to
the bay, and one person's progress is not part of the procedure. The footer
gains the print date beside the version, so a stale copy pinned to a noticeboard
can be spotted.
Also corrects the model and migration found by running this against a real
database rather than SQLite: BaseModel contributes createddate, modifieddate and
isactive to every table and the migration never created them, so the first read
failed on unknown columns; the model had also declared duplicate timestamps on
top, now removed with the API field names kept stable. And the slug is 191
characters, not 300 - it is UNIQUE-indexed, MySQL caps an index key at 767
bytes, and utf8mb4 is 4 bytes per character.
The knowledgebase plugin is a link registry - a description, a URL and some
keywords pointing at a document held somewhere else. That leaves the documents
themselves as HTML files on a share: no search, no history, no way to know
whether the copy pinned to a noticeboard is current, and no way for a technician
to tick off where they are.
A guide is stored as an ordered list of typed blocks rather than a blob of
HTML, because three of the four things this feature exists for depend on it. A
tickable step needs a stable id to be remembered; dark mode needs semantic
markup rather than pasted colours; and print needs to know where a step begins
so it does not split one across a page. Free HTML gives none of that and lets
pasted styling fight the theme.
Inline text is a restricted Markdown subset - bold, italic, inline code, links,
line breaks - escaped BEFORE formatting, so nothing an author pasted can reach
the DOM as markup. No fonts, sizes or colours: emphasis is semantic and the
theme decides how it looks. That is what makes a guide written next year match
one written today, and dark mode work without an author thinking about it.
DRAFTS AND VERSIONS. The guide row is the working copy; every save that changes
something writes an immutable revision; and a separate pointer says which
revision READERS get. Without that pointer, editing a live procedure changes it
under a technician who is halfway through following it. Publishing promotes an
existing revision rather than snapshotting, so a publish with unsaved edits
cannot ship a version nobody reviewed. Restoring writes a new revision with old
content, so the record of what people were told in between survives.
Saves dedup on a content hash covering title, summary, tags and blocks, so an
autosave that changed nothing writes no version - otherwise the history fills
with identical entries and hides the three edits that mattered.
Progress lives in the browser, not the database: shopfloor PCs share a login, so
per-user ticks would show one technician's progress to the next person on that
bay, which reads as "already done". Keyed by guide and version, because step 4
of version 3 is not step 4 of version 2.
Five permissions, and the split between edit and publish is the point: a
technician can write a procedure up while somebody else decides it is fit for
the floor. create-without-edit works too - an author may edit their own guide
until it is published, after which other people are following it.
CI caught this: judging a host by its stalest kind treated a never-confirmed
revision (lastseenat NULL) as the worst case, so it won. The verdict for a NULL
confirmation is tri-state None, which renders as NO BADGE - so a host with a
healthy ntlars backup and one backfilled row showed nothing at all. Hiding a
working backup is the one thing that badge must never do.
A NULL confirmation is not stale, it is unmeasurable. The stalest kind is now
chosen among kinds that can actually be judged, which is the rule staleness.py
already applies: a chain becomes measurable the first time its PC posts and says
nothing until then. If every kind is unmeasurable the host still reports which
kind it has, with a None verdict and no badge.
The test that caught it pinned the old newest-wins behaviour, which part markers
made wrong; it now pins stalest-measurable-wins and keeps its NULL clause, which
was right all along and is what found the bug.
Also worth recording how it was missed: the change was verified against
test_geenforce_reporting.py and test_backups.py, and this assertion lives in
test_geenforce_publish_gate.py, which was never run.
The publication scrub gate rejects internal references, and a real site
hostname in a test is still one in the published repo. The rewrite does not
care which server it is.
Snapshots written before the client was fixed stored "W:\backup\partmarker\..."
as their location. That mapping exists only inside the SYSTEM logon session,
only during an enforcement cycle, and only on the bay that wrote it, so the path
recorded against a marker resolved to nothing for the person most likely to open
it.
The script rewrites backuprevisions.sharepath and nothing else - no revision is
created, deleted, re-filed or re-hashed, and no file on the share moves, since
the bytes are already where the UNC will point. Dry run by default.
It matches on the drive letter alone and leaves everything else untouched,
including paths that are already UNC and paths on another drive. This repairs a
specific known mistake rather than normalising whatever it can reach, which is
the version of this script that would eventually rewrite something it should
not have.
rewrite() is kept pure so the decision of WHICH paths to touch is testable
without an app context; the tests cover the already-UNC, other-drive, empty,
case-difference and trailing-separator cases.
The plugin shipped five settings and no page for any of them. The core settings
catalog is a hardcoded list a plugin cannot extend, so backups_shareroot and the
collection interval were reachable only through the API or a flask shell on the
server - which is not a configurable setting in any useful sense, and it is how
today went: the setting simply was not there to set.
The page is contributed the ADR-010 way rather than by editing core - a card
from get_settings_cards plus a route in the plugin's own routes.js, both of
which disappear when the plugin is disabled. Filed under Machines, next to
Machine Types, because that is where the assets these back up live.
It also states the two things the settings do not say for themselves: that
health is judged on when a backup was last CONFIRMED rather than when it last
changed, and that retention prunes database revisions only - files on the share
stay, so enabling it on a share-backed kind leaves orphaned snapshot
directories.
Snapshot paths are now shown as full UNC text in both places: a Snapshot row on
the Marker Contents card, and the existing path on the history page, which
gains a copy button. Deliberately NOT a link - browsers refuse to navigate from
an https page to file://, so a clickable path looks actionable and silently
does nothing, which is worse than no link. Copy, then paste into Explorer.
The path comes off the revision, never composed from the configured share root:
a snapshot written before the layout changed still lives where it was put.
Until part markers, a PC reported exactly one backup kind and "newest revision
wins" was the same thing as the truth. A marker bay reports two: ntlars for the
machine's DNC settings, partmarker for the controller's patterns.
With newest-wins, the daily ntlars post keeps the badge green while the marker
backup has been failing for a month. The failure shows only on that marker's
asset page, which is not where anyone looks to find out something stopped - and
"is this still being checked" is the one question this badge exists to answer.
So a host is green only when everything it backs up is current, and the badge
names the kind that is not. Freshest revision per kind first, because
lastseenat only ever moves on the newest revision of a chain; judging every row
would report a healthy, frequently changing config as permanently stale.
Known consequence, and the right direction to err: a device retired from a PC
leaves its last revision behind, so that host stays stale until the revision is
pruned. It complains about a backup that genuinely is not running rather than
staying quiet about one.
The dashboard card needed no change - it already lists every stale chain
independently, so a stale partmarker chain was never hidden there.
A 'share' kind keeps its bytes on the file share and gives ShopDB only a hash
and a UNC path. That is right for an opaque vendor file, but it means the app
cannot answer the question a tech actually arrives with - what does this part
marker hold, and which pattern disappeared last month. Reading the share from
the app server is not an option: the download routes refuse to mount it on
purpose, because a permissions slip there becomes an unexplained empty panel.
So the collector now accepts an optional manifest alongside the hash: the
index, not the content. Names, sizes, per-file hashes, and the marking system
each pattern was authored for. Advisory throughout - a client that sends none
behaves exactly as before, and dedup still keys on contenthash.
The marker asset gains a Marker Contents card built from it: counts by type,
the controller family, and a diff against the previous snapshot in the same
source chain. Removed files are named rather than counted, because "1 removed"
only tells someone to go and look. Files edited in place are named too - the
file set can be identical while the content hash moved, and reporting no
change there would contradict the reason the revision exists.
The manifest is deliberately NOT covered by contenthash, which attests the
bytes on the share. A client that starts describing itself therefore posts an
unchanged hash, so a no-op backfills a manifest the row does not yet have -
otherwise the card would appear only on markers that had recently changed, and
stay hidden on every healthy one. The same no-op refreshes sharepath, so a
snapshot that moved on the share stops being pointed at where it used to be.
Manifests are validated at ingest rather than at render: a malformed one would
otherwise become the latest revision and fail every view of the asset page
until the content next changed, outliving the client bug that caused it.
sharedir() for the marker kind is overridden to the layout the site actually
uses, so what ShopDB displays and validates matches what the collector writes.
Canonical names chosen by the site: Dell Inc., HP, Zebra. The rule is the same
in each case - keep the spelling that holds the entries - it just lands on a
different form per company. Dell Inc. carries 310 references and 33 models
against 1 apiece for Dell and DellInc.; HP carries 51 and 24 while HP Inc. has
none; Zebra carries 7 and 2 while Zebra Technologies has none.
So HP Inc. and Zebra Technologies are gone from the shipped catalog. Both were
empty rows that only ever offered a second way to spell a vendor already
present, and neither exists in this site's database at all - they would have
arrived as duplicates at a NEW site and nowhere else.
Also removed UnscopedLiveVendor, which is a test artifact. It reached the
catalog because export_catalog.py reads the live development database, so a row
left behind by a lab run was published as reference data and has shipped to
every site seeded since.
Vendors 51 to 48. Verified nothing referenced any of the three before removing
them, and the catalog still has no duplicates by any spelling.
The seed catalog is what a new site inherits, so its duplicates propagate.
Removed the four space-stripped twins it carried: vendor 'DellInc.', model
'OptiPlex7080', and operating systems 'MicrosoftWindows10Enterprise' and
'Windows10', each of which had a correctly spaced original already present.
'OptiPlex5000' was NOT a duplicate - no spaced twin existed - so it was
renamed to 'OptiPlex 5000' rather than dropped, matching every sibling in the
list. Its vendor moved to 'Dell Inc.', which this site treats as the one
canonical Dell.
Vendors 53 to 51, models 128 to 127, operating systems 14 to 12. Verified
afterwards: no exact duplicate rows, no near-duplicates, no orphaned
model-to-vendor or supply-to-model reference.
The legacy data carried space-stripped twins: 'DellInc.' beside 'Dell Inc.',
'Windows10' beside 'Windows 10', 'OptiPlex7080' beside 'OptiPlex 7080'. They
rode into the shipped seed catalog, so every site that ran `seed catalog`
inherited them, and they stay invisible until somebody opens a vendor picker
and finds the same company listed twice.
Not a code fault. The collector strips only leading and trailing whitespace,
matches with ilike, and treats the OS list as lookup-only, so it can neither
have created these nor be creating more. They arrived with the classic-ASP
import, which holds the same three pairs today.
`flask catalog dedupe` merges only whitespace-and-punctuation twins, because
those are one name typed two ways. 'Dell' and 'Dell Inc.' normalise apart and
are left alone: that is a naming choice and only a person can make it. The
surviving row is the most referenced, tie-broken toward the spaced spelling,
which is the one somebody pressed the space bar for.
`flask catalog merge` makes that judgment call explicitly, and refuses to
invent a target that does not exist, so a typo cannot move rows into a new
vendor. `flask catalog rename` handles the case neither can reach: a stripped
name with NO twin, where there is nothing to merge into.
All three report and change nothing unless --apply is passed, and dedupe is
idempotent so it is safe in a deploy script.
References are discovered from the LIVE schema rather than listed in code.
There are 15 foreign keys into these three tables across 7 tables, and
machines.controllervendorid and printerdrivers.modelnumberid are not ones
anybody would enumerate by hand. Every reference is repointed before the row
is deleted, so nothing is orphaned, and a plugin adding a table is covered
without editing this command.
Running `python scripts/export_catalog.py` on a deployed site died with
1045 "Access denied for user 'root'@'localhost'", naming a user that site
does not have.
Nothing loads .env for a plain script run. wsgi.py calls load_dotenv() for the
served app and the flask CLI loads .env for itself, so both reach the real
database; a bare `python scripts/...` gets neither. DATABASE_URL then stays
unset, FLASK_ENV defaults to development, and config.py falls back to a
hardcoded root@localhost URL.
The read-only failure above is the harmless version. Six of these seven
scripts WRITE - backfill, reclassify, retype, collapse, refile, import - and
on any box where those fallback root credentials happen to resolve, they would
have silently rewritten the wrong database with no error at all. That is also
why this survived: the dev .env points at root, so the fallback matched what
the developer expected and nothing ever looked wrong here.
All seven now load .env from the repo root, resolved from __file__ rather than
the working directory, since these are run by hand during a deployment and
rarely from the repo root.
tests/test_maintenance_scripts.py pins it for any future script: every script
that calls create_app must call load_dotenv, before create_app rather than
after it, and with an explicit path rather than a cwd-relative search.
PLUGINS.md listed 14 of the 15 bundled plugins. `backups` had no row: the
per-asset configuration backups the collector takes off shopfloor machines,
kind-pluggable, with view and download as separate permissions because a .reg
is a restorable artifact. Added, with the pointer that separates it from
BACKUP-RESTORE.md, which is the database and a different subject.
PLUGIN-LAB-PRINTEDPARTS said the finished implementation lives on branch
feat/printedparts-plugin. That branch was deleted in the v0.8.0 cleanup; only
the 17 lab-stage tags survived, and they are enough. Says so now, and warns
that the tags are a record of contract 0.11-0.13 rather than today's
conventions. Its manifest example carried core_version >=0.12.0 against a
shipped >=0.16.0.
The printer-assignment proposal still said the client script is not built. It
shipped and released, across v0.11.3 and v0.12.0, and the collector half that
contradicted the assigned-vs-observed split was retired in adc7ef4.
PROJECT-MAP regenerated for the test count.
`flask plugin new` wrote views to frontend/src/views/<name>/ and a route file
to frontend/src/router/routes/<name>.js. Every one of the 16 bundled plugins
keeps its frontend in plugins/<name>/frontend/ instead, staged into
.plugins-staged by stage-frontend.mjs and aggregated into routes.gen.js.
It was not visibly broken, which is why it lasted: the router still globs
./routes/*.js during the Phase 4 dual-location transition, so a scaffolded
plugin's pages loaded. What a new plugin author silently lost was ADR-013 lean
builds. Pruning works off the staged tree, so a frontend in the core tree
cannot be excluded from a site build, and deleting the plugin strands its
views behind.
The templates for the correct layout already existed; only the output paths
were wrong. Also fixed the imports they carry: core imports now use the @
alias (@/api, @/components/...), which resolves both before and after staging,
where the previous ../../api was correct only in the old location. Sibling
views stay relative and travel with the route file.
frontend_dir is gone from scaffold_plugin. There is no second output tree to
point at, and a plugin developed in its own repo now gets the same frontend as
a bundled one rather than being skipped.
Proven end to end: scaffolded a throwaway plugin, confirmed it stages, builds
real chunks, and that SITE_PLUGINS=network prunes it, which is what the old
layout could not do.
A NOTICE result linked to /notifications, so clicking the row search had
already identified dropped the reader on the list page to find it again by
hand. The id was in the payload the whole time; only the URL ignored it.
`/notifications/<id>` is the existing notification-detail route.
The subnet searcher had the same defect two functions away: an IP search
resolves which subnet the address falls inside, then linked to /network, the
hub. It now links to /networks/<id>, the subnet detail route.
Every other search result type already carried its id. These were the only
two that did not.
The printer feature rests on one rule: a collector report is OBSERVED state and
never ASSIGNED state. If a drifted bay's own state can become what it is told to
install, enforcement stops meaning anything, because a bay that installed the
wrong printer makes itself right simply by reporting it.
_sync_printer_links broke that rule. It ran on every collector PC push and, when
the payload carried defaultprinter or printers, wrote defaultprinter and
connectedto rows from the report - defaultprinter being one of the two types
_reconcile_edges is meant to own alone, from authenticated routes a human calls.
It was recorded as dormant. It was not. The collector schema ADVERTISED both
keys and the example client in COLLECTOR-INTEGRATION.md set
$payload['defaultprinter'], so the documentation was recruiting callers for it -
while the same document, under "Observed is not assigned", promised that nothing
on this path writes a usesprinter or defaultprinter row. The document
contradicted itself and the code followed the wrong half. Nothing in the fleet
armed it, because the first-party reporter never sent those fields, but any
third-party client written from the docs would have.
Retired the function, its _sync_one helper and the now-unused
PRINTER_LINK_ORIGIN label, and removed both keys from the schema.
IGNORED, NOT REJECTED. Unknown payload keys are not validated, so a client
written against the old docs keeps pushing successfully and only its printer
keys stop doing anything. The response warns, naming the keys and pointing at
POST /api/collector/printers for reporting queues and seed-from-observed for
adopting them. Dropping them in silence would leave an integrator believing
assignments were still being written, which is a worse failure than the one
being fixed.
The docs were the larger half of this. The schema table rows are gone, the
"PC -> printer relationship sync" section is replaced by a note saying what
happens now to anyone still sending the keys, and the example client no longer
enumerates Win32_Printer at all - it points at Report-PrintersToShopDB.ps1,
which posts to the observed endpoint and posts NOTHING on a failed enumeration,
since an empty list would erase the host's rows.
test_collector_printers.py pinned the old behaviour, so it now pins the
opposite: the keys create no relationships, the push still succeeds and says
why, a payload without them stays quiet, a human-made default survives a report
naming a different printer, and the schema no longer offers the keys.
Reported: the warranty page cannot be searched by machine number.
It was there to be found. _related_machine puts the machine's number in the
payload, and it does so for a stated reason - a shopfloor PC is bought,
warranted and replaced as a PC, but it is FOUND by the machine it drives, and
nobody walks the floor looking for an asset number. The table renders it as a
chip. The filter looked at vendor, service level, service tag and asset number,
and never at the machine. So the one identifier the feature was designed around
was the one thing the search box could not match.
This is how a field list goes wrong: a column was added to the table and not to
the filter, and nothing connects the two. Moved the field list into
warrantySearch.js so a test can see it, following controllingPc.js and
labelPages.js. Machine name is searched as well - somebody who knows the bay by
"the Makino" should not have to look up its number to find its warranty.
The specs also pin the case that made the old inline version fragile: a covered
asset whose machine is null, which is every printer and any PC nobody has linked
yet, must not throw. And a missing field must not match: String(undefined) is
'undefined', so a naive coercion makes a search for "und" hit every row with an
empty vendor.
Not changed, worth knowing: the covered asset's NAME is still not searched, only
its number. That looks like the same omission, but it is a wider change than the
report - the placeholder has always said "asset" without saying which half - so
it is left for a deliberate decision rather than folded in here.
Reported: `flask plugin upgrade-all` failed the tools plugin with "table
securesendsecrets already exists".
Every post-cutover baseline emitted a plain op.create_table. That is correct
only while a table's existence and its plugin's Alembic stamp can never
disagree, and they can. Anything that builds tables from the models -
db.create_all(), which `flask db-utils create-all` calls - creates them and
stamps nothing. A run that creates a table and then fails before its stamp
commits leaves the same split, and re-running is the obvious thing to try. In
both cases upgrade-all meets existing tables with an unstamped chain and the
create is fatal.
Nothing caught it because the migration test DROPS the post-cutover tables
db.create_all() built before running upgrade-all. That arranges the one state
where an unguarded create works, and never the state the report came from. The
new test is the same sequence with the drop removed for the plugin under test.
create_table_if_missing and create_index_if_missing are in alembic_template.py
next to create_plugin_tables, which had the idempotency rule right and turns out
to be called by nobody - its docstring claimed every 0001 used it. Fixed the
docstring rather than rewiring the baselines onto it, because the reason they
hand-write their DDL still stands: the helper's per-plugin MetaData cannot
resolve a foreign key to a core table at CreateTable-compile time.
INDEXES ARE MATCHED ON COLUMNS, NOT NAME, and that distinction matters. The two
creators disagree about names - db.create_all() takes SQLAlchemy's default
convention (ix_securesendsecrets_token) and a baseline spells it the way this
repo's naming rules require (ixsecuresendsecretstoken). A name-keyed guard would
have called the index missing and added a second one over the same column, which
MySQL permits with a warning, so every affected site would have quietly grown
duplicates. The test asserts no table ends up with two indexes over the same
columns.
Applied to all nine table-creating plugin migrations, not just the tools
baseline that was reported: the four later migrations that create tables have
the same exposure, and geenforce, measuringtools, printedparts and backups are
the same defect waiting for a different site.
WHAT THIS DOES NOT FIX. It makes a baseline idempotent, not a whole chain. A
longer chain can still meet an ALTER whose column is already present -
printedparts adding gagelabtag to a table create_all built at head is the case
that showed up while testing. Guarding every op one at a time is the wrong
answer to that; a database built from the models is already AT head and its
chains want stamping rather than replaying, and there is no `flask plugin stamp`
today. Left alone deliberately rather than half-built, and the new test is
scoped to single-baseline plugins so it does not pretend otherwise.
Reported: moving machine 3016 on the map and saving answered "this
relationship already exists" and refused the save.
The map had nothing to do with it. Every save of the machine form re-POSTed the
controlling-PC link, and the server was right to refuse it - the row was already
active, and assetrelationships is unique on (source, target, type). Any edit to
a machine that has a controlling PC hit this; a position change is just the one
somebody happened to be making.
The form asked the wrong question. It used "do I know the existing link's id" as
a stand-in for "is there an existing link", and read that id from
rel.assetrelationshipid - a key the API has never emitted. to_dict serialises
column names and the column is relationshipid, so the read was undefined on
every load while controllingPcId was populated from the same relationship. The
two answers disagreed, and the disagreement was invisible: undefined is not an
error, it is just a falsy id.
That single conflation caused both failures. Saving anything re-created the
link, and CLEARING the controlling PC did nothing at all, because "no selection
and no known id" reads as "nothing to do" - so removing a machine's controlling
PC silently kept it. Only the first of those was reported, and the second is the
one somebody would have trusted.
Fixed by making the LOADED PC the authority on whether a link exists, with the
relationship id used only to delete a row. If that id ever goes missing again
the worst case is a link that cannot be removed, never a duplicate create.
The decision moved to controllingPc.js as a pure function so it could be pinned
by tests, following labelPages.js. Unchanged is now explicitly no-op rather than
delete-then-create: re-linking a machine to the same PC opens a window where it
has no controlling PC at all, and resolve_asset_position walks that edge, so a
failure between the two calls would drop the machine's inherited position along
with its link. The specs also cover a select handing back '42' against an API
that gave 42, which would make an untouched link look like a swap and reach the
same duplicate POST by another route.
Not the printer assignment, which was the first guess: PrinterAssignmentPicker
makes no relationship calls, and its save is wrapped in a catch that only logs,
so it cannot surface an error from this form at all.
Secure Send shipped on 21 August and nothing outside the code caught up. The
roadmap still listed it under Deferred as "tabled", and gave as the reason that
the tools plugin is browser-only by design with get_blueprint and get_models
both empty - which stopped being true in the same day's commits that gave it a
blueprint, a table, a migration chain, a permission and a setting. A reader
following the roadmap would have concluded the feature does not exist.
The generated API docs were fine: api-inventory.json and openapi.json both
carry /api/tools/secrets, so the contract gates did their job. What was missing
was every word written for a person, and that is the half where the security
properties live.
docs/SECURE-SEND.md is the new page. The parts worth having in prose rather
than inferring from the code: that a lost secret is lost to everybody including
an administrator, which is the same property as "a compromised database yields
nothing" seen from the other side; that the 4096-character limit is the entire
server-side enforcement, because a server that cannot read what it holds cannot
have an opinion about content, so raising it turns a password tool into an
unaudited file transfer; that redeeming is a POST because preview prefetch in
chat and mail clients would otherwise spend a one-view secret before the
recipient ever sees it; and that somebody should send a link to themselves
through their own mail or chat client and confirm the key survives after the #
before a real credential goes through it, because a product that rewrites links
can strip the fragment and nothing on this side can help.
securesend_allow_guests was in the database, on a settings page, and in no
document. It is now in CONFIG.md with what it actually does - an unauthenticated
write endpoint, rate limited per address, failing closed - rather than only its
one-line description.
Also corrected along the way: site_base_url was documented as scheme + host
while the code deliberately tolerates a subpath in it, and the bundled plugin
table in PLUGINS.md had no row for tools at all.
Still missing after this: PLUGINS.md has no row for backups either.
Reported: no Tools section in Settings. The setting existed in the database and
had no page, which from a person's side is indistinguishable from the feature
not existing.
get_settings_defaults SEEDS a setting. get_settings_cards (ADR-010) is what puts
it in the rail. Declaring only the first is a silent half-feature: nothing fails,
no test notices, and the switch is unreachable.
The page states the cost where the DECISION is made rather than in a document
nobody opens - turn it on and it says a guest-created link records no creator,
so the audit trail can say when and from where but never who.
Admin-only, like every other settings page, because this one decides whether an
unauthenticated caller may write to the site.
Also caught while wiring it: the card asked for a 'shield' icon, which is not in
the settings iconMap, so it would have rendered the fallback puzzle piece - the
glyph that reads as "nobody chose this". Uses wrench, matching Tech Tools.
The row still comes from `flask plugin upgrade-all`; the card appears either
way, and without the row the page loads empty.
/tools, /tools/codes and /tools/secure-send were all behind requiresAuth. None
of them needed to be, and one of them actively contradicted a setting added an
hour earlier: securesend_allow_guests lets somebody with no account create a
link, and the page they would create it on was behind a login wall.
The label generator runs entirely in the browser - no API, no data, nothing to
protect - so gating it only ever cost a guest the use of it.
THE SERVER STILL DECIDES who may create a secret. A signed-in user needs
tools.secrets; a guest needs the site to have opened guest creation. Making the
page reachable is not the same as making the action allowed, and the two were
conflated.
The page now reads the setting on load, so a guest at a site that has NOT opened
guest creation is told to sign in instead of being handed a form that was always
going to be refused - with a line saying that receiving a link never needs an
account, because somebody who lands there confused usually has a link they
should be opening instead.
A signed-in user always sees the form, even without the permission: the server's
"you do not have permission" is a different answer from "sign in", and
collapsing them would send somebody to a login page they are already past.
Two asks, and the first is a real security decision so it is a SETTING that
defaults to the safe answer rather than a code change somebody inherits.
securesend_allow_guests, OFF by default. With it off the model is unchanged: a
login plus tools.secrets. With it on, somebody with no account - at a shared bay
terminal, a contractor - can create a link too.
The cost is named in the setting's own description and pinned by a test: a guest
secret has NO CREATOR, so the audit trail can say when and from where but never
who. That is the whole reason it is a switch a person turns on deliberately.
IT FAILS CLOSED. A missing row, a half-migrated site, an unhappy database - the
answer is no. An error must never be the thing that opens an unauthenticated
write endpoint.
A SIGNED-IN USER STILL NEEDS THE PERMISSION even when guests are allowed, which
is surprising enough that a test pins it: opening the door to guests is a
decision about the door, not about revoking what a role holds.
Guest creation is rate limited per address, same shape as the login limiter. Not
a security boundary - a determined person has more addresses - but an open
endpoint should not be an easy place to park data or fill a disk.
VIEWS ARE NOW A NUMBER, 1 to 100, instead of a four-item dropdown. How many
people need a password is the sender's business: a shift handover is not one
colleague on a call. The floor is what matters, not the ceiling - unlimited
views would just be a password on a web page.
Also folded in, from the retention question: nothing removed a secret that
EXPIRED WITHOUT BEING OPENED. The recipient goes on holiday, the link is never
clicked, and the ciphertext sat there for good - not readable, but it made
"nothing lingers" untrue and grew the table in exactly the case where the
feature failed. Any create now sweeps expired rows and a late click tidies its
own, so it stays bounded without a scheduler an air-gapped site would never add.
Two gates caught mistakes on the way: current_user is a LocalProxy and is never
None, so `current_user is None` silently took the signed-in branch for a guest;
and cache was imported straight from shopdb.extensions when it is already on the
shopdb.api contract surface.
1836 tests green.
Answering "did I set it wrong": no. The setting was right and the description
was misleading, which is what sent the last two commits chasing the wrong fix.
It said "Public base URL of this site (scheme + host), e.g.
https://shopdb.example.net". That is true only of a site mounted at the root.
Both existing consumers append the in-app route to this value and add nothing
themselves:
`${await getSiteBaseUrl()}${config.detailPath(detailId)}` asset labels
`${baseUrl}${defaultPath}` QR codes
so a site served at /shopdb must carry that here or every QR code it prints
404s. The email copy of the same setting has always documented it correctly, as
https://host/ops.
The description now says so, and the comment above the defaults records why -
including that a subpath left out produces links that fail rather than anything
that looks like a configuration error.
Existing rows keep the old text until a site reseeds; the value is unaffected.
a381174 fixed a link missing its subpath and produced one carrying it twice:
https://<host>/shopdb/shopdb/s/<token>.
site_base_url is DOCUMENTED as scheme + host, and sites really do set the full
base including the subpath. Trusting the description rather than tolerating what
is configured is the same mistake as the last one, pointed the other way.
The mount is now appended only when the configured URL does not already end with
it, which covers every combination a site can have: base with the subpath, base
without it, a trailing slash on either, and a site mounted at the root where
there is no subpath at all.
Checked all five by evaluating the expression rather than reasoning about it.
Reported from production: clicking a secret link gave an IIS 404, and the link
read /s/<token> where the site lives at /shopdb/s/<token>.
The link was assembled from window.location.origin, which is scheme and host
only. site_base_url already exists for this and says so in its own description -
"used for QR codes and any absolute link the app emits" - with getSiteBaseUrl()
as the helper the QR pages use. This reached past all of it.
Both halves were wrong, and the second is worse because it fails silently:
MOUNT PATH. A site served under /shopdb or /ops carries that in front of
every route. Leaving it off is the reported 404.
HOST. Whoever creates a secret may have reached the app by an IP, a short
hostname, or a name that resolves in one VLAN, and the recipient opens the
link from somewhere else entirely. The configured site URL is the one address
known to work for both of them.
The shape of this failure is what makes it worth spelling out: the secret was
created SUCCESSFULLY and the sender was handed a link that could never work, so
the failure landed on the recipient, the sender had no reason to doubt it, and
the secret expired unread. A 500 would have been kinder.
Falls back to the browsing origin when a site has not set site_base_url, which
is what every other absolute link here does.
BASE_URL is injected at build time, so the subpath half of this cannot show up
in dev and only appears on a subpath deployment.
The backend has been there since d4b49b9 with nothing to click. This is the
half a person actually touches.
A card under a new "Secrets & Access" heading on Tech Tools, which is where
somebody looks for a utility - the whole reason the plugin moved there.
THE CREATE PAGE encrypts before anything leaves the browser and shows the link
once. The plaintext is dropped from memory as soon as it is sealed, and the page
says plainly that the key is in the link and exists nowhere else, because
somebody who does not know that will paste the link somewhere durable.
THE READ PAGE IS PUBLIC, at /s/<token>, outside the app shell: the recipient
usually has no account here, so it carries no navigation and asks for no login.
Short path because the whole link gets pasted into a chat message.
IT NEVER REVEALS ON LOAD. It asks whether the link is still live, then says
"opening this destroys it" and waits for a click. A page that revealed
automatically would be burned by the first scanner that fetched it - which is
the same reason redeeming is a POST.
A MISSING FRAGMENT GETS ITS OWN MESSAGE. If a link rewriter strips the part
after the #, the recipient is told the link is missing its key and what to ask
for, rather than being shown a generic failure. That is the one failure mode
this design cannot recover from, so it is worth naming to the person looking at
it.
The client-side guards live in securesend.js and are advisory by nature - the
server cannot see plaintext, so its size limit is what enforces. They refuse a
pasted FILE outright, which removes the easiest wrong path without anyone
reading a rule, and they warn on something document-shaped: over the length a
credential needs, more than twenty lines, or starting with a PDF/zip/image
signature.
Web Crypto only, no dependency, AES-GCM so a tampered ciphertext fails to
decrypt rather than producing plausible rubbish. The key is base64url for the
fragment, which survives copy-paste and link rewriting better than the standard
alphabet.
VERIFIED THAT THE PAGES RENDER, headlessly, against the dev server: the card
appears under Secrets & Access and the create form draws with its counter and
options. NOT VERIFIED END TO END - no secret has been created, sent, opened or
decrypted, because that needs the dev API restarted and the table migrated.
9845867 gave it its own plugin because Tech Tools is browser-only and this needs
storage. That was the right reading of the code and the wrong answer for the
person using it: Tech Tools is where you go for a utility, and a tool nobody can
find is a tool nobody uses.
So Tech Tools now has a blueprint, a table and a migration chain where it had
none. Its docstring and manifest say why, because "every tool here runs in the
browser" was load-bearing documentation and is no longer true.
WHAT MOVED
/api/securesend -> /api/tools/secrets
securesend.create -> tools.secrets
securesend0001baseline -> tools0001securesend
PLUGIN_TABLE_OWNERS, EXPECTED_HEAD_REVISION, EXEMPT_ENDPOINTS, the universal
profile and the api-inventory surface all follow it.
THE OLD "NO BACKEND" TEST IS NARROWED, NOT DELETED. It asserted
get_blueprint() is None, which is the sort of guard that gets removed when it
becomes inconvenient. It now asserts the blueprint serves EXACTLY the three
secret routes, with their methods - so a future tool that quietly grows an
endpoint or a table has to edit this test, and therefore make the decision on
purpose rather than by drift.
Asserting methods rather than paths also pins the detail this design turns on:
redeeming is POST-ONLY. Chat and mail clients prefetch GET urls to build
previews, and a preview that could redeem would consume a one-view secret before
its recipient opened the link. Previously that was a comment; now it fails a
test.
UNINSTALLING THE OLD PLUGIN COULD NOT BE DONE WITH THE PLUGIN TOOLING. `flask
plugin uninstall securesend` loads a plugin before uninstalling it, so a plugin
whose directory has been deleted cannot be removed by the command that manages
plugins - the registry entry had to be edited by hand. Worth fixing separately;
it is the same registry-versus-code divergence the device work hit from the
other side.
MIGRATING A CHAIN THAT HAS ALREADY RUN needs the old table and its version row
dropped first, because the relocated chain has no stamp and tries to create a
table that exists. Only relevant to anyone who installed securesend during the
hour it existed - it did reach the publication repo in 4242028 - and the fix is
DROP TABLE securesendsecrets plus DROP TABLE alembic_version_securesend before
running plugin upgrade-all.
1829 tests green. Still backend only: there are no pages yet, so nothing here is
reachable by a person.
The habit this replaces is pasting a credential into Teams, where it stays until
someone deletes the conversation. Tabled on 2026-08-21 pending a decision about
scope; the decision was text only, so this is the half that survives it.
BACKEND ONLY. There is no page yet, so nothing here is reachable by a person.
TEXT, NOT DOCUMENTS, and that shapes everything. Dropping files removed the
export-control question entirely - a password is not technical data - along with
blob storage, size and MIME handling, and the need to authenticate the recipient
so a document fetch could be pinned to a name.
TWO DOORS. Creating needs a login AND securesend.create: anonymous creation
would be an untraceable way to move data out of a system that otherwise records
who did what, and the creator is the only person these rows can ever name.
Reading needs neither, because the recipient usually has no account here and the
link is handed over on a call - the unguessable token IS the authorisation,
which is why it is 128 bits of randomness and why the row is spent on first read.
REDEEMING IS A POST, and that is the detail that would have broken this in
practice. Teams, Outlook and Safe Links prefetch GET urls to build previews and
scan for malware, so a GET that redeemed would let a preview window consume a
secret before its recipient ever saw it. The GET route that remains reports only
whether a link is still live, so the read page can say "opening this will
destroy it" without spending the only view to find out.
THE SIZE LIMIT IS THE WHOLE ENFORCEMENT. The key lives in the URL fragment,
which browsers never transmit, so the server holds ciphertext it cannot read and
cannot tell a password from a page of a drawing. MAX_CIPHERTEXT is therefore not
a tuning knob: raising it quietly turns this into the document transfer it was
scoped not to be. The refusal says what the limit is FOR, because somebody
hitting it is usually trying to send a file and the answer is a different tool.
Line counts, file signatures and refusing a pasted file belong in the browser,
guide an honest person, and are bypassable by anyone willing to POST directly.
NOT ENUMERABLE. Missing, expired and spent all answer identically - telling them
apart would confirm to anyone holding a guessed token whether it ever named
anything. A spent or expired row is deleted on the way past rather than waiting
for a sweeper, because it has no reason to keep its ciphertext.
Its own plugin rather than a Tech Tool: that plugin is browser-only by design,
and this needs storage, expiry and a view counter. Not an asset extension
either, so no assetid and no asset type - the scaffold assumes one and it was
removed.
FOUND BY ITS OWN TESTS: expiryhours=0 returned 201. `int(x or DEFAULT)` treats
an explicit zero as absent, so "expire immediately" silently became "expire in a
day". A caller who sends a number is now told when it is not allowed rather than
quietly given a different one.
BEFORE ANYONE RELIES ON THIS, test that your link rewriter preserves URL
fragments: create a secret, send the link to yourself in Teams, click it from the
message, and check the key is still after the #. If Safe Links strips it the
secret is unrecoverable and the server cannot help, which is the point of the
design. The fallback is passphrase mode, worth building only if that test says so.
Three product gates caught what a new plugin has to declare, and each was right
to: the route-parity gate wanted the endpoints described in api-inventory.json,
the lean-build guard wanted the plugin listed in the universal profile the
released installer builds from, and the authz sweep wanted the public redeem
route named in EXEMPT_ENDPOINTS with a reason. That last one matters most - a
mutating route that answers an unauthenticated caller is a decision somebody has
to write down, not a default, and reading a secret is a mutation because it
spends it.
Reported as a barcode rendering 176 by 0. A bar height of zero is what does it:
the number went straight into a CSS length, height: 0in is perfectly valid, and
an image sized height: 100% of a zero-height box is zero pixels tall. The label
comes out full width with nothing on it, which reads as a rendering failure
rather than as a field doing exactly what it was told.
A cleared field was the same bug wearing a different hat. An emptied number
input holds '', not 0, so the page emitted `NaNin`; the browser dropped the
declaration and the box fell back to auto height. That one merely looked wrong
instead of vanishing, which is arguably worse for finding it.
Measurements now go through labelVars(), which will not emit NaN, an empty
length, or a zero for anything whose zero means the box stops existing. Padding,
the quiet zone and the gap still take zero, because there zero is a real answer,
and offsets stay signed. It is a separate module because this is exactly the
kind of arithmetic that needs tests rather than a preview: every case in
labelVars.spec.js is a value a number input can actually hold.
The stylesheet now carries fallbacks on its var() reads as well, but they are a
net, not the fix: a custom property set to garbage is substituted and then
dropped at computed value time, and the fallback does not apply - it only covers
a property that is absent entirely.
Two separate floors were collapsing a live cartridge into a spent one, and
removing either alone changes nothing.
EMPTY_LEVEL was 5, so everything at or below 5% was assigned daysleft 0
outright. The reasoning was that a printer at 1% is out of toner as far as
anyone standing at it is concerned. But the row exists to say how long is
left, and 2% draining a point a day has two days in it. Flooring put that
cartridge beside ones that genuinely are empty with no way back to the
difference. It is 0 now: empty means empty.
int(level / rate) then truncated the division, so anything under a full day
arrived as 0 whatever the floor did - a cartridge with six hours in it was
indistinguishable from one with nothing, and the band read it as empty.
daysleft is fractional now, rounded to two places, which is about a quarter of
an hour: finer than the estimate deserves, but it costs nothing and keeps the
ordering of two nearly-spent cartridges meaningful.
daysText reads in whatever unit carries meaning: "6 hours", "1 hour", "29 min",
"4 days". Below an hour it goes to minutes with a floor of one, because
rounding hours would land back on "empty" - the same bug one rung down.
BEHAVIOUR CHANGE worth knowing: a cartridge at 1-5% with NO history used to get
daysleft 0 from the floor and land on the order list. A rate needs two
readings; with none there is nothing to divide, and the old answer was right by
accident - it said "empty" about a level nobody had watched move. It now
reports 'no history' and shows under "No estimate yet". This reaches only
printers newly added to Zabbix; anything that got to 3% the ordinary way has
the history to forecast from.
Five existing tests pinned the old rule. They recorded a real decision, so they
are rewritten to the new one rather than deleted. One of them was passing for
the wrong reason: its series (2.0, 1.7, 1.4, 1.1) drops 0.9, under
MIN_DROP_FOR_ESTIMATE, so it never had a rate at all and only passed because
the floor short-circuited ahead of the rate check. It now uses a real 20-day
drop at a tenth of a point a day and asserts the ten days its docstring always
described.
The forecast fixture's black cartridge moves from 1% to 0% so the empty band
keeps its API-level coverage, and a magenta at 3% covers hours-left end to end.
Reported as the CODE128 not appearing in the preview. It was appearing, in 44
of the 45 preset and layout combinations swept - as a 36 pixel stub. The code
box and the caption were both allowed to grow, so a caption of any length took
half the label and then kept taking, and on 1.00 x 0.50in stock a few words of
caption left a barcode too narrow to scan. Present, useless, and indisinguishable
from a failed render. The caption now takes the room it needs and no more, and a
CODE128 holds at least 40% of the label, or all of it when stacked: past that
there are too few narrow bars left to read, and a caption that wraps or clips is
the better trade.
So that a real failure can never again look like this one, an empty code box now
says so in red in the preview. It is screen-only - the marker must never reach a
printed label.
The settings were a narrow column with the preview trailing below them, which
put the labels being adjusted off the bottom of the screen. Settings on the
left, labels on the right in a column that stays in view and scrolls on its own,
folding back to one column on a narrow window.
That scrolling column is a print hazard worth naming: an ancestor with its own
overflow clips a multi-page print to whatever happened to be on screen. Print
resets it, and the check is a six-page two-sided run coming out as six pages
with the label as the only clipping box in the chain.
Reported as "CSF04 now reports all 0%", on the printer's own page as well as
the reports. Every supply on one host reading zero at the same instant is not
what a set of cartridges does; it is what a host that stopped answering looks
like.
getsuppliesbyip turned every unreadable item into 0:
try:
level = int(float(item.get("lastvalue", 0)))
except (ValueError, TypeError):
level = 0
An item that has never collected returns lastvalue as an EMPTY STRING, so
float('') raised and the handler substituted zero. lastclock was fetched in the
same request and read by nothing at all, so a reading from three weeks ago was
presented as current.
Zero is a legitimate level meaning spent, which is what made the substitution
dangerous rather than merely wrong. Downstream it reached `level <=
EMPTY_LEVEL`, which set daysleft to 0, which put the printer on the order list:
the report asked purchasing to buy a full set of cartridges for a printer
nobody had measured. On a waste container it was worse - waste is scored as
100 - level, so an unreadable one read as 100% full, the alarm state.
_readlevel now returns (level, lastseen) with level None for no usable reading:
empty, missing or unparseable lastvalue, or a lastclock older than 24h. A
lastclock that is absent is not treated as stale - a reading that cannot be
aged is not thereby wrong, and discarding it would lose real levels. A genuine
"0" is still 0, which is the half that keeps an actually-empty cartridge on the
order list.
classifysupply gains an 'unknown' status with remaining None. It is not
critical: critical is a claim someone acts on, and it may not be made about a
supply nobody measured.
The low-supplies report skips unknown rather than listing it as needing
replacement, and counts it in summary.unknown - a printer that has gone quiet
should be visible as that, not absent and not masquerading as empty.
_annotate_supply reads .get('level') with no 0 default, which would have put
the whole bug back.
PrinterDetail already rendered `level !== null ? ... : 'N/A'`, so it starts
telling the truth as soon as the backend sends null; the UI had been written
for a case the service never produced. It gains the unknown style - muted, not
a severity colour - and a tooltip separating "no reading at all" from "no
reading since <date>", which are different faults with different fixes.
Alerts needed no change: float(None) raises and that loop already continues,
so unknown supplies stop firing low-toner emails as a consequence.
12 of the 16 new tests fail on the old code. The 4 that pass either way pin
that real zeros still behave, which is the regression this fix could cause.
Not verified against CSF04's actual Zabbix data - dev has no reachable Zabbix.
This fixes the mechanism that turns unknown into 0%; /api/printers/<id>/supplies
now distinguishes them, with null for no reading and 0 for a measured zero.
The forecast identified a cartridge by its part number alone. 'W2020X' only
tells you which toner it is once you already know, and both places that name a
supply - the Order now list and the Part column - showed nothing else, so the
report read as a list of codes.
Both now show marketingname, which is what the box says: '414X Black'. The
part number moves to the title attribute, one hover away, because it is still
the thing an order is placed against. A supply with no name on file falls back
to the number rather than showing an empty cell; marketingname is nullable.
'Copy list' is deliberately untouched. It writes the part number verbatim,
which is what purchasing pastes into an order, and a spec pins that the name
does NOT reach that text.
firstPart() now returns the whole part row rather than its number, since the
cell needs both halves. It has one caller.
Dropped the .partnumber style with its .unmapped variant: nothing renders that
class any more, and the variant was already dead - the unmapped block lists no
part number by definition.
Verified by mounting the component against a fixture. The dev box has no
Zabbix, so the page itself has no rows to look at there.
Correction only helped on the global search page, and people search from the
list they are already on. Extending it turned out to be a shape question rather
than a volume one: thirty routes take a search parameter across a dozen files,
so carrying a suggestion in each of their responses is a large change today and
one more thing every future plugin author has to remember.
So the suggestion moved to ITS OWN ENDPOINT, /api/search/suggest, which any page
can call after rendering no rows. A page that has not adopted it shows nothing,
which is exactly what it showed before - nothing breaks by omission.
The plumbing lives in useListQuery, which already owned the search term, so a
list needs three lines: take `suggestion` and `reportCount` from the composable,
call reportCount(rows.length) after a fetch, and drop SearchSuggestion into the
empty state it already has. A list that never calls reportCount never offers a
suggestion.
Wired: global search, machines, printers, PCs, network devices, measuring tools,
knowledge base, vendors. NOT wired, deliberately: the type and reference lists
(machine types, PC types, VLANs, subnets, operating systems and the rest), which
are small controlled vocabularies nobody typo-searches, and USB, whose empty
state has a different shape and wants doing by hand rather than by pattern.
TRAP FOUND WHILE WIRING IT, and left commented in every page: applying a
suggestion by calling setSearch alone updates the box and the URL and does NOT
reload the list. setSearch only syncs the URL, and the watcher that would reload
is suppressed because search.value already holds the new term - the same trap
the global search page documents in performSearch. Each page calls its own load
function directly.
The composable guards a stale answer arriving after a newer search was typed,
never offers back the word that was typed, and swallows its own errors: a search
that found nothing is already the answer, and failing to improve on it is not
worth an error in front of anyone.
The route-parity gate caught the new endpoint being served without an entry in
docs/api-inventory.json, which is hand-written on purpose; added, and the spec
regenerated from it (283 paths, 418 operations). That regeneration also carries
the openapi version to 0.12.0, left over from the release.
The back side landed last commit carrying text by default, and a back laid out
as text-only renders no code box at all. So a two-sided run came out with a
blank reverse, which reads as the code having failed rather than as a setting
doing exactly what it said. The back now defaults to carrying a code, takes its
own content per row, and chooses its own symbology - a QR on the front that
opens the record, a CODE128 on the back that the bench scanner already reads.
Text-only is refused while the back carries a code, and the contradiction is
reconciled on load, so a saved setting from the broken combination repairs
itself rather than printing another blank box of cards.
A picture can go on a label: above the code, beside it, or behind everything as
a watermark, on either side or both. The file is read in the browser and never
sent anywhere, which is what makes it usable at an air-gapped site, and it is
deliberately not saved between visits - a data URL runs to megabytes and would
take every sizing value with it when the quota gave out. The watermark says
plainly that it costs the scanner contrast.
The generator was a full-screen route with no navigation and a narrow column of
controls against the left edge. That shape exists so a print cannot put the
sidebar on label stock, which is a real hazard and the reason core's label pages
are built that way - but it is a hazard the print stylesheet can answer, and
style.css now hides the shell at print time. A tool someone spends ten minutes
in gets the navigation and the full width; the stock still gets nothing but
labels, confirmed against a print PDF rather than a preview.
Two smaller things found while checking the above. A test print paired each page
with a rendered code by loop position, while the codes are keyed to position in
the whole run, so a two-sided test card could carry the wrong code; the index
now travels with the page. And the back's dotted preview outline outspecified
the print reset, drawing a dashed rule down the edge of every second card.
The corrector landed in fa0f6d7 unwired, pending a week of zero-result logs to
say what it should be aimed at. Attaching it now instead: the guards that make
it safe here are principled rather than tuned to any particular data, so the
logs would confirm the shape of the problem without changing the answer.
ONLY ON ZERO RESULTS. A search that finds something pays for none of this - no
vocabulary build, no scoring, no extra query.
The vocabulary is harvested from columns people actually search - asset numbers
and names, vendors, models, applications, asset types - cached ten minutes and
capped per column. Long enough that someone mistyping twice does not rebuild it,
short enough that a vendor entered this morning is correctable before lunch. The
cap is there because this exists to catch a typo against the names in use, not
to mirror the database into memory to answer one query.
SUGGESTED, NEVER APPLIED. The response carries the better spelling and the
caller decides. Searching for something else on a person's behalf is how
somebody orders the wrong cartridge.
FAILS SOFT. A missing column on a lean install, or anything else, returns no
suggestion and the search answers normally. No vocabulary is worse than the
alternative; it is not fatal.
Three tests through the endpoint rather than only the unit: a misspelling gets a
suggestion, a search that succeeds gets none, and CSF18 against an existing
CSF17 returns no results AND no suggestion - the digit guard holding across the
whole stack, which is the one that would cost somebody a wrong bay.
The zero-result logging stays. It answers "what can people not find", which the
corrector does not: a typo is only one of the reasons a search comes back empty,
and the others are vocabulary nobody has entered and records that do not exist.
NOT DONE: the search page does not render the suggestion yet, so this is
invisible to a person until that Vue change lands.
Two halves of one question - how to handle a misspelled search - deliberately
kept separate, because only one of them should be turned on today.
WHAT NOBODY CAN FIND IS NOT RECORDED ANYWHERE. A search returning zero results
is the only evidence of the gap between what people look for and what is there,
and it vanished. Logged now at INFO with a stable prefix, so a week of it greps
into a list. That list is what should decide whether correction is worth wiring
in: multi-word matching and cross-field matching both changed in the last day,
so a good share of what used to fail may already be found. The failures left
over might be typos, or vocabulary nobody has entered, or records that genuinely
do not exist - and each wants a different answer.
THE CORRECTOR IS BUILT AND NOT WIRED IN. shopdb/core/services/spellfix.py, with
tests, ready to attach in about ten lines once there is evidence about what to
attach it to.
NOT SOUNDEX, which was the obvious candidate. MySQL has it and SQLite does not,
and the suite runs on SQLite while production runs MySQL - so a SOUNDEX() in a
query is either an error in every test or a production path no test executes.
It is also wrong for this data: soundex is English-name phonetics, four
characters wide, and it DISCARDS DIGITS, so CSF16 and CSF17 hash identically.
Half of what people search here is an identifier.
So: character distance in Python, same behaviour on both dialects.
THE VOCABULARY IS THE DATA. No dictionary holds Genspect, Telesis, Keyence or
wax-trace. Terms come from the columns being searched, which also means a vendor
added this morning is correctable this morning.
TWO GUARDS, and they are the point rather than a detail. Digits must match
EXACTLY: CSF16 to CSF17 is one edit and a different bay, so anything carrying
digits is either right or not correctable - while cfs16 to csf16 still works,
because the guard is on the digits and not on identifiers wholesale. And the
first character must match, since typos land mid-word far more often than on the
first key, which costs almost no recall.
Distance is Damerau-Levenshtein so a transposition costs one edit rather than
two - Keyecne for Keyence is the commonest error there is, and plain Levenshtein
pushes it past the threshold on short words. Allowed distance scales with
length. A tie returns NOTHING: two equally good candidates means there is no
answer, and offering either implies a confidence that is not there.
It suggests; it never rewrites. Silently searching for something else is how
somebody orders the wrong cartridge.
13 tests, weighted toward the refusals, because those are the cases where being
wrong costs something.
2bf1531 fixed the reported symptom and introduced the opposite one: markers that
had been landing left came right, and then sat well ABOVE where they belonged.
It was the right diagnosis and the wrong remedy. The drawing is painted with
object-fit: contain inside a fixed 390x300 preview, so it is letterboxed on one
axis and a percentage of the BOX is not a percentage of the DRAWING. That part
was true. But the fix resized the frame to shrink-wrap the image, and the
centring transform above it translates by percentages - which resolve against
that same element. Making the frame image-sized silently broke the transform's
assumption that the frame is the box. Width still matched at 390, height no
longer did, which is exactly x-right and y-wrong.
The frame fills the box again and object-fit: contain is back. Where the bars
fall is now computed instead: paintedBox works out the fraction of the box the
drawing covers and the gap before it, per axis, and the marker maths folds that
in. The transform's assumption stays true because the element it translates
never changes size.
Checked per axis rather than asserted. A tall drawing pillarboxes, so x
compresses and offsets inward - a left edge at 2 percent of the drawing lands at
13.1 percent of the box, which is where the drawing actually starts - and y is
untouched. A wide drawing does the reverse. A level shaped like the box is a
no-op, so nothing that already looked right moves. Centre maps to centre on
every shape.
NEW COUPLING, deliberately noted at both ends: PREVIEW_WIDTH and PREVIEW_HEIGHT
must stay in step with .map-preview in the styles. Resize the preview without
touching them and this drifts back.
Still not confirmed against a live instance, and this is the second attempt at
this bug, so it wants eyes on three cases: one that was landing left, one that
was landing high, and one that was right all along.
Reported from the floor: some assets showed their hover mini-map marker way off
to the left, outside the map, while the SAME asset sat correctly on the main
map. The data was right; the preview was drawing it wrong.
The preview is a fixed 390x300 box and the drawing was painted into it with
object-fit: contain, which letterboxes or pillarboxes to keep the drawing's own
aspect ratio. The marker, though, was positioned at a percentage of the BOX. So
the marker and the drawing were measured in two different coordinate spaces, and
they only agreed when the level's aspect ratio happened to be 390:300.
A level near that ratio looked perfect - 3300x2550 is 1.294 against the box's
1.300 - which is why this read as "some devices" rather than "the feature is
broken". A taller drawing pillarboxes, every marker's x shifts, and one near the
left edge lands in the grey gutter beside the map.
The image now SIZES the frame instead of filling it: the transform box
shrink-wraps the image and the image scales down to fit. A percentage of the
frame is therefore a percentage of the drawing by construction, at any aspect
ratio, and the letterboxing moves out to the preview around it where it does no
harm. dimensionsFor still converts the stored pixels to percentages - that part
was always right.
NOT visually confirmed against a live instance. The reasoning is traced and the
build is clean, but the shape of this bug is that it looks correct on any level
close to the old box ratio, so it wants a look at an asset that was landing left
and one that was already fine.
Related and NOT addressed: the wheel zoom applies translate percentages that
resolve against the element and are then multiplied by scale, so a zoomed
preview may still drift. At rest, which is the reported case, it is exact.
An article's searchable text lives in three places: its title, its keywords,
and the name of its topic. Nobody typing a search knows or cares which word
came from which, so 'CMM Community' means "the article tagged community, under
the CMM topic". Both search paths returned nothing for it.
The plugin's own listing, which is what the KB page calls, built a single
ilike('%CMM Community%'). That needs the whole phrase contiguous in ONE field,
and no article has it: 'CMM' is only in the topic's name, 'Community' only in
the keywords. Global search already split the query into words but only ever
looked at title and keywords, so a word that only the topic could satisfy
failed there too - the same bug wearing a different face.
Every word must now be found somewhere across all three fields, in any order.
_word_match gains an `extra` hook for a word that a RELATED row satisfies
rather than a column of this table, which is how the topic joins in without
colliding with the sort=='topic' join.
The retired-topic rule is unchanged and pinned on both paths: matching more
words is not a way past it.
Verified against the 235-article dev library, where all three of these
returned nothing before: 'Fieldglass Jefferson' (title word plus topic word),
'outage notification' (two keywords, not adjacent), 'compucom Jefferson'.
The same phrase-only pattern is repeated in about ten other list endpoints
(computers, machines, printers, network, printedparts, measuringtools, usb,
notifications). Left alone here - fixing them properly means promoting the
word-match helper onto the shopdb.api contract surface rather than copying it
per plugin, which is a contract bump.
Evaluated a Password-Pusher-style service as a Tech Tool and tabled it. The note
carries the decisions rather than just the idea, because those are what cost the
time: it is NOT a Tools-plugin candidate as it stands (that plugin is browser
only by design, with get_blueprint and get_models both empty), the sender must
always be authenticated, a FILE recipient should be logged in too - which costs
nothing on an intranet-only server where anyone who can resolve the link already
has an account - and zero-knowledge encryption is right for a credential but
questionable for a controlled document, where a server that cannot say what was
shared is a liability rather than a feature.
Intranet-only hosting is what makes the idea viable at all: a link mailed
outside is dead and no ciphertext reaches a third party. Pointing staff at the
public hosted instance instead would reinstate every problem this avoids.
Three things the code generator could not do, each of which had someone
working around it with a control that meant something else.
The gap between a code and its caption was the quiet zone. That is the
clear space a scanner needs around the code, so closing up a caption meant
shaving the thing that makes the code readable, and the only way to get a
tight label to scan was to push the caption further away than it looked
like it should be. Quiet zone now means the code's own margin and nothing
else; gap means the distance to the caption, and adds to the quiet zone
rather than eating into it. Each stock preset carries the gap that suits it.
Position had one pair of controls and they moved the label box on the
stock. That is the fix for a printer whose origin is off, and it is not the
fix for a code sitting in the wrong place on a label - but it was the only
lever there, so it got used for both. The label div is now the media and
holds still; everything printed sits in a content wrapper that Align,
Vertical and Content X/Y move within it. The old pair kept its behaviour
and is labelled Media X/Y.
A badge card has two sides and the generator only ever emitted one page per
card. The back can now carry the same code again, a fixed block of text,
its own per-row content, or nothing. Page order is a choice because the two
kinds of printer want opposite things: interleaved for a duplex card
printer, which takes pages two at a time, and grouped for a single-sided
one whose stack gets flipped and reloaded. A row that leaves its back
column empty still gets a blank back page - a page count that varies per
row shifts every later back onto the wrong front, which prints a box of
perfectly readable cards that are all wrong.
CSV parsing and page building moved to plugins/tools/frontend/labelPages.js
(committed in 875fde9), where the column contract and both page orders are
covered by tests rather than by looking at the preview.
A review of d60ed60 and 8b9b936 found five things. Three were introduced by
those commits.
REFUSING A NAMED DEVICE MADE A MARKER PC CLAIM ITS OPERATION. _sync_partmarker
returned [] both for "not a marker PC" and for "a marker PC that linked
nothing", and the caller reads [] as the first - so a bay whose asset-id.txt
named something unresolvable fell through to the ordinary machine link, and for
a marker PC the machine number IS the operation. It took an active link to a
record that can only have one holder while several markers share it, and the
warning said "not linked". The previous behaviour minted a twin; this traded
that for a contested operation. None now means "not a marker PC" and is the only
answer that lets the machine link run; the response normalises it away so the
API shape is unchanged.
A DORMANT CHALLENGER WAS PROMOTED BY DELETING A FILE. Recording a challenger
dormant leaves a row that the next cycle finds as `reuse` and reactivated with
no incumbent check - so a second PC took a live device by its enrollment file
becoming unreadable. Both reuse branches re-check incumbency now, which is what
_sync_machine_link always did.
AN INCUMBENT UNDER ANOTHER COLLECTOR LABEL WAS INVISIBLE. Incumbency was queried
on our own label, but on a CMM the instrument IS the reported bay, so the
incumbent's link is the machine sync's row. A second PC naming that instrument
found no incumbent and linked actively: two live holders of one instrument, each
invisible to the other. Incumbency now counts any collector-owned label. A row
made BY HAND carries none of them and is still excluded - a person's link is not
the collector's to archive.
THE NETWORK FORM LOCKED OUT THE ROWS IT NEEDED TO FIX. Asset number is disabled
while editing, correctly, but the payload is built in script so the blank was
still sent - and the new server-side guard rejects it. A device with no asset
number could not be saved at all, and the field could not be typed into. It now
unlocks only for a record that loaded without one, with a hint saying why.
"2 in 4.0d" WAS THE LABEL LYING. Replacements are counted across the whole
history window; basisdays is only how long the current cartridge has been in.
Joining them with "in" claimed two changes inside four days - the exact shape
reported as unbelievable, except here the data was right. Now "2, this one 4.0d".
Two toner dips the same review found:
A MULTI-POLL OUTAGE STILL MINTED A PHANTOM SWAP. Only single readings were
dropped, so 90, 0, 0, 90 survived and 0 -> 90 scored as a change. Dips of any
length are handled now. One bad sample stays a candidate whatever the polling
cadence, because a reading is an instant; several consecutive low ones only
count as one outage when they are close together, since days at zero is a real
empty period. That time bound also separates an outage from a swap, ordinary
consumption, and a second swap, which have the same shape in levels alone.
THE DIP FILTER ATE REAL SWAPS OF NEARLY-FULL CARTRIDGES. Recovery was tested
with an absolute difference, so 95 then 5 then 100 read as a recovery because
100 and 95 are close, and the swap evidence was deleted. Toner only falls: a
recovery comes back at or BELOW where it left, a new cartridge comes back
higher.
The review also proved by reverting each feature that the previous tests did not
pin the median burn rate or the near-full rule - both passed with the bug
restored. Verified by the same method that all four toner behaviours now fail
when reverted, and the burst assertion is tight enough to tell 0.2 from 0.88.
Reported from the floor: two printers showing 81 and 83 percent, both forecast
to run out in a fortnight. That is a rate near 5.8 percent a day on a cartridge
barely touched.
burn_rate required four readings and a 2 point drop, and checked only that some
time had passed - not how much. Supply items are commonly polled every few
minutes, so four readings can span a quarter of an hour, and a 2 point drop
across fifteen minutes extrapolates to nearly 200 percent a day. The report then
sorted that confident wrong number into "soon", next to cartridges that really
are about to run out. It did not look broken; it looked urgent.
A rate now needs two days behind it. Two days is the smallest span that survives
a printer's daily rhythm, so one heavy morning does not become the whole picture.
Below that the answer is "not enough history yet", which the report already has
a home for: band() returns None and the row lands in the "No estimate yet"
section with its reason shown, rather than competing for attention with real
urgency.
This is the third face of the same fault. A phantom replacement truncated the
run - "2 replacements in 2.3 days" was the same printer saying so - and the rate
was then fitted to whatever short stub remained. The near-full rule and the dip
filter stop the truncation; this stops a stub from producing a number at all.
Two things reported from the floor, one cause each.
"5 CHANGES IN 90 DAYS, THAT'S HARD TO BELIEVE." It was. A replacement was any
+10 rise between readings, with no check on where it landed, so two shapes that
are not swaps scored as swaps: a supply reading 0 or near-0 while it was out of
the machine and then reading normally again, and a coarse gauge ticking back up
after a reseat or a power cycle.
A swap must now also LAND near full, because that is what a new cartridge reads,
and a single dip that RECOVERS to roughly where it came from is dropped before
anything looks at it. The dip filter keys on shape rather than cause, which is
why it holds for all of them - a supply pulled out to be shaken, a door open
mid-poll, or a site whose preprocessing maps the Printer MIB's unknown
sentinels onto 0. It is NOT a Zabbix timeout: an item that does not answer
records nothing rather than writing a zero. A genuine near-empty reading before
a real swap does not recover, it jumps to full, so it survives and its swap
still counts.
find_replacements and current_run now read one predicate. When they disagreed, a
phantom rise reset the run and threw away the history the estimate needed - so
the bad count was quietly damaging the rate as well, which is why both were
wrong at once. Expect replacement counts to FALL and per-cartridge history to
lengthen.
A BURST BIASED THE RATE FOR THE LIFE OF THE CARTRIDGE. The rate was the slope
between the first and last reading of the run, and two endpoints cannot tell
"steady" from "burst then stopped". A cartridge that lost 20 percent in two days
and then barely moved for a month read as 0.83 percent/day forever after, so the
report kept promising it would run out long after printing slowed. It is now the
median of the per-interval rates: the burst is one interval among many rather
than one of two points. Rising intervals are dropped as noise; flat ones stay in
at zero, because a cartridge that did not move is real information. If every
interval is flat or rising yet the run dropped overall, it falls back to the
whole-run slope rather than reporting nothing.
Where the intervals disagree by 5x or more the rate carries a marker and an
explanation on hover. The number is still the best estimate available; the flag
stops it reading as a measurement.
The "Changed" column is "Replacements", and its cell says "2 in 90d" rather than
"2 / 90d", which was read as a date, a ratio and a version number.
A review of last week's device-identity work found these; two were writing bad
data and one was reproduced against a live server before being fixed.
AN UPDATE COULD BLANK AN ASSET NUMBER, on all six asset update paths. Create
validates it and the column is NOT NULL, but the conflict check only runs when
the value DIFFERS, and '' collides with nothing - so an empty assetnumber went
straight through to a required column. This is the likely source of the assets
found with no number: a form that loaded blank and was then saved.
MACHINEFORM COULD LOAD BLANK AND LET YOU SAVE IT. One try/catch wrapped eight
reference loads AND the machine fetch, so a single transient failure among them
- one page of listAll() timing out during a collector cycle is enough - rejected
the whole block and rendered a fully editable EDIT form with every field empty,
the error banner far below next to Save. Typing an asset number and saving then
wrote the blanks over a real machine. The record now loads in its own try, and a
failure shows the reason INSTEAD of the form: an empty edit form is
indistinguishable from a record whose fields are genuinely empty.
NAMING A DEVICE THAT DID NOT RESOLVE STILL MINTED A TWIN. Both device paths
warned "not linked" and then fell through to mint <HOST>-PARTMARKER or
<HOST>-CMM - the hostname-derived twin the resolution order exists to prevent.
The warning was true about the typo'd number and false about the twin. Naming a
device is a commitment: if the name does not resolve, or resolves to the wrong
kind of thing, link nothing and say so. Silence still means "work it out", so a
bay with no file keeps the reuse-then-mint behaviour it always had.
TWO PCS COULD BOTH HOLD ONE DEVICE, ACTIVELY, WITH NO WARNING. Verified against
a live server: report as one host, then as another naming the same marker, and
both controls rows stayed active. Neither device path had ever looked at who
else held the target - only at links whose source was THIS PC - so a replaced PC
kept its link forever and an asset-id.txt copied to a second bay claimed the
device silently. It now reuses the machine link's rule rather than inventing a
second one: an incumbent that has gone quiet past the claim window or been moved
off In Use has yielded and is archived, never deleted; a live incumbent keeps
the device and the challenger is recorded dormant.
The swap test asserted the old behaviour and now asserts the new one, split in
two: a live incumbent keeps it, and handover completes once the incumbent
yields. Two other tests were passing while their names lied - the unknown-device
one checked only that the typo'd asset was not created, not that nothing was
linked, and it passed while a twin was minted beside it.
Three fixes, all of them a page not doing what its siblings already do.
MODELS WERE NOT FILTERED BY VENDOR. PCForm, MachineForm and PrinterForm each
narrow the model list once a vendor is chosen; NetworkDeviceForm bound the whole
catalogue, so picking Palo Alto still offered every Dell and Zebra model. Same
computed as the others, including the same rule that no vendor selected shows
everything - an empty dropdown reads as "no models exist" when it means "pick a
vendor first". Audited the rest: this was the only gap. The other views holding
a modelnumberid have no vendor picker to filter against, and settings ModelsList
is where a model's vendor is ASSIGNED, where filtering would be circular.
THE HERO RAN THE LABEL INTO THE VALUE - "Asset #FW-OAV..." as one string. The
page used detail-item / label / value, which match nothing in the stylesheet, so
the two spans got no layout at all. Every other detail page uses hero-detail /
hero-detail-label / hero-detail-value, which stacks a small uppercase label above
the value. Renamed to those; no CSS added, because the styles already existed and
this page simply was not using them. It was the last page using the unstyled
names.
MACHINES LIST LINKED BY THE WRONG ID on its fallback path. `/machines/:id` keys
on machineid, the plugin extension id, and the row click and View button fell
back to `item.assetid` - which lands on whichever machine happens to carry that
number: a wrong page that looks right, which is worse than a 404. That defect has
been fixed twice before in other views (AssetRelationships, then the GE-Enforce
reports table) and BackupHistory carries a comment warning about it; this was the
fourth copy. The list endpoint always sets item.machine, so the fallback could
not actually fire here - it is removed as a latent trap rather than a live bug,
and with no machineid the cell now shows plain text rather than a link that
misleads.
Phase 3 was written as "the collector reads a registry value". It is all three
components moving together: imaging writes DeviceId, GE-Enforce resolves device
identity registry-first, and the collector reads the registry ahead of the file.
The argument for it is already in the fleet's own code. Install-FromManifest
resolves TargetMachineNumbers gating from the eDNC registry BEFORE
machine-number.txt, and says why: the file is written ONCE by startnet.cmd at
the PXE menu and is not updated on reassignment, so it goes stale.
asset-id.txt inherits that defect exactly. Nothing rewrites it when a device is
swapped, moved between bays or replaced, and phase 1 put it at the TOP of the
resolution order - so a stale value is adopted with no warning, because it still
resolves. A file only imaging writes cannot be the top of a resolution order for
something that changes during a PC's life.
Also records the consequence: phase 3 needs a reassignment path the way
Set-MachineNumber exists for the bay number, or it moves the staleness from a
file into a registry value instead of curing it.
The phase 3 section named the key and left the scope to be inferred from the
fact that deviceid is one field. Spelling it out: DeviceId is a single value,
not a key per device type, and the pc-type decides which sync consumes it - so
CMM, Keyence, Genspect, wax-trace and part markers all read the same value, and
a family declared later through subordinatedevice_<pctype> is covered the day
it is declared.
Also records what the value REPLACES per family, which differs: for a CMM it
replaces cmmid.txt as identity only, for a part marker it replaces minting from
the hostname, and for the other three it replaces nothing because they never
had an identity at all.
Phase 1 shipped without a plan around it, and the next two steps both touch a
different repository and the live boot image, so the reasoning needs to outlive
this week rather than being rediscovered from the code.
The finding that shapes the whole thing: cmmid.txt CANNOT simply be retired. It
is a PATH KEY, not just an identity - Verify-And-Heal-Staging stages
installers-post\cmm\backups\<cmmid>, 09-Setup-CMM locates the staged set by it,
and Restore-CMM restores from <BackupRoot>\<cmmid> as a same-bay operation. Only
one of its five consumers is an identity use. That one is also the one doing the
job badly, reporting a CMM's INSTRUMENT id in the machinenumber channel so the
server can resolve it back to a measuring tool at step 4 of its own fallback
chain.
So phase 2 has imaging write asset-id.txt ALONGSIDE cmmid.txt - two files, two
questions - and demotes the identity use while every path use stays exactly
where it is. cmmid.txt survives phase 3 too; it is not on any deletion list.
Phase 3 is the registry, HKLM:\SOFTWARE\GE\ShopDB\DeviceId. No new key and no
new read path: the collector already opens that key for BaseUrl and ApiToken and
Install-GEEnforce already writes it. The proposal argues for doing phase 3
BEFORE phase 2 rather than after, because then imaging writes the registry for
new bays and skips the file era entirely - the file survives only for bays
staged between phases, and the cleanup is deleting a branch instead of sweeping
a fleet.
Also recorded: the rail that must not be crossed (machinenumber is what
TargetMachineNumbers gates on, so a device id in that channel stops every
bay-gated entry from matching, silently, forever), what each phase does NOT
change, per-phase rollback, and a note that phase 1 stops new duplicates without
merging the ones a PC swap has already made - so the reconciliation backlog
should be read BEFORE imaging starts writing identities over the top of it.
A bay with an NTLARS/eDNC MachineNo registry is identified by it, and a CMM by
cmmid.txt. Everything else - Keyence, Genspect, wax-trace, part markers, and
whatever a site declares next - had no stable identity at all, and the two
mechanisms that stood in for one both key off the PC: reuse looks for a prior
link from THIS PC asset, and minting builds `<PC number>-<SUFFIX>` where a PC's
number is its hostname.
Both survive a re-image. NEITHER survives a PC SWAP. A new hostname is a new PC
asset with no prior link and a predicted number that has never existed, so the
same physical device gets a second record while the first keeps its config and
its backup history under a dead PC's name. That is how 43 legacy MT-#### tools
ended up shadowed by minted twins, three records deep in places. The metrology
path learned this and gained an explicit id file; the part-marker path was
modelled on the metrology path as it stood BEFORE that fix, so it inherited the
defect - and its own docstring said so, describing minting "the same way it
already mints a CMM".
C:\Enrollment\asset-id.txt is now that identity for both, and for anything
declared later through subordinatedevice_<pctype>. It holds one line, the
device's assetnumber, and deliberately does NOT record what kind of device it
is: the pc-type already does, so a new device type needs no new file and no
client change. Resolution puts it first, ahead of everything PC-derived.
NOT machine-number.txt. machinenumber answers "which bay is this" and is what
GE-Enforce TargetMachineNumbers gates on, so naming a device there would
silently stop every bay-gated manifest entry from matching. A part marker still
files partof the operation from machinenumber; asset-id.txt changes which marker
the PC controls, not which operation that marker belongs to.
The wire field is `deviceid` rather than `assetid`, because assetid is already
this contract's RESPONSE field for a PC's integer primary key - the two would
have sat side by side meaning different things. measuringtoolid, shipped in
0.12.0, is accepted as an alias and the client still reads the older file, so a
bay staged in the last day keeps reporting; deviceid wins when both arrive.
Two guards, shared by both device families: a value that resolves to nothing
warns instead of minting a phantom, and a value that resolves to the WRONG KIND
of asset is refused with the asset named, so a machine number pasted into the
file cannot be filed under a device label the collector also owns.
Row creation is now one helper instead of a copy per path, because the named and
minted branches both need get-or-create on the extension row and the control
link.
1744 tests green, including a new file that pins the swap case both ways - with
the file one device, without it two. Five of its tests failed first time because
the fixtures built bare assets with no extension row and the type guard refused
them, which is the guard working.
VERIFIED ON WINDOWS 11 (build 26200), five cases: the new file alone; both files
present, new winning; the legacy file alone; neither, sending no field; and a
padded value with a trailing line.
The server has accepted `measuringtoolid` since the adoption work landed, and it
is the FIRST entry in the resolution order precisely because it is the identity
that survives a PC swap. Nothing ever sent it. The reporter read the eDNC
registry, cmmid.txt, machine-number.txt and pc-type.txt, and its own comment
said metrology bays have no per-bay id and therefore send nothing - so a Keyence
or Genspect bay fell through all four steps to minting, which the server's own
docstring calls the last resort.
Minting derives the asset number from the HOSTNAME, so a permanent instrument
inherits the identity of whichever PC drove it that week: replace the PC and
either the number lies or a second tool appears for the same physical unit. That
is how 43 legacy MT-#### tools ended up shadowed by minted twins. The half that
prevents it was built, tested and undeliverable.
The reporter now reads C:\Enrollment\measuringtool-id.txt and sends it when
present. Its own file, NOT machine-number.txt: machinenumber answers "which bay
is this" and is what GE-Enforce TargetMachineNumbers gates on, so naming a tool
there would silently stop every bay-gated manifest entry from matching.
The paste-ready reporter in COLLECTOR-INTEGRATION.md is a second implementation
of the same payload, so it gets the same resolver rather than being left to
drift. The field was also missing from the payload table and from the classic
api.asp mapping, and there was no prose anywhere describing how a tool is
resolved - added, including why minting is last and what the two guards refuse.
VERIFIED ON WINDOWS 11 (build 26200), four cases: a named instrument is read and
sent; no file sends no field and exits 0; a whitespace-only file behaves as
absent rather than sending an empty string; and a padded value with a second
line yields the first line trimmed.
Printers become a property of the bay, and the container stops throwing away the
site's own files every time it is updated.
Printer assignment is now end to end. The assignment belongs to the MACHINE and
reaches whichever PC controls it, so a reimaged or swapped bay comes back with
its printers and nothing had to be saved off the old box. One picker serves both
forms rather than two implementations of the same override. A relationship type
can declare itself singular, so setting a second default REPLACES the first
instead of silently losing to the older row. Alongside the assigned half there is
now an observed half: a bay reports what it actually has, kept strictly apart
from what it is told to have, because a drifted bay's own state becoming its
desired state would make every configuration error permanent. The client script
corrects a drifted queue in place rather than only installing a missing one, and
a driver rollout can be spread across waves so 300 bays do not pull 30 GB through
one five-minute window.
DOCKER SITES SHOULD READ THIS PARAGRAPH. The compose stack never persisted the
instance directory, so `docker compose build api && up -d api` - the update path
the docs themselves gave - discarded plugins.json and every upload with it. It is
a volume now, and DEPLOY.md carries the one-time rescue for a stack that predates
it. The image also could not be built at all: the frontend stage never copied the
plugin staging script its own prebuild hook runs, so every build since that
script landed failed. Air-gapped sites need a fresh offline bundle, because the
stack moves to MySQL 8.4 LTS and an existing tarball carries only the 8.0 image.
The Windows installer already bundled MySQL 8.4, so this closes a gap between the
two halves rather than moving anyone.
Adds a topology migration guide: IIS to Docker, Docker to a new host, and back.
The data moves cleanly; what costs time is that the server address is baked into
GE-Enforce manifests, the generated collector script, the printer client scripts
and printed QR codes.
1736 tests green. Contract stays at 0.20.0 - nothing under shopdb/api changed.
INSTALL-WINDOWS-IIS.md has said MySQL 8.4 LTS is standard for new installs since
8.0 reached end of life in April 2026, while both compose files and the offline
bundler still pinned 8.0. A site reading the Windows runbook and a site reading
the Docker one were being told to run different servers, and the migration page
written this week sent people onto the dead one.
Verified against a real server rather than by editing a tag: 8.4.11, core chain
plus five plugin chains applied clean, 66 tables at a single utf8mb4_unicode_ci
collation, six alembic version tables. The image's PyMySQL authenticates against
8.4's caching_sha2_password, which is what requirements.in already pins
cryptography for.
Existing servers need one thing done FIRST: 8.4 removes mysql_native_password,
so an account created on 5.6 or 5.7 must be moved to caching_sha2_password
before the upgrade or it cannot authenticate afterwards. In-place also has no
downgrade path, and 5.7 cannot reach 8.4 in one hop. For databases this size a
dump into a fresh 8.4 server is the better trade: same outage, and the old
server stays as the rollback.
Air-gapped sites need a fresh offline bundle, because the tarball carries the
MySQL image alongside the app image.
Also here, found by having it bite during that verification: the db healthcheck
pinged over the unix socket, and the entrypoint's init pass answers on the
socket while running the server with --skip-networking. The probe therefore
reported healthy DURING init, which is what `depends_on: service_healthy` gates
api and migrate on. A ping passed at 8 seconds and the next query failed because
the server was mid-restart. Probing 127.0.0.1 keeps it red until the real server
is listening.
The pieces existed across three pages and nothing connected them, so "can we go
from IIS to Docker" had no answer to point at. It is a fair question with a
short answer: the application keeps state in exactly two places, the database
and the instance directory, and nothing is encrypted at rest with SECRET_KEY or
JWT_SECRET_KEY, so a move is a dump plus a directory copy. The schema is
identical across topologies.
What the page spends its length on is the part that is NOT the data, because
that is where the time goes. The server address is baked into things that are
not the server: GE-Enforce manifests, the generated collector script, the
printer client scripts, and printed QR codes, which cannot be swept at all.
Keeping the hostname and repointing DNS makes the migration invisible to the
fleet; changing it does not.
Three other traps, each of which has a symptom that shows up later rather than
at cutover: an aliased IIS site needs MOUNT_PATH and a dist built for that
subpath, while the image builds for the root; an older dump can carry latin1 or
3-byte utf8 table definitions that load quietly into a utf8mb4 server and only
misbehave on the first accented name; and a restored instance directory needs
chown, because docker cp writes under the copying user's uid and the container
runs as shopdb.
Covers both directions plus Docker to a new Docker host, and ends with a cutover
checklist that leaves the old stack stopped rather than removed until a bay has
checked in on a working day.
Also fixes two links in START-HERE that pointed at files which are not there:
the ADR index needed its adr/ prefix, and LLM-GUIDE.md is llms.txt.
`db_data` was a volume and the instance directory was not, so the documented
update path - `docker compose build api && up -d api` - recreated the container
and discarded everything the site had written. `plugins.json` is only the loud
part: maps, branding, model and application images, employee photos, warranty
proofs, slides, printed-part files and the Dell OAuth token all live under
instance_path too. MySQL rows survive and point at files that are gone, so the
second symptom is images 404ing rather than an error anybody sees.
Reported by an adopting site, which read it as having updated too fast. It had
not; nothing it could have done differently would have kept those files.
DEPLOY.md had been telling sites to back up `instance/` since it was written.
The template never gave them anything to back up.
The air-gap `migrate` service mounts the volume too, because
`flask plugin upgrade-all` rewrites plugins.json and that service exits
immediately after.
The image now creates instance/ ITSELF, owned by the app user. Docker seeds an
empty named volume from image content at the mountpoint, ownership included;
with no such directory in the image the mountpoint is created root-owned 0755
and the container, which runs as shopdb, cannot write into its own instance
directory. Caught by running the built image rather than by reading it: the
volume mounted clean and `touch` came back Permission denied. Verified fixed the
same way.
A stack that predates the volume needs its files moved across ONCE, while the
old container still exists - the volume is seeded from image content, and the
image ships instance/ empty, so it comes up empty rather than inheriting the old
container's writable layer. DEPLOY.md carries the procedure, including the chown
after `docker compose cp`, which writes files under the copying user's numeric
uid rather than the app user's.
Also here, found while checking what an upgrade actually runs: the connected
update steps ran `flask db upgrade` and stopped. Per-plugin Alembic chains
(ADR-008) are not part of that, so a connected site taking an image with a
bumped plugin migration ran the core chain and silently skipped every plugin
chain. The air-gap stack had it right all along. Both commands are in Step 9
now, plus a `db current` check against `db heads`.
`npm run build` fires a `prebuild` hook that runs
`node ../scripts/stage-frontend.mjs`, the script that copies each plugin's
frontend into the Vite tree and codegens routes.gen.js. The Dockerfile's
frontend stage copied `frontend/` alone and flattened it to the stage root, so
`../scripts` resolved to `/scripts`, which does not exist. Every image build
since the staging script landed has died there on `Cannot find module`.
The stage now keeps the repo-relative layout - frontend/, scripts/ and plugins/
as siblings under /build - because the script resolves its repo root as its own
directory's parent. The final stage copies from /build/frontend/dist to match.
Verified with a real build, not by inspection: 16 plugin frontends staged, vite
produced dist, and the finished image carries frontend/dist with 241 assets.
This also unblocks SITE_PLUGINS lean image builds, which until now only
scripts/build-site.sh could do.
GE-Enforce gives each PC a start offset of SHA256(hostname) % 5 MINUTES and then
repeats every five minutes. That was sized for reading a few KB of manifest
JSON. A driver set is 100 MB for the two universals and 226 MB for a full site,
so the day a driver entry lands, every bay pulls it inside one five-minute
window: roughly 30 GB across 300 bays, at something like 800 Mbps, on the same
share the whole floor needs for everything else. The failure mode is not slow
printers, it is a floor that stops converging.
-WaveStart with -Waves spreads that out. Each PC derives its own wave from its
hostname, so there is no central coordination, no per-bay configuration, and no
list of who has had it yet. The hash is the same idiom Register-GEEnforce.ps1
already uses for its start offset, SHA-256 rather than MD5 because FIPS-enforced
bays disable MD5 outright and would throw.
Measured over 300 hostnames at 10 waves: 23 to 44 bays per wave against a mean of
30, so the peak wave moves about 4.3 GB rather than the 3 GB an average implies.
Hash bucketing is uneven and the peak is what sizes a link, so do not quote the
mean.
THE GATE RUNS BEFORE THE MANIFEST IS READ, because the manifest is on the share
too. A bay that is not due must not touch the share at all - one read is cheap,
300 bays deciding to read in the same five minutes is the entire problem.
It FAILS CLOSED on an unparseable date. Failing open would restore exactly the
stampede this exists to prevent, and 30 GB cannot be un-sent, whereas a typo that
installs nothing says so in the log every cycle and is fixed in a minute.
A bay powered off during its wave installs on its next cycle instead. The wave is
an earliest-time, not a deadline, so nothing needs chasing afterwards.
-TestOnly reports a bay whose wave has not opened as COMPLIANT, because not
installed is genuinely its desired state today; DSC would otherwise call
SetScript every pass to be told to wait. -IgnoreWave is for proving a pilot bay
before opening anything.
Verified on Windows 11 build 26200, six paths: not-due installs nothing and exits
0; TestOnly while not due exits 0; a garbage date exits 1 having installed
nothing; -IgnoreWave installs against a future start; an opened wave installs;
and no wave arguments at all installs, which is what imaging needs.
ShopDB knew what a bay SHOULD have and nothing about what it DOES. Adding the
observed half makes a rollout a review instead of a typing exercise: the floor
reports itself in, you look, and you adopt.
The collection uses the mechanism that already exists rather than a new one.
POST /api/collector/printers dispatches to the printers plugin's
apply_collector_payload, the same ADR-006 hook the computers and backups plugins
implement. New client script, new plugin-owned table, no new transport and no new
credential.
OBSERVED AND ASSIGNED STAY APART, and that is the point rather than a detail. A
collector report can never write an assignment row: _reconcile_edges is the only
function that writes usesprinter/defaultprinter, it has two call sites, and both
are authenticated routes a human calls. If a drifted bay's own state were allowed
to become what it is told to install, every configuration error would become
permanent the next time that PC checked in.
Seeding an assignment from observed state is explicit -
POST /assignments/seed-from-observed - because a rollout adopts many machines at
once. It routes through the same _reconcile_edges as the editor, so there is one
write path with two doors, and a queue matching no known printer is REFUSED
rather than guessed into an assignment. That last rule is the lesson from the
measuring tools: adopting on a weak key produced 43 duplicate instruments.
Two fixes on top of what the agents built. The replace deleted a host's previous
rows by exact case-folded name while the read path treats a short name and its
FQDN as one machine, so a PC that changed spelling appeared to hold every queue
twice - which reads as drift that is not there. And the client sent 'reportedat'
where the declared schema said 'observedat'.
Also here: the legacy loader now imports machines.printerid, the classic system's
record of each machine's default printer, which it silently dropped - the
production import would have lost every one. And Set-ShopdbPrinters.ps1 finally
registers the per-user logon task, staging Apply-ShopdbDefaultPrinter.ps1 to
C:\ProgramData first because the share it lives on is mounted only during the
enforcement cycle and the task runs at logon when it is gone.
VALIDATED ON WINDOWS 11 (build 26200), not just on Linux pwsh, which parses these
scripts happily and executes none of the spooler branches.
The reporter: posts a correct payload with the X-API-Key header; resolves BaseUrl
and CollectorKey from HKLM when given no arguments; suppresses the virtual queues
by port; resolves port addresses; and reads the CONSOLE USER's default out of
HKU rather than SYSTEM's own, which is a different and usually wrong answer.
Two results matter more than the rest. With the spooler stopped, both the cmdlet
and the CIM path fail and the script posts NOTHING - verified against a capture
server that recorded zero requests, where an empty list would instead have
erased that host's observed rows and read as a bay that lost its printers. A
genuinely empty host still posts [], because that is a real and different fact.
The logon task registers as the Users group at Limited, and falls back to the
well-known SID S-1-5-32-545 when the group name will not resolve, as it will not
on localised Windows. It was then run with the source directory RENAMED AWAY, to
stand in for the share being unmounted, and it still moved the user's default -
which is the whole reason the script is staged to C:\ProgramData rather than run
from where it lives.
The guarantees against damage were re-checked rather than assumed: an empty
assignment changes nothing, an unreachable server changes nothing, -WhatIfOnly
leaves no queue, no task, no staged file and no registry value behind, and a
drifted queue is repointed IN PLACE with Set-Printer so whoever has it as their
default keeps it.
Not covered by any of this: the driver-staging path, which needs a real vendor
package rather than the class drivers a VM ships with.
A queue was matched by NAME alone, so a bay whose printer had moved, or whose
queue was built on a driver the site has since replaced, looked converged and
printed to the wrong device. Absence was fixed; drift was not.
Set-ShopdbPrinters.ps1 now repoints a queue whose port does not match the
address ShopDB holds, and swaps a queue built on the wrong driver. Both are
things ShopDB is authoritative about: where the printer IS, and what drives it.
CORRECTED IN PLACE with Set-Printer, never removed and recreated. The queue keeps
its name, its sharing, its permissions, and whoever has it as their default keeps
it - which is what makes this safe to run every cycle on a live floor. There is
still no removal code path in this script at all.
Two guards, because a repair that breaks a working printer is worse than drift:
the driver is only swapped when the wanted one is actually staged, and
-WhatIfOnly reports both kinds of correction without making either.
Verified on Windows against a queue that had the right name, the wrong port, the
wrong driver AND was the logged-on user's default: both fields were corrected and
the queue was still the default afterwards. The earlier no-op guarantees were
re-run and still hold - nothing assigned changes nothing, and an assignment with
no default leaves the user's own default alone.
The assignment belongs to the MACHINE, and until now there was no way to set it
except the generic relationships card or the API - the form for the thing the
feature is about did not exist. MachineForm now carries the picker, and PCForm
uses the SAME component rather than its own copy: the PC's set overrides the
machine's, and two implementations of that would drift, with the two ends of an
override disagreeing being exactly the bug nobody would spot.
The shared picker also fixes what PCForm did on save. It wrote row at a time
through the generic relationship endpoints, which is a non-atomic reconcile: an
HTTP failure part way left a PC half-assigned with nothing recording what was
meant. It now calls the reconcile endpoint, which validates the default before
writing anything.
A relationship type can now say it allows one active row per asset
(relationshiptypes.issingular, migration 7d34), and defaultprinter says it.
Cardinality belongs to the type rather than the printers plugin: core's create
path is where every hand-made link passes, and the next type meaning "exactly
one" gets the rule for free. Setting a second default REPLACES the first instead
of refusing, because "make this the default" means that - and a card answering
409 would leave the user hunting for the old row.
Without it the schema was happy to hold two defaults: the unique constraint is
(source, target, type), so two different targets are two valid rows, and the
resolver takes the OLDEST - the new default silently lost. Proven by disabling
the new rule and watching the tests fail.
FOUND WHILE TESTING IN A BROWSER, and it was not mine: MachineForm read
.data.data off computersApi.listAll(), which resolves to the ARRAY - fetchAllPages
has already unwrapped every page. The whole parallel load threw into the catch,
so every dropdown on the machine edit form came up empty and the machine's own
values never loaded. A build cannot see this; only opening the page can.
GET /api/printers/assignments/for-asset/<id> returns an asset's OWN assignment,
without inheritance, because the editor must show what this asset's rows say -
otherwise a machine's printers appear ticked on the PC that inherits them and
unticking one silently creates an override.
The last of the buildings-and-levels bugs, and the shop-floor clients brought
into the product.
Anyone on 0.11.0 through 0.11.2 should take this. Every hover mini-map in the
product reported "this asset has a position but no level" - the levelid prop
added in 0.11.0 was passed by none of its seven call sites - and the map PDF
printed markers from every floor onto one sheet, which nobody can correct once it
is carried onto the floor. The legacy import loader, still to run against
production, created markers with no level at all.
The gate that should have caught all three asked whether a FILE mentions levelid
rather than whether each position does. It now checks per occurrence.
Also: printers can be assigned to a MACHINE and reach whichever PC controls it,
so a reimaged bay reinstalls its own printers with nothing saved off the old PC;
printerdrivers can name a vendor, so two rows cover 41 of 44 printers instead of
twenty-one near-duplicates; and the collector reporter and EventSaver now live in
the repository with no site baked into either, the reporter generated per site on
request.
The client scripts were validated on Windows 11 against a live ShopDB, not only
by the suite: a bay with no rows of its own created both queues from its
machine's assignment, bound them to the right universal drivers, and set the
per-user default.
The version and the changelog are the release; the detail is in the entry.
Two rows now cover 41 of 44 printers. printerdrivers could only bind a driver to
ONE modelnumberid, so the HP and Xerox universal drivers - which between them
cover almost the whole floor - would have needed 21 near-duplicate rows pointing
at the same package. That is a table nobody keeps true, and it is why 42 of 44
printers resolved no driver at all.
printerdrivers gains vendorid, and resolution runs most-specific-first: the
printer's model, then its vendor, then the pre-vendorid convention of matching
the vendor word in the driver's name so a site that populated the table before
the column existed does not silently lose every driver on upgrade. A row that
names a vendor is never matched by its text, because a mis-set vendor resolving
to the wrong package is worse than resolving to none.
Six rows now resolve 44 of 44 printers at the reference site, and the DesignJet
correctly takes its own driver over the HP universal one.
Set-ShopdbPrinters.ps1 is the client half: ask for-host, create the queues that
are missing, record the desired default. It NEVER removes a queue - a bad minute
from the API must not take printers away from a working bay - and it never
fetches a driver, because downloading 48 MB while somebody waits to print is the
wrong moment. The common scope stages those.
Apply-ShopdbDefaultPrinter.ps1 applies the default in the USER's context, which
is the only context that can: SYSTEM cannot set a per-user default for somebody
else. It also turns off "Let Windows manage my default printer", without which
Windows silently overwrites the choice the next time anyone prints elsewhere -
a fix that undoes itself within a day.
VALIDATED ON WINDOWS 11 AGAINST A LIVE SHOPDB, not only by tests. Printers were
assigned to a MACHINE; a PC controlling it, holding no rows of its own, created
both queues bound to the right universal drivers, recorded the default and set
it, and a second run changed nothing. The first attempt failed with
"Relationship types are not seeded - run: flask seed reference-data", which is
the deployment trap the plan predicted, caught by an explicit error rather than
silently resolving nothing.
Printers belong to the bay, not to the box currently driving it. The assignment
goes on the MACHINE asset and reaches whichever PC controls it, so a reimaged or
swapped PC comes back with the right printers and nothing had to be saved off the
old one. The asset register is the backup.
New relationship type usesprinter ("this printer is installed here"), beside the
existing defaultprinter ("which of them is the default"), both seeded and both
given a propagation rail through controls. The rails are consumed at READ time
only: the create-time fan-out skips directional through-types, and controls is
directional, so assigning a printer to a machine does not copy rows onto its PC.
That is what keeps own-beats-inherited possible.
Resolution for a PC is its OWN rows if it has any, otherwise one hop out along
controls to the machines it drives. Whole set at a time, not merged: a PC with
its own assignment is overriding the bay deliberately, and the UI has to say so
or a tech "fixing" a bay by editing the PC will shadow the machine's record and
wonder why they keep disagreeing.
GET /api/printers/for-host/<hostname> is what the convergence client asks every
cycle. Resolved by hostname because the collector upserts PCs by hostname and an
office PC has no machine number. An unknown host, a site without the computers
plugin, and nothing assigned all return an empty set - that is the client's
designed no-op and it must stay indistinguishable from "assigned nothing".
PUT /api/printers/assignments/for-asset/<id> reconciles the whole set in one
call. The endpoint was specified, documented and asserted by three tests, and
never written - the verification pass caught that, with four failures. It
validates the default BEFORE any write, so a rejected request changes nothing;
soft-deletes rows that went away; and REACTIVATES soft-deleted rows rather than
inserting, because the unique constraint spans inactive rows and a blind insert
after an unassign raises IntegrityError on MySQL while passing on SQLite.
One default per asset, enforced here because the schema cannot: the constraint is
(source, target, type), which accepts two different defaults quite happily. Two
active defaults are still reachable through the generic relationships endpoint,
where the oldest silently wins - recorded in the proposal as the next thing to
close.
printerdrivers gains drivername: the exact string the INF declares, which
Add-PrinterDriver matches on and nothing else. Deriving it by parsing INFs on
hundreds of bays is fragile; a human confirming it once is not.
Assigning a printer to a bay is useless if the bay cannot install it, and the
fleet data says why that mattered: 42 of 44 printers could not resolve a driver.
This is the delivery half - the drivers themselves, staged once per bay, so that
creating a queue never waits on a download.
Install-ShopdbPrinterDriver.ps1 does one driver: trust the package's signer, then
pnputil /add-driver, then Add-PrinterDriver. Install-ShopdbPrinterDrivers.ps1
does a site's whole set from drivers.json, and answers a compliance question with
-TestOnly, which is what makes it a clean DSC Script resource rather than a
fire-and-forget install.
Deliberately SEPARATE from assignment. Drivers are large, near-identical across a
fleet and change rarely; assignments are small, per-bay and change often. Staging
the set in the GE-Enforce common scope means the assignment client only ever
creates a queue - it never fetches a 48 MB package while somebody is waiting to
print, or discovers the share is unmounted at the worst moment.
THE SIGNER TRUST STEP IS THE WHOLE TRICK, and it took a real driver to find it.
certutil -addstore on the .cat file satisfied the Xerox package and failed every
HP INF with "The publisher of an Authenticode(tm) signed catalog has not yet been
established as trusted" - a coin toss, not a mechanism. The certificate is now
extracted with Get-AuthenticodeSignature and added to Trusted Publishers, for
every catalog under the package rather than the first INF's neighbours. On a
locked bay there is no prompt to answer, so the old failure was silent.
Verified on Windows against real packages, not by reading: all six drivers this
site needs install through the script, a second run is a no-op, a wrong driver
name fails with the names the package actually offers, and the DSC cycle behaves
- TestOnly exits 1 on a clean box, install exits 0, TestOnly then exits 0.
The packages themselves stay out of git: they are licensed vendor binaries, and
they belong on the share beside the other imaging payloads.
DEPLOYING-DRIVERS.md carries the GE-Enforce entry, the DSC configuration and the
Intune shape, plus the constraint that has cost a session before: the SFLD share
is mounted only during the enforcement cycle, so this runs as a manifest entry
and never as its own scheduled task.
A site adopting ShopDB had to be handed two files and told what to edit in them.
Both are now the product's, and one of them the server writes for you.
GET /api/computers/client-script (admin) returns Report-AssetToShopDB.ps1 with
this site's values already in it: site_base_url becomes the -ApiUrl default and
the new computers_routableranges setting becomes -AllowedRanges. Only the
PARAMETER DEFAULTS are substituted - the copy in plugins/computers/client/ stays
runnable, so there is no second version to drift from the first - and everything
stamped stays overridable by argument or registry, because a bay may need to
differ from its site. Settings > Computers > Asset reporter edits the ranges,
downloads the script and shows its SHA-256.
The collector key is deliberately not stamped in, and a test fails if it ever
is. That file lands on every shop-floor PC, and a token spread across hundreds
of bays cannot be rotated quietly; it stays in the registry, provisioned per
ADOPTING-AT-ANOTHER-SITE.md.
The routable ranges are the last thing that was hardcoded in that script. They
are now a setting, so West Jefferson's two CIDRs move out of source code and
into that site's own configuration - which is what ADR-015 asks for - and a site
that sets nothing still works, because the script falls back to the NIC carrying
the default route.
EventSaver joins it in plugins/slides/client/, source only: EventSaver.cs and
EventSaver.ini, no compiled .scr - a binary is a release asset, like the
installer exe. The share path that was compiled into Config.Folder is gone. It
used to be the fallback when the ini was missing, which silently pointed a new
site at the reference site's file server; it is now empty, and failing visibly
beats displaying another site's slides. Verified by compiling the edited source
in the Windows VM with the in-box csc.exe: 15,872 bytes, exit 0.
Also: the DSC example in the adoption guide gains a CollectorRanges resource and
stops passing -ApiUrl to a script that already reads BaseUrl from the registry
the same example writes, and the guide points at the generated download instead
of hand-editing a URL.
The contract test caught the endpoint importing shopdb directly for the version
string, which ADR-002 forbids a plugin from doing. The product and contract
versions are in app.config now, which a plugin reads through current_app.
Adds docs/proposals/printer-assignment.md: assign printers to a PC in ShopDB and
let the bay install them, with what the fleet data says about drivers - HP and
Xerox cover 41 of 44 printers with universal drivers, there are no Brother
printers at all despite 208 files of Brother inkjet drivers in the installer,
and printerdrivers holds one row pointing at a per-model folder instead of a
universal driver.
Report-AssetToShopDB.ps1 lived on one site's imaging share and was, per the
adoption guide, "provided on request" - which is not a distribution mechanism for
a product meant to be adopted. It now lives in plugins/computers/client/, beside
the collector contract it implements (ADR-006), so the two version together.
Three things named West Jefferson and no longer do (ADR-015):
- The server. It resolves from HKLM:\SOFTWARE\GE\ShopDB BaseUrl - the value
Install-GEEnforce.ps1 already writes and the enforcement client cannot run
without - or from -ApiUrl. With neither it logs and exits 0 rather than
posting somewhere wrong. Any bay running this script runs the enforcement
client, so the value is present wherever it is deployed.
- The corporate VLANs. Two hardcoded CIDRs decided which NIC's address was
reported, with a comment reading "update if site re-VLANs". A site may now
name its ranges (-AllowedRanges, or a CollectorRanges registry value); with
none configured the NIC carrying the DEFAULT ROUTE is used, which expresses
"the routable NIC, not the controller NIC" without knowing any site's
addressing.
VERIFIED IN THE WINDOWS VM, not by reading it - and the VM earned its keep. The
local array was called $allowedRanges, which is the SAME VARIABLE as the [string]
parameter $AllowedRanges because PowerShell names are case-insensitive; the array
was silently coerced to an empty string, and .Count on a scalar string is 1. The
script therefore believed a range was configured, skipped the default-route
fallback, and reported no IP at all. Linux pwsh parsed it happily. Renamed to
$rangeList, and the four paths were then confirmed on Windows: no config skips
cleanly, BaseUrl resolves from the registry, an unconfigured site picks the
default-route NIC, a configured range selects or excludes as asked.
The hover mini-map said "This asset has a position (2835, 1410) but no level"
for every asset in the product. When 0.11.0 gave LocationMapTooltip a levelid
prop, NONE of its seven call sites were taught to pass one - printer, machine and
PC detail pages, the toner report, enforcement reports, the warranty chip and the
dashboard cards - so the component correctly reported a missing level and the
preview never drew. Two payloads behind those views also emitted mapx/mapy with
no level: the toner report and the enforcement report.
The map PDF export had the ORIGINAL bug still in it: it plotted every filtered
asset onto the sheet, so exporting the ground floor printed second-floor markers
on it. Worse than on screen, because nobody can correct a sheet once it has been
printed and carried onto the floor. It now exports only the level being viewed.
The legacy import loader sent mapleft/maptop with no level at three call sites.
That loader is the one still to run against production, and every marker it
created would have been undrawable. It now resolves the site's default level -
the legacy schema predates levels and has one floor plan, so that is what its
coordinates mean.
THE GATE MISSED ALL OF THIS because it asked whether a FILE mentions 'levelid',
not whether each position does: one module emitted 'mapx' six times and 'levelid'
once and passed. It now checks per occurrence, covers scripts/ as well as shopdb/
and plugins/, and fails any Vue file that binds tooltip coordinates without
:levelid. Both new rules were confirmed to fail the build against planted
violations before being relied on.
Printer QR labels: the asset number is no longer printed. A label now reads name
(8201-HPLaserJetPro), QR, FQDN, then IP. The name falls back to the assetnumber
because that is where sites actually keep it - every printer here has an empty
name field, so preferring the Windows queue name alone would have printed a blank
line on every label.
2026-08-18 09:36:45 -04:00
667 changed files with 94916 additions and 4618 deletions
@@ -4,6 +4,16 @@ Follow these when suggesting code. They are enforced by a naming/style hook and
by CI (`.github/workflows/ci.yml`) - suggestions that break them fail the build.
by CI (`.github/workflows/ci.yml`) - suggestions that break them fail the build.
`CONTRIBUTING.md` is the full authority; this is the short version.
`CONTRIBUTING.md` is the full authority; this is the short version.
## General Guidelines
- Finish the in-flight task before starting new workstreams.
- When asked to "take notes" on a future idea, capture it in the notes file only - do not implement it, and do not expand on it at length in replies.
- Use a temp file for multi-line git commit messages instead of heredoc syntax: `git commit -F <file>`.
- Avoid using `powershell.exe -File` for launching scripts, as it fails if the script file is missing or quarantined.
## Installer Logic
- The endpoint agent installer shipped to endpoints is generated by `_install_script()` in `plugins/endpoints/api/siteagent.py`, NOT `agent/installer/install-shopdbagent.ps1`. Any installer logic change must be made in `siteagent.py`, followed by rolling the containers.
- The user's workflow: the agent hands over only the built EXEs (`shopdbagent.exe`, `shopdb-endpoint-tray.exe`); the user uploads them via the "Upload New/Legacy Agents" field in the Build Site Agent UI and builds the package with the newest version selected at the top.
## Naming (LOCKED - the hook rejects violations)
## Naming (LOCKED - the hook rejects violations)
- **DB tables**: lowercase, concatenated, plural. No underscores, no dashes.
- **DB tables**: lowercase, concatenated, plural. No underscores, no dashes.
with quantities are standalone tables, not assets.
with quantities are standalone tables, not assets.
- **Ledger pattern**: a cached `quantityonhand` moves in the SAME commit as the
- **Ledger pattern**: a cached `quantityonhand` moves in the SAME commit as the
signed transaction row it reflects.
signed transaction row it reflects.
- **Configuration for endpoint agent**: The endpoint agent's live collector key and base URL are provisioned in the Windows registry `HKLM\SOFTWARE\GE\ShopDB` (values `CollectorKey`, `BaseUrl`), not committed in `appsettings.json`; `appsettings.json` holds only non-secret runtime settings.
- MySQL 5.7+ database (5.6 works with extra utf8mb4 config; see docs/DEPLOY.md)
- MySQL 5.7+ database (5.6 works with extra utf8mb4 config; see the DEPLOY wiki page)
- JWT authentication
- JWT authentication
- Plugin architecture for extensibility
- Plugin architecture for extensibility
@@ -100,7 +100,7 @@ To maintain consistency with the legacy ShopDB database and codebase, the follow
- Python 3.14
- Python 3.14
- Node.js 18+
- Node.js 18+
- MySQL 5.7+ (5.6 works with extra utf8mb4 config; see docs/DEPLOY.md)
- MySQL 5.7+ (5.6 works with extra utf8mb4 config; see the DEPLOY wiki page)
ShopDB Flask uses MySQL as the canonical database. SQLite is used only for the
ShopDB Flask uses MySQL as the canonical database. SQLite is used only for the
test suite (`TestingConfig` in `shopdb/config.py` points at an in-memory
test suite (`TestingConfig` in `shopdb/config.py` points at an in-memory
@@ -121,7 +121,7 @@ so a container deploy needs no separate Node build step.
```bash
```bash
cp .env.example .env
cp .env.example .env
# Edit .env: set SECRET_KEY, JWT_SECRET_KEY, DATABASE_URL, CORS_ORIGINS,
# Edit .env: set SECRET_KEY, JWT_SECRET_KEY, DATABASE_URL, CORS_ORIGINS,
# and the MYSQL_* passwords (see docs/CONFIG.md for every variable).
# and the MYSQL_* passwords (see the CONFIG wiki page for every variable).
docker compose up -d --build
docker compose up -d --build
@@ -172,7 +172,7 @@ including VS Code and troubleshooting, is the DEVELOPMENT-SETUP page in the
project wiki.
project wiki.
To import a site's legacy data, use the HTTP import surface: an admin API
To import a site's legacy data, use the HTTP import surface: an admin API
token plus [docs/IMPORT-API.md](docs/IMPORT-API.md) drive the whole migration
token plus the IMPORT-API page in the project wiki drive the whole migration
through documented endpoints (`X-Import-Mode` preserves original timestamps).
through documented endpoints (`X-Import-Mode` preserves original timestamps).
`scripts/site_imports/wjf/` is the reference site's loader, kept as a worked
`scripts/site_imports/wjf/` is the reference site's loader, kept as a worked
example of the whole sequence.
example of the whole sequence.
@@ -181,11 +181,11 @@ example of the whole sequence.
| Target | Use |
| Target | Use |
|---|---|
|---|---|
| **Windows Server + IIS** (how sister sites run) | **[docs/INSTALL-WINDOWS.md](docs/INSTALL-WINDOWS.md)** - one installer `.exe`, offline, no manual IIS work. Day 2: [docs/OPERATE-WINDOWS.md](docs/OPERATE-WINDOWS.md) |
| **Windows Server + IIS** (how sister sites run) | **the INSTALL-WINDOWS page in the project wiki** - one installer `.exe`, offline, no manual IIS work. Day 2: the OPERATE-WINDOWS page in the project wiki |
| Linux / Docker, air-gapped | [docs/DEPLOY-AIRGAP.md](docs/DEPLOY-AIRGAP.md) |
| Linux / Docker, air-gapped | the DEPLOY-AIRGAP page in the project wiki |
| Linux / Docker, connected | [docs/DEPLOY.md](docs/DEPLOY.md) |
| Linux / Docker, connected | the DEPLOY page in the project wiki |
For every environment variable and Setting key see [docs/CONFIG.md](docs/CONFIG.md).
For every environment variable and Setting key see the CONFIG page in the project wiki.
### What the Windows installer requires
### What the Windows installer requires
@@ -205,7 +205,7 @@ The binding constraint is the operating system rather than IIS: the payload is
Python 3.14 plus HttpPlatformHandler and URL Rewrite, and the handler itself runs
Python 3.14 plus HttpPlatformHandler and URL Rewrite, and the handler itself runs
on older IIS. An older host is untested rather than known-broken.
on older IIS. An older host is untested rather than known-broken.
If you must deploy onto something older, [docs/INSTALL-WINDOWS-IIS.md](docs/INSTALL-WINDOWS-IIS.md)
If you must deploy onto something older, the INSTALL-WINDOWS-IIS page in the project wiki
is the manual procedure and has no OS gate - it even documents MySQL 5.6, which is
is the manual procedure and has no OS gate - it even documents MySQL 5.6, which is
that era of machine. The trade is stated there: it produces a server the
that era of machine. The trade is stated there: it produces a server the
installer will not subsequently upgrade.
installer will not subsequently upgrade.
@@ -248,7 +248,7 @@ Query parameters for list endpoints:
## Where to start
## Where to start
[docs/START-HERE.md](docs/START-HERE.md) routes by what you are here to do -
the START-HERE page in the project wiki routes by what you are here to do -
standing up a site, deploying the shop-floor tools, writing a plugin,
standing up a site, deploying the shop-floor tools, writing a plugin,
integrating with the API, or diagnosing something. On the published mirror the
integrating with the API, or diagnosing something. On the published mirror the
same page is the wiki's front door.
same page is the wiki's front door.
@@ -259,7 +259,7 @@ ShopDB supports plugins for extending functionality. See `CONTRIBUTING.md` for p
The image bundles these plugins; only the ones a site installs are loaded. The
The image bundles these plugins; only the ones a site installs are loaded. The
current count and each plugin's version and migration head are in
current count and each plugin's version and migration head are in
[docs/PROJECT-MAP.md](docs/PROJECT-MAP.md), which is generated:
the PROJECT-MAP page in the project wiki, which is generated:
- **backups** - Machine configuration backups (NTLARS/DNC, CMM, part marker, UDC) with revision history and diffs
- **backups** - Machine configuration backups (NTLARS/DNC, CMM, part marker, UDC) with revision history and diffs
- **computers** - Shopfloor PCs and workstations, collector fleet ingest
- **computers** - Shopfloor PCs and workstations, collector fleet ingest
| GE-Enforce client (`Install-GEEnforce.ps1`, `Invoke-ShopdbEnforce.ps1`, `ShopdbEnforceClient.psm1`) | This repository, `plugins/geenforce/client/`. Present on any installed server under the install directory. |
| GE-Enforce client (`Install-GEEnforce.ps1`, `Invoke-ShopdbEnforce.ps1`, `ShopdbEnforceClient.psm1`) | This repository, `plugins/geenforce/client/`. Present on any installed server under the install directory. |
| `Report-AssetToShopDB.ps1` | **Not in this repository.** It lives on the reference site's imaging share and is provided on request. It is planned to move to `plugins/computers/client/` so it versions with the collector contract it implements. |
| `Report-AssetToShopDB.ps1` | `plugins/computers/client/`, so it versions with the collector contract it implements. It names no site: the server comes from `HKLM:\SOFTWARE\GE\ShopDB``BaseUrl` (which Install-GEEnforce.ps1 writes) or `-ApiUrl`, and the NIC it reports is the one carrying the default route unless the site names its ranges via `-AllowedRanges` or the `CollectorRanges` registry value. |
| EventSaver (`EventSaver.scr`, `EventSaver.ini`, `EventSaver.cs`) | **Not in this repository.** Provided on request; the source is a single C# file that builds with the in-box .NET Framework compiler, so a site can rebuild it rather than trust a binary. See [EVENTSAVER.md](EVENTSAVER.md). |
| EventSaver (`EventSaver.cs`, `EventSaver.ini`) | `plugins/slides/client/`. The source is a single C# file that builds with the in-box .NET Framework compiler, so a site can rebuild it rather than trust a binary. The compiled `EventSaver.scr` is a release asset, not a file in the repository. Neither file names a site. See [EVENTSAVER.md](EVENTSAVER.md). |
Ask the maintainers for the two that are not here. A site that would rather not
A site that would rather not run a binary it cannot rebuild should take
run a binary it cannot rebuild should take EventSaver's source and compile it
EventSaver's source and compile it locally - the build needs no SDK and is one
locally - the build needs no SDK and is one command.
command.
---
---
@@ -52,8 +52,13 @@ is simply absent; the server upserts on `hostname` and leaves the rest alone.
| Input | Where it comes from |
| Input | Where it comes from |
|---|---|
|---|---|
| Server URL | `-ApiUrl https://<your-shopdb>/api/collector/computers` |
| Server URL | `-ApiUrl`, or `HKLM:\SOFTWARE\GE\ShopDB` value `BaseUrl`. A script downloaded from Settings has your URL already in it |
| API key | `-ApiKey`, or `HKLM:\SOFTWARE\GE\ShopDB` value `CollectorKey` |
| API key | `-ApiKey`, or `HKLM:\SOFTWARE\GE\ShopDB` value `CollectorKey`. Never stamped into the script |
| Routable ranges | Optional. `-AllowedRanges '10.20.0.0/23,10.21.4.0/26'`, the `CollectorRanges` registry value, or the `computers_routableranges` setting. Unset reports the NIC carrying the default route, which is right at most sites |
**Shortcut: download it pre-configured.** Settings > Computers > Asset reporter
generates this script with your server's URL and ranges already in the parameter
defaults, and shows its SHA-256. The key is deliberately not included.
@@ -109,7 +109,7 @@ carries value or liability of its own:
| Endpoint | Requires | Why |
| Endpoint | Requires | Why |
|---|---|---|
|---|---|---|
| `GET /api/applications/package/<filename>` | `applications.view` | Licensed vendor installers. An anonymous URL is a redistribution channel. |
| `GET /api/applications/package/<filename>` | `applications.download` | Licensed vendor installers. A separate permission from `applications.view` on purpose: seeing that an application exists is not the same as being handed the vendor's installer, and an anonymous URL is a redistribution channel. The script-body download is gated the same way. |
| `GET /api/warranty/proof/<filename>` | `warranty.view` | Invoices and certificates carry pricing and a service tag. |
| `GET /api/warranty/proof/<filename>` | `warranty.view` | Invoices and certificates carry pricing and a service tag. |
Their sibling upload/delete routes are `applications.edit` / `warranty.edit`
Their sibling upload/delete routes are `applications.edit` / `warranty.edit`
@@ -135,14 +135,23 @@ like any other mutation.
| `GET /api/plugins/enabled` | List enabled plugins (no claims used). |
| `GET /api/plugins/enabled` | List enabled plugins (no claims used). |
| `GET /api/dashboarddefaults/display-role` | Display role resolution (see section 1). |
| `GET /api/dashboarddefaults/display-role` | Display role resolution (see section 1). |
| ~~`GET /api/employees/search`, `/lookup/<sso>`, `/lookup`~~ | NOT public. These need `employees.view` and answer 401 anonymously. A kiosk resolves a badge through its own gated endpoint, not through these. |
| `GET /api/employees/photo/<filename>` | Serve an employee photo. |
| `GET /api/employees/photo/<filename>` | Serve an employee photo. |
| `GET /api/notifications` (and `/types`, `/<id>`, `/active`, `/calendar`, `/dashboard/summary`, `/employee/<sso>`, `/shopfloor`) | Read shop-floor notifications for the public display and kiosks. |
| `GET /api/notifications/shopfloor`, `GET /api/notifications/employee/<sso>` | The two the public display and kiosks actually read. The rest of `/api/notifications` (the list, `/types`, `/<id>`, `/active`, `/calendar`, `/dashboard/summary`) needs `notifications.view` and answers 401 anonymously. |
| `GET /api/slides/feed` | Slide feed for the lobby display and screensaver. |
| `GET /api/slides/feed` | Slide feed for the lobby display and screensaver. |
| `GET /api/slides/img/<surface>/<filename>` | Serve a slide image. |
| `GET /api/slides/img/<surface>/<filename>` | Serve a slide image. |
| `GET /api/printedparts/image/<filename>` | Serve a printed-part image. |
| `GET /api/printedparts/image/<filename>` | Serve a printed-part image. |
| `GET /api/printedparts/kiosk/item/<itemcode>` | Kiosk part lookup (deliberately open; a kiosk carries no JWT). |
| `GET /api/printedparts/kiosk/item/<itemcode>` | Kiosk part lookup. Carries no JWT, but is NOT open: the routes 404 unless `printedparts_kiosk_enabled` is on AND the request carries the site's `X-Kiosk-Token`. |
| `POST /api/printedparts/kiosk/take` | Kiosk part checkout (deliberately open, per decision record). |
| `POST /api/printedparts/kiosk/take` | Kiosk part checkout, behind the same flag and device token. |
| `GET /api/usb/kiosk/device/<device_id>` | USB kiosk device lookup, behind `usb_kiosk_enabled` plus `X-Kiosk-Token`. |
| `POST /api/usb/kiosk/checkout`, `/checkin` | USB kiosk custody moves, same gate. |
**"No authentication" is not "open."** Both kiosks are off by default and refuse
every request without the site's device token; with the flag off they answer 404
rather than 403, so a kiosk a site has not turned on does not announce itself.
The token is a credential for a screen, not authorization for a person: it says
which cabinet is asking, and the badge on the request says who is standing at
| `hostname` (required) | string | Identity. Matches `Computer.hostname` (case-insensitive), then falls back to `Asset.assetnumber`. New asset created if no match. |
| `hostname` (required) | string | Identity. Matches `Computer.hostname` (case-insensitive), then falls back to `Asset.assetnumber`. New asset created if no match. |
| `machinenumber` | string | Identifies the MACHINE this PC drives, never the PC. A new PC always takes `assetnumber = hostname`, and an existing PC's `assetnumber` is left alone. The number resolves a machine asset and builds a PC -> machine `controls` relationship; an unknown number warns rather than creating a machine. The placeholder `9999` and empty string link nothing. See "Machine links" below. |
| `machinenumber` | string | Identifies the MACHINE this PC drives, never the PC. A new PC always takes `assetnumber = hostname`, and an existing PC's `assetnumber` is left alone. The number resolves a machine asset and builds a PC -> machine `controls` relationship; an unknown number warns rather than creating a machine. The placeholder `9999` and empty string link nothing. See "Machine links" below. |
| `deviceid` | string | Identifies the DEVICE hanging off this PC, read from `C:\Enrollment\asset-id.txt`. ONE file for every bay with no NTLARS/eDNC `MachineNo` registry to identify it. It names the asset and nothing else - it does NOT say what kind of device it is, because the pc-type already does - so the same file serves a part marker, a Keyence, a Genspect or anything declared later. SEPARATE from `machinenumber` on purpose: "which bay is this" and "which device is this" are different facts, and `machinenumber` is what GE-Enforce `TargetMachineNumbers` gates on, so naming a device there would silently stop every bay-gated manifest entry from matching. An unknown value warns rather than minting, and a value resolving to the wrong KIND of asset is refused with the asset named. See "PC -> device links" below. |
| `measuringtoolid` | string | The 0.12.0 name for the same thing, before the file was generalised, read from `C:\Enrollment\measuringtool-id.txt`. Accepted as an alias so a bay already staged with that file keeps reporting; `deviceid` wins when both arrive. Prefer `asset-id.txt` for anything new. |
| `pctype` | string | `gea-shopfloor-*` imaging type -> `Computer.computertypeid` via the configurable `pctypemap` settings. Unmapped value -> warning, not error. |
| `pctype` | string | `gea-shopfloor-*` imaging type -> `Computer.computertypeid` via the configurable `pctypemap` settings. Unmapped value -> warning, not error. |
| `pcsubtype` | string | Accepted but not stored yet -> warning. |
| `pcsubtype` | string | Accepted but not stored yet -> warning. |
| `modelnumber` | string | `Computer.modelnumberid`, scoped to the vendor when known. Model row auto-created if missing. |
| `modelnumber` | string | `Computer.modelnumberid`, scoped to the vendor when known. Model row auto-created if missing. |
| `osname` | string | `Computer.osid`. Controlled vocab: looked up in `operatingsystems`, NOT auto-created. Unknown value -> warning (row still written, `osid` left unset). |
| `osname` | string | `Computer.osid`. Controlled vocab: looked up in `operatingsystems`, NOT auto-created. Unknown value -> warning (row still written, `osid` left unset). |
| `installedsoftware` | array of `{name, version}` | `ComputerInstalledApp` rows for applications shopdb already tracks. Unknown app name -> warning, skipped. |
| `installedsoftware` | array of `{name, version}` | `ComputerInstalledApp` rows for applications shopdb already tracks. Unknown app name -> warning, skipped. |
| `defaultprinter` | string | The default printer's identifier (windows name / share / hostname / port IP). Resolved to a printer asset and linked PC -> printer as a `defaultprinter` relationship. Unresolved -> warning. |
| `printers` | array of strings | All installed network printer identifiers. Each resolves to a printer asset and is linked PC -> printer as a `connectedto` relationship (the default is skipped here since it already links as `defaultprinter`). Unresolved entries -> warning. |
Schema source of truth: `get_collector_schema` in `plugins/computers/plugin.py`.
Schema source of truth: `get_collector_schema` in `plugins/computers/plugin.py`.
If you change the payload, change it there and re-check this table.
If you change the payload, change it there and re-check this table.
@@ -261,31 +263,68 @@ so control still follows from controlling the marker, without two markers
contesting a link only one can hold. Backups from a marker PC file against the
contesting a link only one can hold. Backups from a marker PC file against the
marker rather than the operation.
marker rather than the operation.
### PC -> printer relationship sync
### PC -> device links (`asset-id.txt`)
When a payload carries `defaultprinter` and/or `printers`, the collector syncs
Some pc-types mean the PC drives an attached DEVICE: a CMM, Keyence, Genspector
`AssetRelationship` rows so a PC page shows its printers and a printer page shows
wax-trace instrument, or a part marker. `SUBORDINATE_DEVICE_MAP` in
the PCs that use it (both render in the shared Relationships card).
`plugins/computers/pctypemap.py` declares which, and a site can add its own
through a `subordinatedevice_<pctype>` setting without a code change.
- Resolution: each identifier is matched, first hit wins, against the printer's
`C:\Enrollment\asset-id.txt` names that device. One file for every bay that has
`windowsname`, `hostname`, `sharename`, its asset number/name, then any active
no NTLARS/eDNC `MachineNo` registry to identify it, holding one line: the
printer communications IP. Case-insensitive except the IP (exact). An
device's `assetnumber`. It does NOT record what kind of device it is - the
identifier that resolves to nothing adds a warning and is skipped; it never
pc-type already does - so a bay declared later needs no new file and no client
fails the whole push.
change.
- Link types: the default printer links with `defaultprinter` (directional, PC
is the source); every other reported printer links with `connectedto`
Resolution runs most-stable-identity first, and ADOPTS before it mints:
(symmetric). A printer that is both default and in `printers` links only as
the default.
1.**`deviceid`** - the asset named in `asset-id.txt`.
- Idempotent: re-reporting the same set creates no duplicate rows (an existing
2. A prior collector link from this PC, reactivated.
matching row is reactivated if it was archived, otherwise left as is).
3. An existing device this PC already controls that the collector did NOT
- Stale-link archive: on every push, collector-created links to printers no
create - a legacy `MT-####` row, or one somebody made by hand. Adopting
longer reported are set inactive. Collector-created rows are tagged in
stamps the label so it is ours from then on.
`assetrelationships.label = 'collector:printers'`; only tagged rows are ever
4. For a measuring tool only: the reported machine number, when it resolves to
archived, so links you create by hand in the UI are never touched. A payload
one. This is the CMM case, where `cmmid.txt` already reports `CMM4`.
that omits BOTH printer keys leaves all existing printer links untouched
5. Mint, only when none of the above matched.
(report an empty `printers: []` to clear the auto links instead).
- Response: the collector response carries `printerlinkcount` and a
**Why the file exists, and why it is step 1.** Every other identity here is
`printerlinks` list of `{assetid, relationshiptype}` for the links kept.
derived from the PC. Step 2 looks for a link from THIS PC asset; step 5 mints
`<PC number>-<SUFFIX>`, and a PC's asset number is its hostname. All of them
survive a re-image and NONE of them survives a PC SWAP: a new hostname is a new
PC asset with no prior link and a predicted number that has never existed, so
the same physical device gets a second record while the first keeps its config
and backup history under a dead PC's name. That is how 43 legacy `MT-####`
tools ended up shadowed by minted twins, three records deep in places. Treat
anything minted as a placeholder to be reconciled.
Two guards on the file's contents:
- An **unresolvable** value warns and links nothing, rather than inventing a
phantom device nobody can account for.
- A value resolving to the **wrong kind** of asset is refused, naming the asset
it hit. A machine number pasted into `asset-id.txt` would otherwise file that
MACHINE under a device label the collector also owns - a link that reads as an
instrument, or as a marker, everywhere downstream.
A part marker additionally stays `partof` the operation from `machinenumber`,
because several markers serve one operation. Naming the marker in `asset-id.txt`
changes which marker asset the PC controls; it does not change which operation
that marker files under.
### PC -> printer relationship sync (REMOVED)
**A collector report has never been allowed to set a printer assignment, and as
of 2026-08-24 it no longer can.** `defaultprinter` and `printers` used to be
accepted here and written as `defaultprinter`/`connectedto` relationship rows.
That contradicted the rule stated under "Observed is not assigned" below, and
this document carried both claims at once while the code followed the wrong one.
If you are still sending either key: **your push still succeeds** - they are
ignored, and the response carries a warning naming them. Nothing else about your
payload changed. To report what a PC actually has, use
`POST /api/collector/printers`; to turn that into an assignment, a person calls
`POST /api/printers/assignments/seed-from-observed/<assetid>` after reviewing the
comparison.
### pc-type mapping (configurable per site)
### pc-type mapping (configurable per site)
@@ -306,6 +345,7 @@ The fleet's classic-ASP reporter posts form fields to
|---|---|
|---|---|
| `hostname` | `hostname` |
| `hostname` | `hostname` |
| `machineNo` | `machinenumber` |
| `machineNo` | `machinenumber` |
| `deviceId` | `deviceid` |
| `pcType` | `pctype` |
| `pcType` | `pctype` |
| `serialNumber` | `serialnumber` |
| `serialNumber` | `serialnumber` |
| `loggedInUser` | `loggedinuser` |
| `loggedInUser` | `loggedinuser` |
@@ -321,6 +361,137 @@ classic reporter posts a full `networkInterfaces` array; the collector accepts
only one `ipaddress`, so pick the corp/routable NIC (see the corp-range gate in
only one `ipaddress`, so pick the corp/routable NIC (see the corp-range gate in
the PowerShell below).
the PowerShell below).
### Printers plugin field mapping (observed print queues)
`POST /api/collector/printers`. The other half of the printer story. ShopDB has
always known what a bay SHOULD have - `GET /api/printers/for-host/<hostname>`,
applied by `Set-ShopdbPrinters.ps1` - and has never known what it actually has.
This payload is that missing half. Same dispatcher, same auth, same audit row as
the computers collector; only the payload and the plugin differ.
- Schema + apply: `get_collector_schema` / `apply_collector_payload` in
- Storage: `printerobservedqueues`, one row per observed queue, owned by the
printers plugin. It is a separate table from the assignment on purpose - see
"Observed is not assigned" below.
The identity field is `hostname`. Unlike the computers payload this one is NOT
patch-style: `queues` is required, and the reported set replaces everything
previously recorded for that host.
| Payload field | Type | Server behaviour (`apply_collector_payload`) |
|---|---|---|
| `hostname` (required) | string | Identity of the report. `COMPUTERNAME` or its FQDN; matched case-insensitively. Rows are keyed by the NAME, so a bay that reports before its PC record exists still records everything - the report resolves to an asset the moment that record appears. An unknown hostname is a warning, never an error. |
| `queues` (required) | array of objects | Every real print queue on the host. This REPLACES the host's previous set. An empty array is a valid report meaning "this bay has no queues" and clears the host's rows. A missing key is rejected (400), because one client bug that dropped the field would otherwise erase the fleet's observed state host by host. |
| `queues[].queuename` (required) | string | Windows printer name. A queue with no name is skipped with a warning; a repeated name within one report is dropped with a warning (Windows cannot hold two queues of one name). |
| `queues[].drivername` | string | Driver name verbatim, as the INF spells it. Compared against the assignment's driver to detect drift. |
| `queues[].portname` | string | Windows port name. Also tried as a match key, since a port created outside the client is usually named after the host address. |
| `queues[].portaddress` | string | `PrinterHostAddress` of a TCP/IP port - an IP or FQDN. The PRIMARY key for matching a queue to a printer asset. Omit it for a non-TCP port (USB, WSD, redirected); such a queue is still worth reporting and matches on name alone. |
| `queues[].isdefault` | boolean | True on the one queue that is the user's default. If several arrive true, the first is kept and the rest are cleared with a warning - a bay has exactly one default, and keeping both would leave a seed picking one at random. |
| `queues[].isshared` | boolean | True when the queue is shared off this PC. Recorded, not acted on. |
| `observedat` | ISO-8601 datetime | Accepted and ignored, as is any other client timestamp. The server stamps `observedat` at ingest, one value for the whole report, so a bay with a wrong clock cannot report itself fresh or stale. |
Response `data` is the standard collector shape: `action` is always `updated`
(this endpoint replaces rows and creates no asset - calling an identical
re-report `noop` would hide that the bay is still checking in), `assetid` is the
resolved PC or `null`, `extra.queuecount` is how many rows were stored, and
`warnings` carries the soft problems above.
Schema source of truth: `get_collector_schema` in `plugins/printers/plugin.py`.
If you change the payload, change it there and re-check this table.
```
POST /api/collector/printers
X-API-Key: SECRET
Content-Type: application/json
{
"hostname": "WORKSTATION01",
"queues": [
{
"queuename": "Bay Label Printer",
"drivername": "Generic / Text Only",
"portname": "IP_192.0.2.40",
"portaddress": "192.0.2.40",
"isdefault": true
},
{
"queuename": "Office Laser",
"drivername": "HP Universal Printing PCL 6",
"portname": "IP_192.0.2.41",
"portaddress": "192.0.2.41",
"isdefault": false
}
]
}
```
#### The latest report replaces the previous one
This is CURRENT STATE, not history. Each report deletes every row previously
recorded for that hostname and writes the reported set in its place, inside the
dispatcher's transaction. Nothing accumulates, "what does this bay have" is
never a question about time, and a queue removed from a bay disappears from
ShopDB on the next cycle without anyone tidying up.
That has one sharp edge, and it belongs to the client: an empty `queues` array
is a legitimate report that wipes the host's rows. A client whose enumeration
FAILED must therefore send nothing at all rather than an empty list. Reporting
nothing loses one cycle; reporting `[]` after a WMI hiccup deletes real state
and reads as a bay that lost its printers. `Report-PrintersToShopDB.ps1` tracks
this with an `$enumerated` flag and exits without posting when both the cmdlet
and the WMI fallback failed.
#### Observed is not assigned
Stated plainly, because it is the whole point of keeping two tables:
**A collector report NEVER becomes an assignment.** Nothing on this path writes
a `usesprinter` or `defaultprinter` row. The moment a drifted bay's observed
state is treated as correct, enforcement stops meaning anything - a bay that
installed the wrong printer would make itself right simply by reporting it.
Observed state becomes assigned state only when a person asks for it, through
`printers.edit`), after reviewing the comparison. That route refuses rather than
guesses: it will not seed from an ambiguous host, will not seed queues that
match no printer unless told to, and will not write an empty set.
#### Matching, and what "unknown" means
Nothing is matched at ingest - the raw observed strings are stored as reported,
and resolution to a printer asset happens at READ time, so a printer added to
ShopDB tomorrow matches yesterday's report without the bay reporting again.
At read time each queue is matched by PORT ADDRESS first (an IP or FQDN names
one device unambiguously), then by queue name. There is no fuzzier fallback: a
queue that matches nothing is reported as `unknown` rather than guessed, because
a wrong match seeds a wrong assignment, which is worse than no assignment.
#### Reading it back
| Endpoint | Purpose |
|---|---|
| `GET /api/printers/observed/<hostname>` | What the host last reported, every queue classified against the assignment it resolves to (`matching`, `drifted`, `extra`, `missing`, `unknown`), with `observedat`, a summary, and a `seedcandidate` preview. Read-only. Needs `printers.view`. |
| `POST /api/printers/assignments/seed-from-observed/<assetid>` | The one path from observed to assigned, human-triggered. Needs `printers.edit`. Normally posted against the MACHINE, so the assignment survives a reimage. |
#### Key delivery for the printers reporter
The reporter reads its server and its key from the registry the enforcement
client already provisions (`HKLM:\SOFTWARE\GE\ShopDB`, values `BaseUrl` and
`CollectorKey`), or from `-ApiUrl` / `-ApiKey` in the manifest entry's `Args`.
Nothing is baked into the script or the manifest JSON on the share - the same
rule as the computers reporter, for the same reason.
Server side, the key resolves per-plugin first: set `COLLECTOR_API_KEY_PRINTERS`
if you scope keys per collector, or rely on the shared `COLLECTOR_API_KEY`. A
site that has scoped the computers key per-plugin and set no printers key gets
401 on this endpoint until one of the two is in place. A `collector.ingest`
managed token works here exactly as it does for computers.
@@ -703,8 +877,13 @@ fleet's key is scoped to the computers collector only.
| `No collector registered for plugin computers` | 404 | The computers plugin is disabled or not loaded on that instance. Enable it. |
| `No collector registered for plugin computers` | 404 | The computers plugin is disabled or not loaded on that instance. Enable it. |
| `hostname is required` / `No data provided` | 400 | Empty body, non-JSON body, or missing identity field. Check `-ContentType 'application/json'` and that `hostname` is set. |
| `hostname is required` / `No data provided` | 400 | Empty body, non-JSON body, or missing identity field. Check `-ContentType 'application/json'` and that `hostname` is set. |
| `Internal error processing collector payload` | 500 | Generic by design - the server does not leak the cause to the caller. The real error (DB, unexpected exception) is in the server log (`Collector upsert failed for computers`). Check there. |
| `Internal error processing collector payload` | 500 | Generic by design - the server does not leak the cause to the caller. The real error (DB, unexpected exception) is in the server log (`Collector upsert failed for computers`). Check there. |
| `No collector registered for plugin printers` | 404 | The printers plugin is disabled or not loaded on that instance. Enable it. A lean site without it simply does not collect observed queues. |
| `queues is required; send an empty array for a host with no queues` | 400 | The printers payload omitted `queues` entirely. Absent and empty are not the same thing here - see "The latest report replaces the previous one". |
| A bay's observed queues all vanished | 200 | It reported `queues: []`, which legitimately clears the host. Check the client log for `posting NOTHING` (a failed enumeration correctly sends nothing) versus `reporting an empty set`. If the bay really has queues, the enumeration on that host is the thing to fix. |
| Observed queues show as `unknown` | 200 | The queue matched no printer asset by port address or by name. Add the printer to ShopDB (with its IP) and re-read - matching happens at read time, so the bay does not need to report again. Never seeded automatically, by design. |
| `warnings` present but `action` is created/updated | 200 | Soft issues only; the row WAS written. Common: unmapped `pctype` (fix the pctypemap setting), unknown `osname` (add it to the `operatingsystems` vocab), unknown app name, `pcsubtype` not stored. No action needed unless the warning matters to you. |
| `warnings` present but `action` is created/updated | 200 | Soft issues only; the row WAS written. Common: unmapped `pctype` (fix the pctypemap setting), unknown `osname` (add it to the `operatingsystems` vocab), unknown app name, `pcsubtype` not stored. No action needed unless the warning matters to you. |
Client-side log for the fleet reporter: `C:\Logs\Shopfloor\collector-YYYYMMDD.log`.
Client-side log for the fleet reporter: `C:\Logs\Shopfloor\collector-YYYYMMDD.log`;
for the printer-queue reporter: `C:\Logs\Shopfloor\report-printers-YYYYMMDD.log`.
Server-side: the Flask app log (the collector logs upsert failures and per-plugin
Server-side: the Flask app log (the collector logs upsert failures and per-plugin
| `setup_complete` | `false` | Set true once the first-run wizard finishes; gates the `/setup` route. |
| `setup_complete` | `false` | Set true once the first-run wizard finishes; gates the `/setup` route. |
| `employee_directory_mode` | `selfhosted` | `selfhosted` (tables in this app) or `external` (a separate HR database, see `EMPLOYEE_DB_*`). |
| `employee_directory_mode` | `selfhosted` | `selfhosted` (tables in this app) or `external` (a separate HR database, see `EMPLOYEE_DB_*`). |
| `usb_directory_mode` | `selfhosted` | `selfhosted` or `external` (a separate `cmmc_usb` database, see `CMMC_USB_DB_*`). |
| `usb_directory_mode` | `selfhosted` | `selfhosted` or `external` (a separate `cmmc_usb` database, see `CMMC_USB_DB_*`). |
| `site_base_url` | (empty) | Public base URL (scheme + host) for QR codes and absolute links. Blank = use the browsing origin. |
| `audit_retentiondays` | `0` | Delete audit entries older than this many days. 0 keeps everything forever. Saving a period deletes nothing by itself: purge from this page or run `flask audit purge --apply`. |
| `site_base_url` | (empty) | Public base URL for QR codes and absolute links. Scheme + host, optionally including the subpath a site is served under (`https://host/shopdb`). Blank = use the browsing origin. |
| `facility_name` | (empty) | Facility name in the dashboard header. Blank = frontend falls back to `ShopDB`. |
| `facility_name` | (empty) | Facility name in the dashboard header. Blank = frontend falls back to `ShopDB`. |
| `pc_access_domain` | `device.geaerospace.net` | Domain appended to a PC hostname for remote-access links. Blank = hostname as-is. |
| `pc_access_domain` | `device.geaerospace.net` | Domain appended to a PC hostname for remote-access links. Blank = hostname as-is. |
| `employeeid_pattern` | `^\d{9}$` | Regex a search term must match to be treated as an employee id. Invalid regex falls back to the default and never 500s. |
| `employeeid_pattern` | `^\d{9}$` | Regex a search term must match to be treated as an employee id. Invalid regex falls back to the default and never 500s. |
@@ -287,23 +288,32 @@ Three flows use it:
### audit
### audit
| Key | Default | Notes |
Retention lives in the `site` section as `audit_retentiondays`. It was
|-----|---------|-------|
documented here as `audit_retention_days` as well, which was a second key that
| `audit_retention_days` | `90` | Days to retain audit logs (0 = keep forever). |
nothing read - the settings page wrote to it and retention stayed off. One key
now, and the page writes that one.
### auth
### auth
**SAML is not implemented yet.** These keys exist so a site can be configured
ahead of the feature, and the settings page says so on screen. Nothing reads
them today: there is no SAML login route and no assertion is accepted, so
sign-in is local accounts only whatever is set here. In particular
`saml_allow_local_login` does NOT disable password login - the toggle is locked
on until there is another way in, because a control that appears to close the
local door while it stays open is worse than no control.
| Key | Default | Notes |
| Key | Default | Notes |
|-----|---------|-------|
|-----|---------|-------|
| `saml_enabled` | `false` | Enable SAML SSO. |
| `saml_enabled` | `false` | Reserved. Enable SAML SSO once it ships. |
| `saml_allow_local_login` | `true` | Allow local username/password login when SAML is on. |
| `saml_allow_local_login` | `true` | Reserved, and locked on. Local login cannot be turned off while it is the only way in. |
| `saml_auto_create_users` | `true` | Auto-create users on first SAML login. |
| `saml_auto_create_users` | `true` | Reserved. Auto-create users on first SAML login. |
| `saml_admin_group` | (empty) | SAML group name that grants the admin role. |
| `saml_admin_group` | (empty) | Reserved. SAML group name that grants the admin role. |
**Personal API tokens.** Besides login JWTs and SAML, a user may create
**Personal API tokens.** Besides login JWTs, a user may create
personal API tokens (PATs) for scripts and integrations, from Settings > API
personal API tokens (PATs) for scripts and integrations, from Settings > API
Tokens (or `POST /api/apitokens`). A PAT is sent like a JWT
Tokens (or `POST /api/apitokens`). A PAT is sent like a JWT
(`Authorization: Bearer shopdb_pat_...`), authenticates as its owning user
(`Authorization: Bearer shopdb_pat_...`), authenticates as its owning user
@@ -322,6 +332,21 @@ intersected with the owner's live permissions at use time, and suspends the
admin bypass, so it is denied on role-gated (admin-only) endpoints and on import
admin bypass, so it is denied on role-gated (admin-only) endpoints and on import
mode. Use an unscoped token for admin-only work and imports.
mode. Use an unscoped token for admin-only work and imports.
### tools
| Key | Default | Notes |
|-----|---------|-------|
| `securesend_allow_guests` | `false` | Let people with no account create Secure Send links. Public (readable before login) so the create page can show a guest the form instead of a sign-in wall. Off by default: a guest secret has no creator, so the audit trail can say when and from where but never who. Receiving a secret never needs an account either way. |
Turning this on makes creation an unauthenticated write, bounded at 10 links per
address per 10 minutes. A signed-in user still needs the `tools.secrets`
permission, so a member without it cannot create one while a guest can - opening
the door to guests is a decision about the door, not about revoking what a role
holds. The setting fails closed if it cannot be read.
See [SECURE-SEND.md](SECURE-SEND.md) for the whole feature, including why the
server cannot recover a secret and what the size limit is doing.
### identifiers (dynamic)
### identifiers (dynamic)
One boolean key per asset identifier per asset type, keyed
One boolean key per asset identifier per asset type, keyed
@@ -40,6 +40,16 @@ Recorded in the comment block in `shopdb/__init__.py`:
| 0.18.0 | Added `DISPLAY_ROLES`, `DISPLAY_ROLE_PATHS` and `normalize_display_role`, and finally exported `DashboardDefault`, which 0.17.0 imported but left out of `__all__`. The role vocabulary became the kiosk's own - `Dashboard`, `Lobby`, `3DPrintRoom` - so a plugin holding its own copy of that map reads core's instead of drifting from it | additive surface (minor) |
| 0.18.0 | Added `DISPLAY_ROLES`, `DISPLAY_ROLE_PATHS` and `normalize_display_role`, and finally exported `DashboardDefault`, which 0.17.0 imported but left out of `__all__`. The role vocabulary became the kiosk's own - `Dashboard`, `Lobby`, `3DPrintRoom` - so a plugin holding its own copy of that map reads core's instead of drifting from it | additive surface (minor) |
| 0.19.0 | **BREAKING.**`get_dashboard_widgets` returns DATA AND SHAPE, not a component name. The old shape (`name` + `component` + `size`) named a Vue component per widget, which cannot survive a lean build - a plugin's component may never be staged into the frontend bundle (ADR-013) - and five plugins were declaring widgets that pointed at components nobody had written. A card now declares `id` / `title` / `endpoint` / `render` / `severity` / `permission` / `empty` / `position`; see PLUGIN-HOOKS.md. Also added `send_upload` so a plugin serving user-supplied bytes gets the headers that keep an SVG from running as script | **contract change (minor, pre-1.0)** |
| 0.19.0 | **BREAKING.**`get_dashboard_widgets` returns DATA AND SHAPE, not a component name. The old shape (`name` + `component` + `size`) named a Vue component per widget, which cannot survive a lean build - a plugin's component may never be staged into the frontend bundle (ADR-013) - and five plugins were declaring widgets that pointed at components nobody had written. A card now declares `id` / `title` / `endpoint` / `render` / `severity` / `permission` / `empty` / `position`; see PLUGIN-HOOKS.md. Also added `send_upload` so a plugin serving user-supplied bytes gets the headers that keep an SVG from running as script | **contract change (minor, pre-1.0)** |
| 0.20.0 | The `Asset` model gained `levelid` (which floor plan its `mapx`/`mapy` are pixels of) and `mapverifiedat`; `Location` gained `levelid` too, and `resolve_asset_position` now returns the `levelid` belonging to whichever source supplied the coordinates. A plugin that writes a map position MUST write the level with it - a position without one renders as "level unknown" rather than being drawn on the default drawing, because a marker on the wrong floor plan looks correct (ADR-017) | additive surface (minor) |
| 0.20.0 | The `Asset` model gained `levelid` (which floor plan its `mapx`/`mapy` are pixels of) and `mapverifiedat`; `Location` gained `levelid` too, and `resolve_asset_position` now returns the `levelid` belonging to whichever source supplied the coordinates. A plugin that writes a map position MUST write the level with it - a position without one renders as "level unknown" rather than being drawn on the default drawing, because a marker on the wrong floor plan looks correct (ADR-017) | additive surface (minor) |
| 0.21.0 | Added the `get_guest_grantable_permissions` hook: a plugin names the WRITE permissions a site may grant to the seeded `guest` role, which is who anyone without a token now holds. Reads (`.view`/`.read`) are grantable without a plugin saying anything; anything else is refused. Also exposed `require_any_permission` on `shopdb.api` for a read two different rights should reach | additive optional hook + surface (minor) |
| 0.22.0 | Added the `register_import_hook` surface: a plugin owns a VIRTUAL COLUMN on an importable CSV table - a header the bulk importer accepts but never writes - plus a post-write row hook that turns the value into a plugin concept. The computers plugin uses it for `machineassetnumber` on `computers.csv`, which builds the PC->machine controls link, so a site writes the machine number in the PC's own row instead of hand-building a separate relationships sheet. Core never learns what the column means. `RowWarning` is exported with it so a hook can report a soft failure without aborting the import | additive surface (minor) |
| 0.23.0 | Added `mint_service_token`, `list_service_tokens` and `revoke_service_tokens` so a plugin can provision, list and revoke a single-scope managed service token without importing core token internals. The endpoints plugin uses them for a UI-managed, regenerable agent key scoped to `collector.ingest.endpoints` alone - narrow on purpose, so rotating or revoking it cannot disturb the computers or GE-Enforce collectors, and revocation is an instant kill switch on agent ingest | additive surface (minor) |
| 0.24.0 | Added `collector_authorized(pluginname)` so a plugin can gate an agent-facing route that lives OUTSIDE the generic collector blueprint on the same per-plugin key and kill switch the ingest route uses. The endpoints command channel needs it: the agent long-polls `/api/endpoints/commands/next` and posts results back, and those are agent callbacks rather than JWT routes. Without it a plugin would reach into core collector internals to authorize them | additive surface (minor) |
| 0.25.0 | Added `caller_holds(permission)`, which answers whether the current caller holds a permission WITHOUT refusing the request - the complement to `require_permission` for a route that serves everyone but should only offer what the caller can act on. The endpoints services panel is the case: it emitted Start/Stop row actions to every reader, because the emission never consulted the caller. Clicking them was always refused server-side, so nothing was exploitable, but offering a read-only viewer controls over a shop-floor PC is the wrong signal, and it becomes a real problem once a site grants that panel to guests. Guests are included in the answer, since a signed-out visitor holds whatever the guest role was granted | additive surface (minor) |
| 0.26.0 | Dashboard cards may declare `shortname`. The dashboard merged its per-source cards into one worst-first queue, and a card's `title` is written as a heading ("Endpoints with crashes or shutdowns") which as a row badge ran wider than the hostname it qualified. Optional, defaults to the title, so a plugin that does not declare one is unaffected | additive field (minor) |
| 0.27.0 | Added `Script` so a plugin can read the ShopDB-hosted script library that backs endpoint `run_script` delivery without importing core model internals. The endpoints plugin resolves a delivered script by name and serves its body by digest to the agent | additive surface (minor) |
| 0.29.0 | Added an optional `group` to `get_asset_panels`, naming which detail-page tab a panel belongs to: `details` (the default) / `monitors` / `software` / `activity`. The detail pages are tabbed, and a page that placed panels by knowing their ids would defeat the hook. Additive: an absent or unrecognised group resolves to `details` server-side, so every existing plugin keeps working with no change | additive surface (minor) |
| 0.28.0 | Added `client_ip` and `strip_port` so a plugin does not hand-roll caller-IP extraction. Two plugins had, and both read the first `X-Forwarded-For` hop, which the caller supplies - so a rate limit keyed on it was defeated by rotating the header. The helper resolves from `remote_addr`, which ProxyFix has already filled from the header when `TRUSTED_PROXY_COUNT` says a proxy is really in front | additive surface (minor) |
| 0.31.0 | The asset-panel vocabulary a plugin may declare grew four optional things: two more tab groups, `serviceprocess` and `reports`, beside the 0.29.0 set; a `format: 'datetime'` on a column or field, so a panel stops shipping pre-formatted date strings the browser cannot re-localise; a `map.body` key, so a keyvalue field can name where its value lives in a nested payload; and a `tabs` section that carries `columns`/`rows`/`empty` and renders as a table, so one card can hold two tables that share no key (volume capacity and physical drive health are the case). All optional and all resolved server-side when absent, so a plugin declaring none of them is unaffected | additive surface (minor) |
docker compose exec -u root api chown -R shopdb:shopdb /app/instance
docker compose restart api
```
The `chown` is not optional. `docker compose cp` writes the files with the
copying user's numeric uid, which is only `shopdb` by coincidence if your host
account happens to be uid 1000. Get it wrong and the site reads its restored
files fine and cannot write new ones.
Then run the migrations and confirm the plugins came back:
```bash
docker compose exec api flask db upgrade
docker compose exec api flask plugin upgrade-all
docker compose exec api flask db current # must match `flask db heads`
docker compose exec api flask plugin list # the site's plugins, enabled
```
Restore `instance/` BEFORE `plugin upgrade-all`: that command works from
`plugins.json`, so running it against an empty instance directory upgrades
nothing and reports success.
The symptom of having missed this is a site that comes back with its plugins
disabled and image URLs that 404: the MySQL rows survived, the files did not.
Re-enabling by hand works, but `flask plugin apply-profile <profile.json>` puts
the same list back in one command and is the thing to keep in version control.
The framework's `__contract_version__` may have moved. Check `docs/adr/` for any new ADRs since the last update. If an ADR introduces a breaking change, the upgrade may require coordinated work; the ADR's "Consequences" section documents it. See [docs/UPGRADE.md](UPGRADE.md) for the full upgrade procedure, including re-seeding and the v0.5+ floor-plan note.
The framework's `__contract_version__` may have moved. Check `docs/adr/` for any new ADRs since the last update. If an ADR introduces a breaking change, the upgrade may require coordinated work; the ADR's "Consequences" section documents it. See [docs/UPGRADE.md](UPGRADE.md) for the full upgrade procedure, including re-seeding and the v0.5+ floor-plan note.
On Windows, install all of them with winget (accept each license, then
On Windows, install all of them with winget (accept each license, then
@@ -293,7 +293,7 @@ section of the plugin lab for the full review checklist.
| --- | --- |
| --- | --- |
| Frontend loads but every API call fails / CORS error | backend not on 5001 (`flask run --port 5001`), or `CORS_ORIGINS` missing `http://localhost:5173`. |
| Frontend loads but every API call fails / CORS error | backend not on 5001 (`flask run --port 5001`), or `CORS_ORIGINS` missing `http://localhost:5173`. |
| App refuses to boot in production config | a required `.env` var (`SECRET_KEY`, `JWT_SECRET_KEY`, `DATABASE_URL`, `CORS_ORIGINS`) missing or a dev default. |
| App refuses to boot in production config | a required `.env` var (`SECRET_KEY`, `JWT_SECRET_KEY`, `DATABASE_URL`, `CORS_ORIGINS`) missing or a dev default. |
| `flask db upgrade` error 1071 (key too long) | MySQL 5.6 without the `innodb_large_prefix`/Barracuda flags; use MySQL 8 for dev. |
| `flask db upgrade` / `flask plugin upgrade-all` error 1071 (key too long) | MySQL 5.6 on the default Antelope file format, where an index prefix caps at 767 bytes - one utf8mb4 `VARCHAR(255)` is 1020. Start mysqld with `--innodb-file-format=Barracuda --innodb-large-prefix=ON` (both are also settable live with `SET GLOBAL`), keep `?charset=utf8mb4` on `DATABASE_URL`, and note that tables CREATED before those flags were on are silently `ROW_FORMAT=Compact` rather than `DYNAMIC` - 5.6 downgrades the row format with a warning instead of an error, so they keep the 767-byte cap until rebuilt with `ALTER TABLE <t> ROW_FORMAT=DYNAMIC`. MySQL 5.7+ and 8.0 need none of this. |
| Nav missing Machines/PCs/... | plugins not installed/enabled (step 2b), or the backend not restarted after enabling. |
| Nav missing Machines/PCs/... | plugins not installed/enabled (step 2b), or the backend not restarted after enabling. |
| "No time zone found with key America/New_York" | `tzdata` not installed - `pip install -r requirements.txt` includes it. |
| "No time zone found with key America/New_York" | `tzdata` not installed - `pip install -r requirements.txt` includes it. |
| npm/git/pip SSL error (`UNABLE_TO_GET_ISSUER_CERT_LOCALLY`, `unable to get local issuer certificate`) | corp proxy (Zscaler) intercepts HTTPS - point each tool at the corp root CA. See section 0b. |
| npm/git/pip SSL error (`UNABLE_TO_GET_ISSUER_CERT_LOCALLY`, `unable to get local issuer certificate`) | corp proxy (Zscaler) intercepts HTTPS - point each tool at the corp root CA. See section 0b. |
@@ -17,7 +17,9 @@ For the architectural decisions behind the contract, see [docs/adr/](../docs/adr
flask plugin new cameras --description "Tracks shop-floor surveillance cameras"
flask plugin new cameras --description "Tracks shop-floor surveillance cameras"
```
```
Output: `plugins/cameras/` with manifest, plugin class, example model, example routes, schemas stub, tests, a README, and a paste-in `frontend-api-snippet.js`. When a `frontend/src/` tree is present, it also writes the frontend starting points: `frontend/src/views/cameras/CamerasList.vue`, `CamerasDetail.vue`, `CamerasForm.vue`, and the auto-discovered route file `frontend/src/router/routes/cameras.js`. (A plugin developed in its own repo, with no frontend tree, gets the backend skeleton plus the snippet only.)
Output: `plugins/cameras/` with manifest, plugin class, example model, example routes, schemas stub, tests, a README, and a paste-in `frontend-api-snippet.js`. It also writes the frontend starting points, and they live **inside the plugin**: `plugins/cameras/frontend/routes.js` and `plugins/cameras/frontend/views/CamerasList.vue`, `CamerasDetail.vue`, `CamerasForm.vue`.
Nothing is written into `frontend/src/`. The plugin owns its frontend because that is what makes it prunable: `scripts/stage-frontend.mjs` copies each plugin's `frontend/` into `frontend/src/.plugins-staged/<name>/` and aggregates the route modules into `routes.gen.js`, so a plugin left out of `SITE_PLUGINS` simply is not copied and its frontend leaves the build with it (ADR-013). A plugin developed in its own repo gets exactly this same layout, and its frontend starts working the moment the directory is dropped into a host repo's `plugins/`.
The generated plugin already passes the framework's contract tests. Verify before editing:
The generated plugin already passes the framework's contract tests. Verify before editing:
@@ -142,28 +144,35 @@ Each hook has a default that does nothing. Override only what your plugin needs.
The scaffold generates the frontend starting points too: three views, a route file, and a paste-in api-client snippet (see Step 1). They build and run out of the box against the example model, so `npm run build` is green immediately after scaffolding. The list below is what you finish by hand once the views exist. Copy patterns from the closest bundled plugin (`network` is the cleanest) as you flesh them out, and work through these in order:
The scaffold generates the frontend starting points too: three views, a route file, and a paste-in api-client snippet (see Step 1). They build and run out of the box against the example model, so `npm run build` is green immediately after scaffolding. The list below is what you finish by hand once the views exist. Copy patterns from the closest bundled plugin (`network` is the cleanest) as you flesh them out, and work through these in order:
1.**View files** - the scaffold created `frontend/src/views/cameras/CamerasList.vue`, `CamerasDetail.vue`, `CamerasForm.vue` from the example model. Replace the `examplefield` columns and inputs with your domain fields. Keep the global `.filters` / `.form-control` / `.card` styles; do not invent per-page input styling.
1.**View files** - the scaffold created `plugins/cameras/frontend/views/CamerasList.vue`, `CamerasDetail.vue`, `CamerasForm.vue` from the example model. Replace the `examplefield` columns and inputs with your domain fields. Keep the global `.filters` / `.form-control` / `.card` styles; do not invent per-page input styling.
2.**Route file** - the scaffold created `frontend/src/router/routes/cameras.js` exporting a route array. The router auto-discovers every file in `routes/` via `import.meta.glob`, so no registration edit is needed. Every route is already tagged with `meta: { plugin: 'cameras' }` - the ADR-009 guard redirects to the dashboard when the backend plugin is disabled - and the form routes already carry `requiresAuth: true`:
**Imports have one rule, and staging is the reason.** Anything from the CORE tree uses the `@` alias (`@/api`, `@/components/PaginationBar.vue`, `@/stores/auth`); anything inside your own plugin is relative (`./views/...`). A relative path into the core tree such as `../../api` is correct where the file sits in your plugin and wrong once staged into `.plugins-staged/cameras/`, and the failure is a build error rather than anything subtle.
2.**Route file** - the scaffold created `plugins/cameras/frontend/routes.js` exporting a route array plus a `toplevel` array. `stage-frontend.mjs` aggregates both into `routes.gen.js`, which the router spreads in, so no registration edit is needed. Every route is already tagged with `meta: { plugin: 'cameras' }` - the ADR-009 guard redirects to the dashboard when the backend plugin is disabled - and the form routes already carry `requiresAuth: true`:
// Full-screen routes that render OUTSIDE AppLayout (kiosk, print pages).
// Export it even when empty: routes.gen.js reads `.toplevel` off every staged
// module, and a missing export is a Rollup warning under strict ESM.
exportconsttoplevel=[]
```
```
3.**API client** - the scaffolded views ship with an inline `camerasApi` client so they run standalone. To graduate to the shared module, paste the generated `plugins/cameras/frontend-api-snippet.js` block into `frontend/src/api/index.js`, then delete the inline const in each view and `import { camerasApi } from '../../api'` instead. The snippet already matches the existing blocks' shape (`list(params)`, `get(id)`, `create(data)`, `update(id, data)`, `remove(id)`).
3.**API client** - the scaffolded views ship with an inline `camerasApi` client so they run standalone. To graduate to the shared module, paste the generated `plugins/cameras/frontend-api-snippet.js` block into `frontend/src/api/index.js`, then delete the inline const in each view and `import { camerasApi } from '@/api'` instead. The snippet already matches the existing blocks' shape (`list(params)`, `get(id)`, `create(data)`, `update(id, data)`, `remove(id)`).
4.**Sidebar entry** - implement `get_navigation_items` on the plugin class. No frontend edit: the sidebar builds itself from `/api/dashboard/navigation`.
4.**Sidebar entry** - implement `get_navigation_items` on the plugin class. No frontend edit: the sidebar builds itself from `/api/dashboard/navigation`.
@@ -12,7 +12,7 @@ These plugins are in `plugins/` in this repo. Enable per site with `flask plugin
| `computers` | Shop-floor PCs and engineering workstations | Fed by the PXE pipeline collector per [ADR-006](adr/ADR-006-collector-contract.md). |
| `computers` | Shop-floor PCs and engineering workstations | Fed by the PXE pipeline collector per [ADR-006](adr/ADR-006-collector-contract.md). |
| `printers` | Network and shop-floor printers | Public installer map page + fleet install contract (`/api/printers/install-list`, `/pc-default`, `/install-batch`; see [PRINTER-INSTALLER.md](PRINTER-INSTALLER.md)). Optional Zabbix integration for supply tracking. Legacy `PrinterData` retiring per ADR-001. |
| `printers` | Network and shop-floor printers | Public installer map page + fleet install contract (`/api/printers/install-list`, `/pc-default`, `/install-batch`; see [PRINTER-INSTALLER.md](PRINTER-INSTALLER.md)). Optional Zabbix integration for supply tracking. Legacy `PrinterData` retiring per ADR-001. |
| `network` | Switches, routers, access points, IDFs as locations | Asset-only; cleanest of the bundled set. |
| `network` | Switches, routers, access points, IDFs as locations | Asset-only; cleanest of the bundled set. |
| `usb` | USB devices issued to shop-floor users | Lightweight checkout / check-in. |
| `usb` | USB devices issued to shop-floor users | Lightweight checkout / check-in, self-hosted tables or an existing `cmmc_usb` database. Optional logged-out kiosk (`/usb-kiosk`, off by default) so somebody at the drive cabinet can take and return a drive without a ShopDB login, attributed to a scanned badge. |
| `notifications` | Shop-floor notifications, recognitions, kiosk feed | Used by `ShopfloorDashboard.vue`. |
| `notifications` | Shop-floor notifications, recognitions, kiosk feed | Used by `ShopfloorDashboard.vue`. |
| `measuringtools` | Metrology and inspection instruments: calipers, micrometers, thread/bore/height gages, indicators | Per [ADR-005](adr/ADR-005-equipment-vs-measuringtools.md). Calibration lifecycle with derived status. First plugin built on the matured scaffold; its walkthrough is [PLUGIN-GUIDE.md](PLUGIN-GUIDE.md). Ships `default_enabled: false`. |
| `measuringtools` | Metrology and inspection instruments: calipers, micrometers, thread/bore/height gages, indicators | Per [ADR-005](adr/ADR-005-equipment-vs-measuringtools.md). Calibration lifecycle with derived status. First plugin built on the matured scaffold; its walkthrough is [PLUGIN-GUIDE.md](PLUGIN-GUIDE.md). Ships `default_enabled: false`. |
| `employees` | Read-only employee directory lookup | Backed by a separate HR database. Ships `default_enabled: false`. |
| `employees` | Read-only employee directory lookup | Backed by a separate HR database. Ships `default_enabled: false`. |
@@ -20,7 +20,11 @@ These plugins are in `plugins/` in this repo. Enable per site with `flask plugin
| `knowledgebase` | Knowledge Base articles linking to external resources | Lightweight article store. |
| `knowledgebase` | Knowledge Base articles linking to external resources | Lightweight article store. |
| `slides` | Slides for the lobby display and shop-floor screensaver | Upload / reorder / delete per surface. Management gated on the `slides.manage` permission, grantable to non-admin curators; display routes are public. |
| `slides` | Slides for the lobby display and shop-floor screensaver | Upload / reorder / delete per surface. Management gated on the `slides.manage` permission, grantable to non-admin curators; display routes are public. |
| `tools` | Technician utilities: barcode and QR label generation, unit conversions, Secure Send | Mostly browser-side by design, so an air-gapped bench gets them with no network. Secure Send is the exception and is why this plugin has a blueprint, a table and a migration chain: see [SECURE-SEND.md](SECURE-SEND.md). |
| `warranty` | Asset warranty tracking | Manual entry now, Dell / Lenovo / HP provider lookups later. Derived coverage status with report buckets. |
| `warranty` | Asset warranty tracking | Manual entry now, Dell / Lenovo / HP provider lookups later. Derived coverage status with report buckets. |
| `guides` | Step-by-step procedures written in ShopDB: typed blocks, pasted screenshots, live checkboxes, tags, drafts and revision history | Content is stored as typed blocks rather than markup, so the same guide renders to screen, dark mode and paper. Readers see the PUBLISHED revision only - a technician following a procedure must not have it change under them mid-task. Checkbox ticks are per-viewer and never edit the guide. See [GUIDES.md](GUIDES.md). |
| `endpoints` | Live endpoint telemetry for shop-floor PCs: liveness, services, processes, inventory, and curated Windows event records, reported by the ShopDB agent | Depends on `computers` - an endpoint is telemetry ABOUT a PC asset, never a second record of it. Adds the Fleet console plus the live panels on a PC's detail page. Four separate permissions: `endpoints.view` to read, `endpoints.fleet` for the fleet-wide console, `endpoints.command` to act on one endpoint, and `endpoints.restart` for a reboot. They are split because read access is grantable to guests and the command channel is not. Ingest is not a route in this plugin: agent reports arrive on the generic collector route and authorize on a service token scoped `collector.ingest.endpoints`, not a user JWT, per [ADR-006](adr/ADR-006-collector-contract.md). That scope doubles as the kill switch for the agent key. See [FLEET-ARCHITECTURE.md](FLEET-ARCHITECTURE.md). |
| `backups` | Per-asset configuration backups with revision history: NTLARS/DNC registry and part marker | Kind-pluggable per [BACKUP-KINDS.md](BACKUP-KINDS.md); a kind is one class in `services/registry.py`. Parseable kinds store a JSON projection and re-render to the native format on download; opaque vendor formats stay on the SFLD share with ShopDB holding metadata and the UNC pointer. Fed by the collector, so ingest authorizes on a `collector.ingest` service token, not a user JWT. `backups.view` and `backups.download` are separate permissions because a `.reg` is a restorable artifact. Not to be confused with [BACKUP-RESTORE.md](BACKUP-RESTORE.md), which is the ShopDB database itself. |
4.`C:\Enrollment\cmm\cmmid.txt` as `machineNo` (grandfathered CMM bays)
Then, and only then, the file branches can be deleted one at a time, newest
first, each when no bay has reported through it for a full inventory cycle.
**`cmmid.txt` survives phase 3 regardless**, as the CMM backup-set path key. It
is not on the deletion list; only its identity use is.
### PXE and GE-Enforce prefer it too, not just the collector
Phase 3 is not only "the collector reads a registry value". All three writers
and readers move together:
| Component | Change |
|---|---|
| PXE / imaging | writes `DeviceId` at enrollment instead of (or as well as) the file |
| GE-Enforce | resolves device identity registry-first, mirroring how it already resolves the machine number |
| collector | reads the registry ahead of the file |
**GE-Enforce already made this exact decision for the machine number, and wrote
down why.** `Install-FromManifest.ps1` resolves `TargetMachineNumbers` gating
from the eDNC registry BEFORE `C:\Enrollment\machine-number.txt`, because:
> the imaging-time `machine-number.txt` is written ONCE by `startnet.cmd` at the
> PXE menu and is NOT updated on reassignment, so it goes stale.
**`asset-id.txt` inherits that defect exactly.** It is written once at imaging
and nothing updates it when a device is swapped, moved between bays, or
replaced. The bay then reports an identity that was true on imaging day, and the
collector - which now trusts that value FIRST, ahead of every other source -
adopts the wrong device with no warning, because the value still resolves.
That makes phase 3 a correctness fix rather than a tidy-up. A file that only
imaging writes cannot be the top of a resolution order for something that
changes during a PC's life.
It also implies a reassignment path, the way `Set-MachineNumber` /
`Update-MachineNumber` exist for the bay number: something that rewrites
`DeviceId` when a device is swapped, and migrates any per-device state with it.
Without that, phase 3 moves the staleness from a file to a registry value rather
than curing it.
### Doing phase 3 early is cheaper than doing it late
If the registry read lands before imaging is changed, phase 2 can write the
REGISTRY for new bays and skip the file era entirely for them. The file then
exists only for bays staged in the window between phase 1 and phase 2, and the
eventual cleanup is deleting a code branch rather than sweeping the fleet.
The cost of adding the read now is one branch in the client and one row in the
docs. The cost of adding it after every bay has a file is a migration.
## 5. What this does not change
-`machinenumber` and its GE-Enforce gating. Untouched in all three phases.
- A part marker still files `partof` the operation from `machinenumber`. The
device identity says WHICH marker the PC controls, not which operation that
marker belongs to.
- Minting. It stays as the last resort for a bay that names nothing, because a
device nobody declared is still better recorded than not recorded. Anything
minted should be treated as a placeholder to be reconciled.
## 6. Before phase 2: find what is already duplicated
Phase 1 stops NEW duplicates. It does not merge the ones a PC swap has already
created. Before imaging starts writing files, run a read-only query for device
assets whose controlling PC no longer exists, or whose asset number carries a
hostname that is not a current PC. That list is the reconciliation backlog, and
it is much easier to read before the fleet starts reporting new identities over
the top of it.
## 7. Rollback
Each phase is independently reversible because nothing is removed until the
phase after it proves the replacement:
- Phase 1: stop sending `deviceid` and every previous fallback still runs.
- Phase 2: stop writing `asset-id.txt` and bays fall back to what they used
before.
- Phase 3: stop writing the registry value and bays fall back to the file.
The only irreversible step is deleting a fallback branch, which is deliberately
outside all three phases.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
