Two tracked files had been edited directly in the work-PC relay clone, which
only fetches bundles and pushes them to GitHub. Nothing there reaches gitea, and
the next bundle pull would have overwritten both silently. Brought across here,
with three changes to what was written.
Kiosk links now need the route AND the plugin. The edit swapped hasRoute for
isPluginEnabled, which trades one wrong answer for another: a full build stages
every route, so a site that turns printedparts or usb OFF still saw a link to a
kiosk its backend refuses - the symptom being fixed - but isPluginEnabled fails
OPEN when it cannot reach the plugin list, and a copied registry can report a
plugin that was never staged into this build. That is what the comment above
those lines was already warning about. Both tests together answer both states.
The wizard shows the filename it is uploading. The browser puts it in chrome the
full-screen wizard hides, so an operator picking a file got no confirmation it
was the one being sent.
Image previews get a cache-buster. The stored path does not change between
uploads, so the preview showed the PREVIOUS logo or blueprint and the upload
read as having silently failed. Display only - saveStep persists form values,
never these, so no cache-busted URL is ever written to a setting.
The wizard also loads the current site logo on mount, so re-running it does not
present an empty slot as though branding had never been set. The shipped-asset
fallback stays a root-absolute path: the template applies withBase, and prefixing
it here as well produced /shopdb/shopdb/ge-aerospace-logo.svg on a subpath mount.
Frontend 391 passing.
Scope: product
A local 'dotnet build' leaves agent/*/obj and agent/*/bin full of absolute
host paths. They are gitignored, but the export rsyncs the WORKING TREE, so
.gitignore does not apply and the scrub gate stopped the whole export - which
is what it is for, but it made publication depend on whoever built the agent
last having cleaned up.
Same shape as the stray .env.bak the excludes already document. Note the trap
that follows from it: an --exclude also PROTECTS a path from --delete, so the
copies the first failed run had already placed in the publication tree had to
be removed by hand before the next run could pass.
Scope: product
Three defects that all shared a shape: the restore path looked checked and was
not.
An unreadable data revision was treated as a legacy set. backup.sh discards the
mysql error when reading alembic_version, so a set could be written with an
empty datarev; the compatibility check then fell through to the code-to-code
head comparison - the very check datarev was introduced to replace - and badged
the set restorable on no evidence at all. A manifest that CARRIES the key and
left it empty is now refused, while one with no key at all is still the genuine
legacy case and keeps the fallback. backup.sh records the failure in run history
rather than passing it over in a log line, since the set it just wrote cannot be
restored.
Dumps were bound to the schema they came from. mysqldump --databases bakes
CREATE DATABASE and USE <name> into the file, so restoring onto a site whose
MYSQL_DATABASE differs loaded every row into the dump's schema, left the live
database untouched, and reported success. New sets are dumped without it and
load into the schema named at restore time. Existing sets are still bound, so
the gate reads which schema they name and refuses a mismatch instead of
restoring beside the live data. Also dropped --no-tablespaces into the main
dump: it needs the PROCESS privilege the app user does not have on MySQL 8.
The gate's own test carried a copy of the gate. test-restore-datarev.sh
re-implemented the predicate in its own shell functions, so it would have stayed
green through any drift in restore.sh - including the plugin-chain hole fixed in
the previous commit, which it could never have caught. restore.sh now takes
RESTORE_CHECK_ONLY, which stops it immediately after the gates having touched
nothing, plus overridable paths for the migration and plugin trees. The test
drives the shipped script and asserts on its exit code, so the verdict under
test IS the verdict that ships. Confirmed non-vacuous by planting a violation:
disabling the plugin revision check turns the suite red.
Nine cases now, up from five, covering both plugin-chain directions, a plugin
the build does not have, and the schema binding.
Moving the schema check earlier was prompted by that test: it had been placed
after the safety copy, so it needed a live database to reach a verdict it can
reach for free. It sits with the other gates now and stops before anything is
written.
Round-tripped against a real stack after the dump format change: backup, mutate,
restore, data back to backup-time state, and the new dumps confirmed to carry no
schema name.
Suite 2607 passed / 9 skipped.
Scope: product
The compatibility gate read alembic_version and nothing else, but plugin schemas
move independently of core. Roll the code back to an older plugin revision and
the core head can be identical, so a dump carrying plugin tables the code has
never heard of passed the gate, loaded - destroying what was there - and only
then failed in 'flask plugin upgrade-all' with "Can't locate revision". A failed
restore that has already replaced what it was restoring over is the worst
outcome this feature can produce, and it was reachable from an ordinary rollback.
Backups now stamp the applied revision of every alembic_version_<plugin> table
alongside the core one, read from the dumped database the same way. restore.sh
checks each against that plugin's own versions directory and refuses before
touching anything, naming the plugin and the revision to deploy past. A plugin
in the dump that this build does not have at all is refused for the same reason.
The list endpoint's compatibility badge asks the same question, so the UI cannot
offer a set the restore will reject.
Sets taken before the stamp existed carry no dataplugins and keep their old
core-only verdict. Their plugin revisions are unknowable, and making every
existing set unrestorable to close this would be a worse trade than the gap.
Proven against a real stack, reproducing the exact case: core head identical,
endpoints data stamped at a revision the build does not have. The restore
refused, the live row was still there afterwards, and it stopped early enough
that the pre-restore safety copy was never needed. The same set with its true
revision restored normally.
Suite 2605 passed / 9 skipped, with the gate covered both ways plus the legacy
fallback.
Scope: product
Closes the three defects in the imported restore path, at Ben's request. Each
was proven against a real stack before and after: two backup sets with different
data and different instance files, then a restore of the OLDER one, which is the
case that was broken.
A restore no longer depends on a tab staying open. The backup sidecar has no
Flask, so it loads the dump and leaves a sentinel for the api container to run
the migrations. The only thing that ever acted on that sentinel was the poll
loop of the modal that queued the restore, so closing the tab - or letting a
token expire - during the minutes a load takes left the data at the old revision
under new code, every query on a newer column failing, and nothing retrying. A
watcher in the api container now claims and runs it. The endpoint no longer runs
the upgrade at all; it only makes sure something is pending, which is also how a
failed upgrade is retried. That takes Alembic out of the request path, where it
could outrun the gunicorn worker timeout and be killed mid-chain, leaving a
half-applied revision the next attempt trips over.
The claim is a rename, because rename(2) is atomic: several workers and a
browser poll can race the same sentinel and exactly one wins. A failed upgrade
parks itself as .upgrade-failed rather than re-arming, since putting it back
would rerun the same broken chain on every poll forever.
Instance files come from the set being restored. The restic call asked for
'latest' regardless of which set was named, so restoring anything but the newest
paired that set's DATABASE with the NEWEST snapshot's uploads and plugins.json -
a mismatch nothing downstream could detect. Now scoped by the set's own tag,
which backup.sh has always written. The tree is replaced rather than restored
over, so files created after the snapshot are gone afterwards: a restore has to
put the site back to the snapshot, not merge into it. Done as a staged swap
rather than 'restic restore --delete --target /', because that aims a delete
pass at the container root and the cost of getting its filter wrong is the whole
filesystem.
Nothing destructive happens before a copy is taken. The dump carries DROP TABLE
per table with no transaction around it, and the instance tree is replaced
wholesale, so a wrong set or a failure part-way through left nothing to go back
to. restore.sh now writes a pre-restore-<stamp>/ set - dump, instance tarball
and a README saying what it is and how to load it - and refuses to touch
anything if that copy cannot be written.
Two more found while proving the above, both of which stopped the overlay
working at all:
* Every script under scripts/backup/ was committed non-executable, while every
other .sh in the repo is 755. The compose entrypoints are
/opt/backup/watch.sh and backup.sh from a read-only bind mount, so the backup
and watcher containers could not start from a clean checkout on any machine,
and nothing could fix it at runtime.
* Dockerfile.backup could not build without the corporate CA staged. A plain
COPY of the gitignored cert fails outright; the trailing-glob form does too,
because BuildKit fails a COPY whose pattern matches nothing - which is exactly
the case being handled. Copying the directory and testing for the file inside
it is the form that works both ways. Verified by building the image with no
cert present.
Suite 2600 passed / 9 skipped. The finalizer has unit tests covering the atomic
claim, the parked failure and the endpoint no longer running the upgrade.
Scope: product
Six findings from a four-agent review of the imported work, plus the contract
bump the import owed. Nothing here changes what the imported features do.
The contract version was not bumped for a change that widens what a plugin may
declare. The asset-panel vocabulary gained two tab groups (serviceprocess,
reports), a datetime column format, a map.body key and a tabs section that
renders as a table. 0.29.0 bumped for exactly this class of change when the
optional group was added, so this is 0.31.0 with a ledger row. All four are
optional and resolve server-side when absent, so a plugin declaring none of them
is unaffected.
Site Backups now answers honestly where it has nothing behind it. The overlay is
a compose profile: a separate container writes sets onto a shared volume and the
api container only reads them. A Windows/IIS install runs no such container and
the default root resolves to a drive-letter path nothing creates, so Back Up Now
either failed on PermissionError or reported a run queued that would never
happen, under a banner reading "No verified backup yet" - which describes a
backup that failed, not a feature that was never installed. The three write
routes refuse with the same 404 shape the configuration-policy routes use for a
disabled feature, the list and status routes report whether a store exists, and
the page says so instead of offering buttons.
Two listAll callers still read a response that is not there. listAll resolves to
a plain array through fetchAllPages, so response.data.data throws: PC Types
settings rendered empty, and in Configuration Policies the throw happened inside
the shared try, so PC types, applications, the action catalog and monitors were
all left unassigned by one bad read.
The agent package upload cannot succeed on IIS as shipped. MAX_CONTENT_LENGTH
went to 256 MB to admit the ~69 MB tray binary, but IIS request filtering caps a
body at 30,000,000 bytes and answers 413 before waitress sees the request, so
web.config needs the matching requestLimits.
The backup image could not be built by anyone outside a TLS-intercepting
network. Dockerfile.backup did a plain COPY of a gitignored corporate root CA,
and a COPY of a missing file fails the build. The trailing glob makes it a no-op
when the cert is absent.
endpointstates.domain was the one new string column written without a clamp.
Under STRICT_TRANS_TABLES an over-long value raises Data too long and aborts the
whole check-in, so a single malformed report would stop a machine reporting
telemetry and events entirely.
Not fixed here, and left with upstream: the restore path finalizes its schema
upgrade only from a browser poll loop, so closing the tab mid-restore leaves the
data behind the code; it restores instance files from the newest snapshot rather
than the set that was asked for; and it takes no safety dump before the
destructive load. All three are in the Docker-only overlay, which this site does
not run, and none can be validated here.
Scope: product
Ports 37 of Benjamin Baker's commits from the wilm branch, mirrored from the
corporate GitHub at 7905917c. wilm rides the publication lineage and shares no
ancestor with wj, so this is a filtered tree patch rather than a merge:
94 files, and every path the publication export strips (docs/, tools/, mcp/,
CLAUDE.md, start-*.sh, .gitea/, the three excluded tests) is held back, because
a raw diff would delete each of them from this tree.
What comes across:
* Endpoint telemetry, phase 2. Physical disk health from two sources, NIC link
state, boot type, Defender health, processor/memory/chassis inventory, the
listening TCP set, mapped network drives and Windows Update history. Five
additive migrations, 0012 through 0016, off the 0011 head.
* Mapped drives are collected by a probe running in the interactive user's
session, because they do not exist in the agent's session 0. Two Windows
sources can disagree about one drive and the worse status wins, so a
half-broken share is never rendered as healthy.
* Windows events are reported once each, keyed on the channel's own record id,
instead of being re-sent whenever a report overlapped the last one.
* PC Detail rework: Details cards flow, the hand-written Network card gives way
to a full address history, Storage and disk health merge into one card with
two sections, Operator Reports and Corporate Tools get their own tabs.
* An opt-in restic backup overlay with a third container, restore gated on the
dump's own data revision rather than the code head, and backup/restore history.
* The agent's version is read from the binary's PE version rather than trusted
from the csproj, and the Fleet view gains an Agent Version column.
* The enqueue route now asks caller_holds instead of the owner's permissions,
which closes the scoped-token elevation on run_script, update_app and
system_restart.
Held back deliberately:
* agent/installer/install-shopdbagent.ps1 stays deleted, as upstream deleted it.
The installer that reaches a bay is generated by _install_script() in
plugins/endpoints/api/siteagent.py; the standalone script had stopped being
shipped and was collecting fixes no endpoint ever ran.
* The .gitignore hunk adding docs/*.md. Correct on the publication lineage,
where docs/ becomes the wiki and its absence is how the gates detect a
published checkout. Here it would silently ignore every new doc.
* Their deletions of docs/DEV-BUILD-TEST.md and docs/PLUGIN-HOOKS.md. Upstream
those were stubs; here PLUGIN-HOOKS.md is the 31KB document CLAUDE.md points
readers at.
* Five zero-byte files committed by an accidental `git add -A`, a run log from
the author's own machine, and docs/notes/, which is working scratch that says
so in its own header and belongs in devnotes/ on this side.
Carried with the import so it lands green: the 17 new routes are described in
docs/api-inventory.json with the spec and project map regenerated, and two new
frontend specs read fixtures through fileURLToPath(URL) which node rejects under
jsdom's own URL global.
Suite 2591 passed / 9 skipped; frontend 391 passed; naming, style and doc gates
clean.
Scope: product
Phase 3 of the device-identity proposal moves device identity to
HKLM:\SOFTWARE\GE\ShopDB\DeviceId, which survives the profile cleanup and disk
hygiene that eats C:\Enrollment and needs no new key: the collector already
opens that path for BaseUrl and ApiToken. This is the backfill that gets a
value in there without re-imaging anything.
The source depends on the pc-type, and that is the whole point. DeviceId names
the DEVICE, while a machine number answers "which bay is this" - a different
fact that coincides for exactly one family:
CMM the machine number IS the instrument id (CMM4), which is why
cmmid.txt has always doubled as identity. Copied.
part marker the machine number is the OPERATION, and several markers serve
one operation. Copying it would name the operation as the
device, which the collector refuses as the wrong kind of asset
because a machine filed under a device label reads as a marker
everywhere downstream. Never copied.
Keyence, no per-bay id has ever existed; asset-id.txt is the only
Genspect, identity these have.
wax-trace
anything else not a device bay, so nothing is written. DeviceId is not
consumed for those pc-types and a value left there is a
landmine for the day one gains a device spec.
Resolution per family mirrors Report-AssetToShopDB.ps1, including the 9999
placeholder skip and reading WOW6432Node before the native eDNC key.
Refuses to overwrite a DeviceId that disagrees, unless -Force. asset-id.txt is
written once at imaging and nothing updates it on a device swap, so a stale
file clobbering a corrected registry value would reintroduce the exact defect
this move exists to fix. GE-Enforce already reached the same conclusion for
machine-number.txt.
Verified in the win11 VM across ten cases: not-a-device bay, part marker,
CMM precedence, idempotent re-run, machine-number exclusion, eDNC fallback,
9999 skip, stale-file refusal (exit 2), -Force, and -WhatIf. The part-marker
guard was then planted - flipping it to treat the machine number as identity
writes operation 0613 into DeviceId, so the passing test is not vacuous.
Scope: product
ADR-005 split machines from measuring tools on the grounds that they are two
asset classes. It named the metrology population as CMMs, vision systems, wax
trace and Genspect; the plugin was then built against a docstring naming
calipers, micrometers and thread gages. Those are not the same population, and
the difference is the whole decision: bay-mounted capital equipment with a
controlling PC on one side, hand tools in a drawer on the other.
Sixteen months on, measuringtools holds zero rows against 204 machines. The
metrology inventory waiting to be entered is 134 instruments - 35 CMM, 32 form
tracer, 23 vision, 18 wax, 14 measuring, 8 Genspect, 2 eddy, 1 spline - every
one bay-mounted with a controlling PC, and not one hand gage among them. CMM is
already a machinetype, and Part Marker is already a machine while Form Tracer
is a measuring tool, so the line is already drawn somewhere no operator can
predict.
Retires the plugin and the measuring_tool asset type. Calibration is not lost:
it moves onto machines as four nullable columns with status still derived at
read, which is a gain rather than a port because a machine could not record
calibration at all before. ADR-005's one concrete objection - that "show me all
measuring tools" would need an enumeration of type names - is answered with
machinetypes.ismetrology, so it stays a single-column filter.
Taken now because measuringtools is empty. After the 134 rows are entered they
carry relationships, audit history, map positions and collector links, and the
migration stops being a drop.
Scope: product
The adoption guide told an adopting site that a missing EventSaver.ini makes
the binary fall back to "the reference site's file server". That was true of
EventSaver.cs until the ADR-015 fix set Folder to empty, and EVENTSAVER.md was
updated then while this page was not, so the two documents disagreed about what
a missing ini does.
Current behaviour is that it shows nothing, which is the point: failing visibly
beats displaying another site's slides. Says that, and adds the case the old
text was accidentally describing - a .scr compiled before the fix still carries
the path, so a binary somebody was handed is not the same as one they built.
Scope: product
CONTRIBUTING.md still said "main is the product every site runs and the
publication mirror exports" and framed wj/wilm as site branches feeding it.
That was true when main was the only thing anyone deployed. It is not now:
WJ production runs wj, the /ops instance runs dev, and main deploys nowhere.
Nothing in the repository mentioned dev at all, so the branch a deploy script
refuses to run without was undocumented.
Replaces that section with the branch/instance table and the dev -> wj -> main
promotion order, and says why promoting before validating is the wrong way
round. Keeps the wilm note but moves it up with the reason it cannot be
merged locally: it sits on the publication lineage and shares no ancestor.
CLAUDE.md gains the branch-to-server pointer and records that
export-github.sh now builds a dist only for wj and dev.
docs/DEPLOY.md is deliberately left alone: `git pull origin main` is correct
there. That document is the generic connected-Docker deploy another site
follows, and main is exactly what another site adopts.
Scope: product
Taken from the other site, who hit it in production today (their commit
db75ca7): agents that fail the TLS handshake retry in a loop, and with four
blocking sync workers a handful of them pinned every worker. Real UI requests
queued for 20+ seconds while the CPU sat idle - the box was not busy, it was
blocked.
gthread, 4 workers x 8 threads, and a 30-second timeout rather than 60. A stuck
connection now holds one thread out of 32 instead of one worker out of four, and
it lets go in half the time, which is the difference between absorbing a retry
storm and queueing behind it.
Their server, our agent. The client behaviour that caused it is the endpoint
agent, so it will reach any site running one - which makes this a product fix
rather than something that belongs on one site's branch.
Applied to the Dockerfile as well as docker-compose.yml. A compose file
overrides the image CMD, but docker-compose.airgap.yml, .channel.yml and
.pull.yml do not specify a command at all, so they take the image default - and
so does anybody running the image by hand. Fixing only compose would have left
every air-gapped install on sync workers.
WJ production is unaffected either way: it runs waitress under IIS, already with
--threads=8, so it was never exposed to this. That is worth knowing before
anyone treats it as a prod hardening item that was missing.
Scope: product
The rule for this site, stated plainly: only `wj` reaches /shopdb (WJ
production), only `dev` reaches /ops, and `main` is the curated release other
sites adopt and deploys nowhere. The tooling did not know that, and in two
places actively assumed the opposite.
export-github.sh refused --dist on any branch but main, because production was
assumed to install main. It does not - it installs wj. That guard fired today: a
wj export produced its bundle and then died at the dist step, so the dists on
the share came from the main export that happened to run next. The code going to
production and the frontend going to production would have come from different
branches, silently, and nothing downstream could tell.
Now one branch builds one dist: wj builds the /shopdb dist, dev builds the /ops
dist, and any other branch is refused because a dist it built could only ever be
installed by mistake. Each dist carries a .dist-source stamp naming the branch,
commit, base path and build time.
The two deploy scripts copy from the clone on WHATEVER branch it is sitting on
and only ever checked the destination (the web.config mount path), so a clone
left on the wrong branch deployed the wrong code to a correctly-identified
server without a word. Both now refuse unless the clone is on their branch, and
both read the dist's .dist-source and refuse a frontend built from anywhere
else. A directory of hashed filenames says nothing about where it came from, and
robocopy /MIR will mirror whichever one it is pointed at.
pull-shopdb-bundle.cmd and update-shopdb-github.cmd silently defaulted to main.
That was safe when main was the only thing anyone deployed; with three branches
it is a way to carry the wrong one to a server without ever typing its name. The
argument is required now, and the error lists what each branch is for.
tools/mirror-wilm.sh is the missing half of fetch-wilm-branch.cmd: that script
only puts a bundle on the share, and wilm does not appear on gitea by itself.
The new one fetches the carried bundle and pushes wilm to gitea, printing the
bundle's age first - a mirror that looks authoritative and is a week old is
worse than no mirror - and noting that wilm sits on the publication lineage, so
it shares no ancestor with main/wj/dev and cannot be merged here.
Scope: product
The last two items from the run.
The PC detail page has always rendered a Business Unit row, and the API has
always accepted the field. Only the form had no control, so it could be read and
never set from the UI - a value that appeared the moment somebody set it through
the API and could not be reached otherwise. The form has the picker now, loaded
with listAll() rather than a capped page.
Global search excluded inactive assets, so a retired record existed, was
reachable by URL, and could not be found by anybody who did not already know
where it was - including whoever wanted to restore it, which the retire dialog
promises they can. Retired assets are searchable now.
That is only safe with the ordering, so it is part of the change rather than a
follow-up: somebody searching a machine number almost always wants the machine
still on the floor, so a retired match drops 60 points of relevance, sorts below
every live one, and carries a 'retired' flag the results page renders as a
badge. Indistinguishable retired results would be worse than none.
Four tests, verified by planting the isactive filter back: three fail.
Scope: product
The last two kiosk findings.
Keypad entry collected EVERY digit in an item's code, and the shipped prefix
contains one: '3DP0004' collapsed to 30004, so an operator typing the 4 printed
on the label was told no part matches that barcode and had to work out that the
leading 3 belonged to the prefix. The number on a label is the number at the
end, so the trailing digit run is what is compared now. An ambiguous number is
still refused rather than guessed - taking from the wrong bin is worse than
being asked to scan the label.
A parts kiosk that had never been provisioned drew its normal "Scan the barcode
on the bin" screen and answered "Not found" to every scan, which reads as "we
do not stock that part" rather than "this screen was never set up". The server
cannot say which: the routes answer a bare 404 on purpose, so a caller without
the device token does not learn the door exists, and that stays. But the BROWSER
knows whether it holds a token, and saying so discloses nothing - an
unprovisioned screen now says it is not set up and how to fix it.
For the case the client genuinely cannot distinguish - kiosk switched off versus
a part that is not stocked, both a 404 by design - three misses in a row add a
line suggesting the kiosk may have been switched off. Truthful without the
server having to leak anything.
Three tests, verified by planting the all-digits match back.
Scope: product
Three findings, all about a screen or a log saying something that is not so.
Editing a PC wrote an audit entry for its named fields and nothing for the two
changes that matter most when somebody investigates later: the primary IP
address, which is how the machine is found, and the remote-access protocols and
ports, which is how somebody connects to it. Both are on the same form, in the
same request, and both were invisible afterwards - a PC that moved address had
no history of moving, and a PC that gained a VNC listener had no record of
gaining one. Both are recorded now, and re-sending the same value still records
nothing, because an audit log full of non-events is one nobody reads.
While proving that, a silent discard turned up in the same lines: both the
create and the update path did `if ip_comtype:` around the IP write and did
nothing at all when the 'IP' communication type row was missing. Somebody typed
an address, saved, got a 200, and the address was gone. A missing reference row
is a seeding gap, not a reason to drop what was typed, so it is created on
demand.
The PCs list rendered a 403 as "No computers found", which tells a person
without the permission that the fleet is empty and sends them looking for the
data rather than for the permission. It now says which permission is missing.
The Site Policies and Configuration Policies pages rendered their empty table
and an Add button while the policy feature flag was off - every route answers
404 "Policy feature is not enabled", so the button could never work. Both pages
now say the feature is off and that the empty list is a consequence of that,
not of nothing being configured.
Nine tests. The audit entries verified by planting: remove either and two fail.
Scope: product
You are right that this is the one we keep hitting. perpage is capped at 100
server-side, and the cap is applied SILENTLY: a request for 500 returns the
first 100 rows with a 200, so a picker built that way shows the first 100
options and drops the rest with nothing anywhere reporting a problem. The
missing vendor simply is not in the list.
Twelve call sites were still doing it - vendor and business-unit pickers on the
machine and measuring-tool forms, and the type lists behind PCs, machines,
printers, network devices, measuring tools and configuration policies. Each now
either pages through with listAll()/fetchAllPages() or asks for what it can
actually have. vendorsApi and businessunitsApi gained listAll(), which they
lacked, which is why those two sites reached for 500 in the first place.
The gate is the point. tests/test_core/test_perpage_cap.py reads every .vue and
.js source under frontend/src and plugins/*/frontend and fails on any perpage
over 100. It has to read the SOURCE rather than the running app, because the
response is a valid 200 either way - no runtime assertion can see this one.
Verified by planting a perpage: 500 back into MachineForm: the gate names that
exact line.
Separately, /api/auditlogs built its own meta with the pagination fields flat
and a 'pages' key, while every other list returns
meta.pagination.{page,perpage,total,totalpages,hasnext,hasprev}. A client
written against the documented contract found nothing where it looked, on the
one list whose whole purpose is being read by somebody checking what happened.
It uses paginated_response now.
Also carried here, from the previous batch's working tree: Asset.to_dict emits
ownlocationid/ownmapx/ownmapy/ownlevelid alongside the resolved values, and the
PC form binds to those. The merge of an inherited location is right for a reader
and wrong for a form, which could not tell an owned value from a derived one and
wrote the inherited one back as the record's own.
Scope: product
Reported from the floor: somebody types the employee's whole name into the
notification or recognition picker and presses Enter instead of clicking the
suggestion, and the entry does not pull from the employee database.
Two causes, one symptom.
The search matched the WHOLE query against first name or last name, so "Dana
Reyes" found nobody - no single column holds that string. Typing a full name is
what a person does before pressing Enter, and it was the one thing guaranteed to
return an empty list. A multi-word query is now treated as a full name: every
word must appear somewhere in the name, in either column, so "Reyes Dana" works
too, and "Dana Reyes" does not also return Sam Reyes. A single word still
searches broadly, and an SSO still resolves. Both directory backends changed,
the self-hosted tables and the external HR database.
With no suggestion to click, the form then turned the typed text into
`NAME:<text>` - a label linked to nothing. On screen that chip is identical to a
real selection; the difference only shows up later as no photo, no directory
link, and a recognition the person it is about cannot be found by. Both pickers
(the notifications form and the shared EmployeeSearch component) now look a
typed name up before deciding it is free text, through one helper.
An exact full-name match wins over a longer partial one, because somebody typing
"Dana Reyes" means that person and not "Dana Reyes-Smith". A genuinely ambiguous
name is NOT guessed at - the candidates are shown and the operator picks, which
is exactly when picking wrong matters most. A directory that cannot be reached
reports failure rather than silently producing a label.
Eleven tests. The search rules verified by planting the single-term query back.
One note for whoever reads the spec: the failure-branch test uses a thenable
that rejects rather than a mock returning a rejected promise. The runner
collects such a promise as an unhandled error even when the code under test
catches it, and fails the very test that proves the catch works.
Scope: product
Two operator-visible defects and three doc corrections, closing the tail of the
2026-09-03 run.
THE BADGE. The two kiosks read the same physical badges and disagreed. The
parts kiosk understood both shapes a scanner emits - the bare SSO and the PayNo
wrapper 0<digits>BZ - while the USB kiosk compared the raw string to an SSO, so
the wrapper resolved to nobody. An operator whose keyboard-wedge scanner emits
it could take a 3D-printed part and could not check out a drive, with the same
badge, at two cabinets a few metres apart.
The shape now lives in shopdb/utils/badges.py on the contract surface, because
a plugin may not import another plugin and because a badge is a property of the
site rather than of a cabinet. Both kiosks use it, and both store the DIGITS
rather than the string that arrived: one row per spelling makes a custody
history that cannot be joined to itself.
That closed a subtler hole in the same place. str.isdigit() is true for
Arabic-Indic and other decimal forms, and int() parses them, so a badge written
in those characters was accepted, matched a REAL employee, and was then stored
as an SSO string nothing else matches. ASCII digits only. An empty scan and an
unreadable one now get different answers, because they are different problems
for the person at the cabinet.
THE IMAGES. /api/applications/image/<filename> and /api/models/image/<filename>
required a permission, and a browser fetching <img src="..."> sends no
Authorization header - so the gate did not protect the file, it guaranteed the
picture never appeared. Both are ungated now, like the four sibling image routes
(branding logo, map blueprint, employee photo, part picture), and registered in
PUBLIC_ENDPOINTS with that reasoning.
/api/printers/lookup?fqdn= never matched anything: both arguments were compared
against Communication.ipaddress, so a hostname lookup 404d on printers whose
hostname was on the record. An FQDN now matches the hostname, with a fallback to
the short name.
The slide manager gained the per-slide hold time. The PATCH route and the feed
have always carried `seconds`; nothing called it, so the one documented
per-slide control was curl-only.
Docs: API-REFERENCE listed the employee lookups and the whole notifications
surface as unauthenticated when they answer 401, and called both kiosks
"deliberately open" when they are off by default and refuse anything without the
site's device token. The installer download says applications.download, which is
what the code requires. api-inventory.json said printers/lookup was
jwt-optional.
Fourteen tests. The badge rule verified by planting the old per-kiosk behaviour:
three fail, one in each kiosk.
Scope: product
The validation tail from the 2026-09-03 run. Each is small; together they are
the difference between a field that checks its input and one that looks like it
does.
Subnets accepted any string containing a slash, so 'notanip/xx' was stored -
and _devices_in_subnet then swallowed the ValueError from ip_network and
returned an empty device list, so the subnet looked real and permanently empty.
It is parsed with the same library that has to read it later, a prefix length is
required (ipaddress reads a bare 10.1.1.0 as a /32, which is legal and never
what somebody typing a subnet meant), and the stored form is normalised so
10.1.1.5/24 and 10.1.1.0/24 cannot become two rows nobody can tell apart.
VLAN numbers had no range check: 99999 and -5 were stored. Worse, a non-numeric
value reached a duplicate probe that compares against an integer column, which
MySQL coerces to 0 - so 'xyz' matched an existing VLAN 0 and the 409 named a row
that had nothing to do with it. 1-4094 per 802.1Q, checked before the probe.
Deleting a VLAN that subnets still reference now answers 409 like every sibling
in-use guard, rather than 400, which told the caller their request was malformed
when the request was fine and the state was the problem.
An appversion sha256 was stored raw, so an over-long value was truncated to the
column width and came back as 64 plausible hex characters - indistinguishable
from a real digest, and wrong. A digest is a claim the delivery path checks
against, so it is 64 hex characters or it is refused. The releasedate goes
through the strict parser for the same reason it did everywhere else.
PUT /api/settings/<key> ignored the declared valuetype. "A6-not-a-number" went
into an integer key verbatim and every consumer then read 0, because
get_typed_value swallows the ValueError - the setting looked set and the
behaviour was the default.
/api/maplevels told a caller who omitted the level name that 'buildingname and
buildingid are required'; buildingname is a field that handler never reads. It
now names the fields it actually wants.
Restoring a guide revision that is already current writes nothing, because
save() dedups on the content hash - and the response still said "Restored
version N as a new draft version", describing a revision that does not exist
while the unchanged revision list disagreed.
GET /api/backups/revisions/<id>/diff?against=<id> accepted any revision id, so
pointing it at another asset's revision rendered that asset's configuration as
the "before" side of this asset's history: a diff that reads as a change on this
machine and describes a different one. Same asset and same kind, or refused.
The default branch was always asset-scoped; only the explicit one was not.
Two frontend fixes: the per-page selector rendered blank on any list defaulting
to a size not in [10,20,50,100] (network devices uses 25), so the operator could
not see the current page size; and an emptied PC alias was dropped from the
payload instead of sent, so it could not be cleared - the form came back showing
the alias that had just been deleted.
Twenty-two tests. The backups guard verified by planting.
Scope: product
Three credential and authz findings, all the same shape: a second path to a
decision, with weaker rules than the path built for it.
PUT /api/apitokens/<id> {"isactive": true} un-revoked a token, and the ORIGINAL
secret worked again - so "revoked" meant "paused until somebody flips it back",
for a credential whose secret is assumed to be out by the act of revoking it.
Revocation is one way now. Deactivating through the same route still works;
bringing one back does not, and the refusal says to mint a new token.
PUT /api/users/<self> accepted a `password` field with no current-password
proof and no length rule, while /api/auth/change-password requires both. When
two routes reach the same state the weaker one is the real rule, so the stronger
one was decoration. Changing your own password through the users route is
refused and points at the right endpoint. An ADMIN may still set one - that is
how a temporary password is issued - with the same 8-character floor, and
mustchangepassword is now set, because a password somebody else chose is
temporary by definition.
tools.secrets was declared guest-grantable, so the role grid offered it and
stored it, and the secure-send route never consulted the guest role at all: it
decides on securesend_allow_guests, which fails closed and carries the rate
limit. Ticking the box changed nothing in either direction, so the screen and
the behaviour disagreed. The declaration is gone; tools.view stays, because it
really does gate the nav entry.
Removing it needed care: update_role refuses the WHOLE request when a submitted
permission is not grantable, and the grid resubmits what the role already holds,
so a site whose guest role held tools.secrets could no longer save that role at
all - the deadlock fixed in 723a4ff, re-created from the other end. Migration
7d41 revokes the grant from the guest role, and nothing is lost because it
granted nothing.
Eight tests. The first two verified by planting: remove either guard and three
fail.
Scope: product
The machine maintenance panel showed every date one calendar day early on this
site, and would show it a day late east of UTC.
lastmaintenancedate and nextmaintenancedate were DATETIME columns holding
midnight, and BaseModel.to_dict serialises a datetime as isoformat + 'Z', so
2026-12-15 00:00:00 went out as "2026-12-15T00:00:00Z" and the browser
converted it into local time. There is no time in a maintenance date: the form
offers a date picker, the CSV importer takes '2026-06-01', and the value is only
ever compared against other dates. Every sibling already models it correctly -
measuringtools.lastcalibrationdate, warranties.startdate and enddate are all
db.Date - so machines were the odd one out.
Migration 7d40 converts both columns; MySQL truncates a midnight time so no
data moves, and SQLite neither enforces nor needs it.
That exposed two more things. The route was assigning the raw request string to
the column, which a DATE column rejects outright - it now goes through the
strict parser from wave 2, so a mistyped maintenance date is refused with a 400
rather than stored or nulled. And BaseModel.to_dict had no branch for a plain
date, so one reached Flask's encoder and came out as an HTTP-date
("Thu, 15 Jan 2026 00:00:00 GMT") - which a browser parses back into a moment
and can still render as the day before. Dates now serialise as "2026-01-15",
with datetime checked first because it is a subclass of date.
Verified against MySQL on a live instance: the column is DATE, the API serves
2026-01-15, a mistyped date is refused with 400, and a rename on a machine that
has maintenance dates saves - the 500 from the audit-log fix in 5d81f50c, now
confirmed end to end rather than only in tests.
Scope: product
The retire dialog says "History and relationships are kept, and it can be
restored." The API always kept that promise - ?active=false lists retired rows,
PUT {"isactive": true} restores one - and no screen in the product could reach
either. A retired asset simply vanished: the lists filtered it out, global
search missed it, and its detail page still rendered Edit and Retire with
nothing anywhere saying it was retired.
Five list pages gained a "Show retired" toggle that passes active=false, and a
retired row now carries a badge. Five detail pages gained a banner saying the
record is retired and what that means, a Restore button, and the suppression of
Edit and Retire while it is retired - offering Retire on an already-retired
record was the part that made the state invisible.
Verified end to end in a browser: retire a machine, the detail page says so and
offers Restore, the list hides it until the toggle is ticked and badges it when
shown, and Restore puts it back (isactive true again on the API).
One thing this does NOT do: global search still does not reach a retired asset.
That is a separate surface with its own filter, and worth deciding on
deliberately rather than sliding in here - a search that returns retired
records by default would change every result page.
Scope: product
Three more findings from the 2026-09-03 run.
Werkzeug answered 405, 415 and a malformed JSON body itself, in HTML. A client
written against the documented {status,data,meta} contract got a page of markup
with no error code, which is indistinguishable from the server having fallen
over. The 405 had a second edge: on a route the caller cannot otherwise see, a
method-not-allowed CONFIRMS the path exists, so a gated kiosk route meant to
read as absent announced itself by refusing the wrong verb. All three now
answer in the envelope, and only under /api/ - everything else is the SPA and
keeps the framework's behaviour, deep links included.
The Toner Report rendered "0 Printers Checked / 0 Low / 0 Critical" and "No
printers match the selected filter" while the API it had just called said
supply monitoring was unreachable. Zeros read as a clean bill of health for a
fleet nobody has looked at. The page now carries the reason the API gave and
suppresses the counts, because a number nobody measured is not a result. Same
defect shape as the supplies card fixed on 2026-08-29; `unavailable` was
already in the payload and the page ignored it.
Backup revisions badged "current" per (asset, kind). NTLARS settings describe a
MACHINE but are read off the controlling PC, so a machine with two reporting
PCs has two independent chains of the same kind - and only the higher revision
id was badged, so the second PC's newest revision showed as history and lost
its last-verified stamp. The machine looked like one source re-reporting rather
than two. sourcehostname is the chain; a revision without one forms its own,
which keeps the old behaviour for everything that has only ever had one source.
Nine tests. The envelope ones verified by planting: remove the three handlers
and three fail.
Scope: product
The driver form and the API both dropped `drivername` and `vendorid`. POST
returned 201 with both null, PUT accepted them and stored nothing, and no
screen offered either field - so there was no interface in the product that
could set them.
That is not a cosmetic gap. `drivername` is the exact string the INF declares,
and Add-PrinterDriver matches on it; `name` is ours to choose and means nothing
to Windows. With it null the shipped converger has nothing to match and
installs nothing, and the drift check that compares an observed driver name
against the expected one could never fire, because the expected value was never
stored. `vendorid` is how a universal driver (HP UPD, Xerox GPD) attaches to a
whole make instead of needing a near-duplicate row per model.
Both are now read on create and update, and the driver form gained the two
fields with hints saying what they are for - the INF name box says plainly that
leaving it blank means the converger cannot install that driver. A whitespace
value clears rather than stores: an empty box means "not known", and a blank
string would look set and match nothing.
Three tests. Verified by planting: drop the two fields from the create path
again and the round-trip test fails.
Scope: product
Three map and aggregation defects. The first is a behaviour change you asked
for; the other two are the 2026-09-03 run's findings.
A dualpath secondary bay was excluded from /api/assets/map unconditionally
while the collapse setting is on, so positioning it was a silent no-op: the
form took the pin, stored mapx/mapy, and the map never drew it. One marker per
pair is right when the pair sits in one place, and that stays the default - but
a dual-bay machine whose second bay is physically elsewhere is exactly what
positioning it separately means. Now: a secondary with its own position draws
its own marker; one without is represented by the primary's, which carries the
combined '2007 / 2008' label. Nothing extra is needed for the fallback, because
the feed already requires mapx/mapy - an unpositioned bay never reaches the
loop.
The combined label is now suppressed when the partner draws its own marker.
Labelling two separate pins '2007 / 2008' would say one machine is in two
places, which is worse than either bay being unlabelled.
Verified against a live instance: unpositioned secondary gives one marker
labelled with both bays; positioned gives two markers each with its own name;
clearing the position puts it back to one. That third case matters - a
one-way door would be its own bug.
machines/dashboard/summary applied the dualpath exclusion to total and bytype
but not to bystatus, so the status bars summed to more than the headline above
them, by exactly the number of secondary bays. A breakdown that does not add up
to its own total is an afternoon of reconciling.
The measuring-tool calibrationstatus filter ran AFTER pagination. It is a
derived value so it cannot be a SQL filter, but applying it to the current
page meant asking for 'overdue' with one overdue tool in sixteen returned an
empty page 1, a total of 16, eight pages, and the row itself on page 3. When
that filter is in play the matching set is now built and filtered before
paging - the same tradeoff the old comment named, spent on correct answers
instead of a cheaper query.
Seven tests. The pagination one verified against the committed version, which
fails it.
Scope: product
Wave 6: two settings pages that told an administrator the site was configured
when it was not. That is worse than a missing feature, because nobody goes
looking for it.
Audit retention had TWO keys. shopdb/core/services/auditretention.py reads
`audit_retentiondays` (category site, default 0). A second `audit_retention_days`
was seeded under category audit with a default of 90, and the Audit & Logging
page wrote to that one - so an administrator set a period, reloaded, saw it
persisted, and nothing ever consulted it. Retention stayed off. The page now
writes the key that is read, the duplicate is out of the seed list, and
migration 7d39 carries a value an operator actually set on the dead key over to
the live one before deleting it: their intent was "keep 30 days", and the fix
should not silently discard it. The page's copy also claimed logs "will be
automatically purged", which nothing does - purging is manual, from the site
settings page or `flask audit purge --apply`.
The Authentication page offered a full SAML configuration - metadata URL, entity
id, ACS URL, auto-create, admin group - and reported "Configured - SAML login
available" once two fields held text. There is no SAML anywhere in the codebase:
no login route, no ACS, no assertion handling. SAML IS planned, so the settings
stay and the page keeps them; what changes is that it now says plainly that
nothing consumes them yet, and the status line never claims availability.
The dangerous control there was "Allow Local Login". Turning it off did nothing
at all, so an administrator could believe they had closed local password login
while it stayed wide open. It is locked on until there is another way in, and
says why. CONFIG.md marks the whole block reserved rather than describing it as
working configuration.
Scope: product
Wave 5, the asset forms. Both defects are the same mistake in different
clothes: a form deciding something it should have been told.
Four forms carried a hardcoded asset-type id - const COMPUTER_ASSETTYPEID = 2,
and the same for machines, printers and network devices - to tell
CustomFieldsInputs which type's fields to render. Those ids come from seed
order. On the instance they were written against, computer was 2; on a freshly
seeded database computer is 1 and 2 is network_device. So the PC form asked for
another type's custom fields, rendered them, and dropped everything typed into
them behind a success toast, because the values were keyed to a type the record
does not have. Verified in a browser against a correctly built dist: the form
now requests assettypeid=1 on an instance where computer is 1.
The name is the stable thing - assettype is a fixed vocabulary and the API
already publishes the mapping - so composables/assetTypeIds.js resolves it once
per page load. A failed load leaves it undefined, which renders no custom-field
panel: a missing panel is honest, a guessed id is what this replaces.
Three of the five forms that emit a map position sent mapx and mapy without
levelid. ADR-017 is explicit that a position without its level cannot be drawn
on the right floor plan, so a picked pin saved a null level and rendered
nowhere - while the form displayed "Position: x, y" back, because it kept the
value it had never sent.
The ADR-017 gate should have caught that and did not: it globbed '*.py' only,
so it policed what the server emits and never what the browser sends. It now
covers .vue and .js too, and accepts both quoting styles. Verified by planting:
drop levelid from PCForm again and the gate names that exact line.
Five composable tests, including a fixture with a DIFFERENT seed order - the
case the constants got wrong.
Still open from this area, and deliberately not in this commit: the PC form
writes an INHERITED location back as the record's own. Fixing it needs the API
to distinguish an owned locationid from a derived one, which is a contract
change rather than a form fix.
Scope: product
Three confirmed findings from the 2026-09-03 run, each independent.
GE-Enforce scope names are directory names on the imaging share, and create_scope
checked only that the name was non-empty. "../escape" wrote a manifest one level
above the share root, "../../escape2" two, and "/tmp/anywhere" wherever it liked
- proven by writing the files and then deleting them. The exposure is the
permission split rather than the traversal itself: geenforce.manage creates the
scope and geenforce.publish writes the file, so a manage-only author could aim a
publisher's write at any path that process can reach. Names are now a single
safe path segment, checked at creation AND again at the write (rows predating the
check exist, and the write is the line that acts), with the resolved target
confirmed to sit under the share root.
Agent timestamps were converted with astimezone(tz=None), which means the
SERVER's local zone, not UTC. An agent reporting 17:26:16Z was stored as
13:26:16 on this EDT box and re-served labelled 'Z', while the same row's
server-written lastreportedat stayed true UTC - one row carrying two time bases
four hours apart. The dashboard's 48-hour stability window compares a real UTC
cutoff against these values, so events left it four hours early. Invisible on a
UTC server, which is how it survived, and nothing in the Dockerfile, the compose
files or the Windows deploy pins TZ. AGENT-CONTRACT.md mandates UTC on the wire,
so a naive value is now taken as UTC rather than guessed at.
The three /api/maplevels reads went back to optional-auth. ADR-017 rule 6, the
API inventory (auth "none"), the module's own docstring and the comment in
PrinterInstallerMap.vue all say they are public, because the printer installer
map runs before anyone logs in and needs a blueprint to draw on. afce2437 swept
165 genuinely-open GET routes behind require_permission and caught these in the
net, so the anonymous installer map has drawn no floor plan since. Restored
rather than amending the ADR: blueprint paths and level names are not secrets,
and the asset positions that go on top already render on public kiosk pages.
The writes stay gated, and a test proves it.
Registered in PUBLIC_ENDPOINTS with the reason. That gate checks a public route
is DECLARED public; the new test checks it actually answers, which is the half
that was broken.
Ten tests. All three verified by planting: restore the non-empty scope check and
eight fail, restore the local-zone conversion and two fail, put the permission
back on the level list and one fails.
Scope: product
Wave 4, second half. PUT /api/machines/<id> returned 500 and rolled the whole
edit back for any machine with a maintenance date recorded - including a rename
that never touched the date. The Machine edit form could not save such a
machine at all, which is most of them once a maintenance schedule exists.
The route builds {'old': getattr(row, field)} straight off the model, so the
changes dict carries whatever the column holds: a date, a datetime, a Decimal.
AuditLog.changes serialised it with a bare json.dumps, which raises on all
three, and nothing caught it. The audit entry and the edit share a transaction,
so the failure took the edit with it.
json.dumps now has a fallback encoder: dates and datetimes become ISO strings,
Decimals become floats, sets and bytes get something readable, and anything
else becomes its repr rather than an exception. An audit entry that reads a
little awkwardly is worth more than a failed write - the alternative, and what
happened, is losing the change AND the record of it.
Fixed at the audit log rather than in the machines route: every audited route
builds its changes dict the same way, so the next one to audit a date field
would have hit the same wall.
Four tests, including the exact shape the machines route produces. Verified by
planting the bare dumps back: two fail.
Scope: product
Wave 4, first half. A personal API token scoped to endpoints.command alone
could queue system_restart, run_script and update_app against a shop-floor PC.
require_permission has always suspended the admin bypass for a scoped token,
and tests pin that. What nothing covered was a route checking a SECOND
permission by hand: the endpoints command route asked
current_user.haspermission(elevated), which reads the token's OWNER. An admin
minting a narrow token for a script therefore handed out remote restart, script
execution and app deployment - the exact bypass require_permission exists to
prevent, reintroduced one hand-rolled check at a time.
caller_holds is now scope-aware, so the answer is about the caller rather than
the person behind it: the owner must hold the permission AND, when the request
carries a scoped token, the scope list must name it. An unscoped token is the
person and is unchanged; a session is unchanged; guests still answer from the
guest role.
Both hand-rolled checks in the tree now use it - the endpoints elevated gate and
the secure-send tools.secrets check, which had the same shape and the same
consequence at a smaller scale.
Two tests, one proving the refusal and one proving an unscoped token still
passes (a fix that quietly broke every service token would be worse than the
bug). Verified by planting the owner-based check back: the first fails.
Reported by the fleet-agent lane of the 2026-09-03 test run and confirmed by an
independent verifier that minted its own token and queued its own commands.
Scope: product
Wave 3. Six findings, one cause: `int(request.args.get('typeid'))` with nothing
catching the ValueError. The caller sent a bad argument and got
500 INTERNAL_ERROR plus a traceback in the log, which says the server is broken
when the truth is that 'abc' is not a number.
shopdb/utils/queryargs.intarg reads it and raises QueryArgError, which an
app-level handler answers as a 400 naming the argument. App-level on purpose: a
route should not have to remember, and forgetting was the entire defect. The
policy differs from get_pagination_params, which falls back to a default for a
bad page number - a nonsense page is harmless and the caller still wants their
list, but silently ignoring a nonsense FILTER returns rows they believe they
excluded.
The gate matters more than the six fixes. tests/test_core/test_bad_query_args.py
sweeps all 131 registered no-parameter GET routes under /api/ with twelve
argument shapes and fails on any 5xx. It found sites the test run had not
reported - statusid and locationid on /api/assets, the second asset-list route,
limit on the employee lookup and the printer supply history - because the
pattern is how a list filter gets written rather than a property of any one
route. Verified non-vacuous by planting the old `int(...)` back into
/api/machines: four tests fail.
Four usb routes are skipped by exact path, with the reason: they reach the
EXTERNAL cmmc_usb database, so their 500 in a test environment is a missing
database rather than an argument fault. That is the complete set - every usb
GET route calling cmmc_usb_connection - not a convenient subset.
One correction worth recording: the first pass rewrote every
`if x := request.args.get(...)` in the touched files, including text arguments,
so a search for 'ZZNOTHINGMATCHES' briefly became a 400. Caught by
test_list_search_identifiers. Only id-shaped arguments are integers.
Scope: product
Wave 2 of the remediation plan. Eight findings from the feature test run were
the same defect twice: a write that reports success while storing something
other than what was sent.
The first half is a date parser that swallowed its own error. Two plugins
carried an identical private _parse_date returning None on ValueError, so an
unparseable date and a deliberately cleared one produced the same write.
PUT /api/warranty/<id> with enddate '30-06-2027' answered 200 "Warranty
updated" and stored NULL, dropping the record out of every expiry surface on
the strength of a typo; the measuring-tool calibration date did the same and
flipped the tool to 'unknown', out of the calibration report and the map
overlay. shopdb/utils/datevalues.py now holds one strict parser on the contract
surface: an explicitly empty value clears the field, anything that is not a
date is refused with a 400 naming the field, and those are different intents
with different answers. The vendor-lookup paths keep a lenient wrapper on
purpose - a provider changing its date shape must not fail the refresh of an
otherwise good record, and no person typed that value.
The second half is why the truncation findings were possible at all. MySQL was
running non-strict: sql_mode on the 5.6 server is NO_ENGINE_SUBSTITUTION with
no STRICT_TRANS_TABLES, so it accepted writes it could not store and adjusted
them. A 257-character username became its first 100 characters (and the next
distinct one then collided with it); an over-long asset number was cut at 50; a
malformed releasedate became '0000-00-00' and permanently broke every read of
that application. All with a 201. The mode is now set on the CONNECTION rather
than left to the server, so a site whose my.cnf disagrees still gets it.
That trades silent corruption for a DataError, which uncaught is a 500 - and a
500 says the server is broken when the truth is that the value does not fit. An
error handler maps it to a 400 naming the column where MySQL names one.
Verified by hand against MySQL 5.6: POST /api/users with a 257-character
username now answers 400 "That value is too long for username." and stores
nothing, where it previously stored a truncated row and answered 201.
Twelve tests. The date guard is verified by planting: restore the swallowing
parser and four of them fail. The DataError tests raise the error directly
because the suite runs on SQLite, which does not enforce VARCHAR length at all -
there is no value that would trigger it there, which is precisely why this went
unnoticed.
Scope: product
Wave 1 of the remediation plan for the 2026-09-03 feature test run. Two kiosks,
one shape of defect, one critical consequence.
Both kiosks read the state, decided in Python, and then wrote. Nothing stops two
requests interleaving between the read and the write, so both see the same
pre-state and both proceed. Reproduced on MySQL 5.6 over HTTP, five concurrent
takes of 5 from a bin holding 5: all five were accepted and quantityonhand
landed on -20, because each request wrote its own arithmetic. The ledger had
recorded every one of them, so the cached quantity and the ledger disagreed
permanently, which is the number the shop floor actually reads. The USB kiosk
had the same shape: concurrent scans of one drive all succeeded and inserted
several open custody rows, so one drive was recorded out to two people and the
documented 409 never fired.
The comparison now lives in the WHERE clause, where the database settles it at
write time. printedparts._move_quantity issues
UPDATE ... WHERE quantityonhand >= :n and treats rowcount 0 as the refusal,
raising InsufficientStock, which take and adjust turn back into the message they
already gave - carrying the quantity the database holds now, not the stale one
the request read. The low-stock alert derives its before-value from what the
write produced for the same reason. Both USB paths claim the device with
ischeckedout (self-hosted) or status (external cmmc_usb) in the WHERE clause;
the external one claims BEFORE writing the log line, so a refused claim leaves
nothing to undo. Check-in is guarded the same way.
After the fix, the same five concurrent takes: one accepted, four refused,
quantityonhand 0 and the ledger sum 0.
Check-in also closes EVERY open row for the device rather than the newest.
One device can only be out once, so a second open row is corruption, and
closing only the newest left the older ones open forever - the device then read
as returned while its custody history said otherwise.
Separately, usbcheckouts now records who handed a drive back
(checkinsso/checkinname, migration usb0004checkinidentity). Returning for a
colleague is a supported flow that the kiosk asks to confirm, but the row it
wrote attributed the check-in to the HOLDER, so the custody history and the
audit report both named somebody who was not there. Nullable and deliberately
not backfilled: copying the holder's badge into old rows would manufacture
evidence for the exact claim this stops making.
Four tests. Both guards verified by planting: strip the condition out of either
UPDATE and its test fails.
Scope: product
Five findings from the review of the WINC import, plus the rendering each fix
needed to be usable.
An undo that hit a conflict wrote status 'undone' anyway, and undo_batch returns
immediately for an undone batch. So the one entry that needed a retry was the
only one that could never get it: the human resolves the conflict, clicks Undo,
and gets a no-op. A conflicted undo is now 'partially-undone', undoneat is
stamped only when the batch is genuinely finished, and the batches table offers
"Retry undo" for that status. The column was String(16) and the new value is
exactly 16 characters, so migration 7d38 widens it to 32 rather than rely on an
exact fit.
_UNDO_REFERENCES listed only assets, so a business unit this batch created was
hard-deleted even when something else still pointed at it. Two of the three
missing referrers are real FKs (dashboarddefaults.businessunitid,
businessunits.parentid): MySQL raises and the whole undo rolls back, so the
batch cannot be undone at all. The third, notifications.businessunitid, has no
FK, so the delete succeeded and left a notification pointing at an id that no
longer exists - silently, on both engines. All four are listed and the guard is
tested through the route.
The reference guard also queried tables that need not exist. A lean site drops
the tables of plugins it does not install (ADR-014), and notifications is
plugin-owned, so on such a site the query raised 1146, rolled the undo back and
left the batch permanently un-undoable. Absent tables are skipped; a table that
does not exist cannot hold a reference.
_PACKET_GROUPS reports on location, but apply_plan only ever creates business
units - it sets locationid only when the location already resolved. A plan
needing a new location asked the admin to acknowledge it, got the tick, wrote
nothing, and reported nothing. build_packet now takes the groups the writer can
actually create; anything else needed is 'uncreatable' and blocks at plan time,
where the admin can act on it, rather than dying at commit on a message about
acknowledging something the screen had no control for. The writer refuses the
same case independently. Both the console packet and the import page name the
blocking groups: setting blocking without printing the reason would leave the
banner pointing at nothing.
The web undo route caught ValueError only, so refusing a batch belonging to
another importer surfaced as a 500. That refusal is new too: the journal is a
general primitive and undo_batch does not filter on kind, so any other
importer's batches were reversible through this endpoint, carrying WINC's
references map - which lists none of that importer's tables, making every
reference check pass and every delete unconditional.
Nine tests. The two guards were verified by planting violations: dropping
dashboarddefaults from the references map, and passing creatable=None, each fail
the test written for them.
Scope: product
Six fixes that were ours regardless of what the sister site does. Four came out
of our own reviews; two you reported from the settings pages.
MAX_CONTENT_LENGTH was set nowhere. Every upload route caps its own payload, but
each of those checks runs after the body has arrived: the first touch of
request.form or request.files makes Werkzeug parse the whole multipart and spool
the file to a temp file, so a route can reject the content and still have paid
for the transfer and the disk. Three separate reviews found that shape in three
different routes, which makes it a property of the app rather than of any
handler. 128 MB, clearing the 64 MB agent binary with room for the envelope.
The naming gate excluded devnotes with --exclude-dir, which matches a directory
of that NAME at any depth - so a force-added plugins/<plugin>/devnotes/ was
invisible to every check in the file, ADR-015 included. Replaced with a path
anchored post-filter. Verified by planting the same site literal in root
devnotes/ AND in plugins/endpoints/devnotes/: the gate now reports exactly one,
the nested one. Before, it reported none.
BaseModel.to_dict walked __table__.columns and published every one, so a model
without an override published whatever it carried - including columns added
later by someone not thinking about this method. User has no override, so
User.to_dict() returned its bcrypt hash. Nothing calls it in a response path
today, which is luck rather than design. A blanket allowlist would empty the
output of all 34 models relying on the default, so this pins a floor instead:
named secret-bearing columns are never emitted, and a model may extend the set.
Eight tests, including a sweep over every mapped model.
ActionPicker's Esc handlers had .prevent but not .stop, so the event bubbled to
the wrapper and closed the whole picker instead of returning to the monitor list.
The branding page reported "error loading image" on the site and badge logos.
Two bugs stacked. The load failure was real: the preview used the stored value
as a raw src, and that value is root-relative, so under /shopdb or /ops it
resolved against the SERVER root and 404d - siteSettings.js wraps every logo in
withBase() and this page was the one place that did not. With that fixed two of
three were still invisible, because the shipped GE mark is fill="white" for the
dark sidebar and was rendering white-on-white. The preview tile now sits on a
mid-tone checker so artwork of any colour reads against it.
The floor-map levels table ran off the right with its cells crushed - nine
columns of inputs, no scroll wrapper, and .data-table is styled nowhere. My
first attempt reached for the global .table-container and made it WORSE: that
class also sets white-space: nowrap, and a full sentence of hint text lives in
one of the cells, so nowrap forced the table wider still. It now scrolls in a
local wrapper, the hint wraps, and the name input has a width instead of
collapsing to one character.
Suite 2377, frontend 324, gate green.
Scope: product
One commit from the sister site, purely additive: a read-only `batches` command
so an operator working from the shell can find a batchid to undo, and a README
covering the four WINC commands. Two new files under scripts/site_imports/winc/,
nothing existing touched.
Verified rather than assumed: it changes none of the files fixed in 13be0ef9
(the workflow YAML, the approval check, the agentpackage description, the upload
audit), the gate passes, all five workflows still parse, and the project map is
current.
Suite 2369.
Scope: product
Eleven commits from the sister site: a reversible Maximo/WINC machine-adoption
import, a versioned agent binary store with upload and rollback, remote-BuildKit
CI, tray version reporting, and UI fixes. Applied as a patch again - the
published mirror rewrites hashes, so the branches have unrelated histories and
git refuses to merge them.
Reviewed by three agents before merging. Four things fixed here.
THE PUBLISH WORKFLOW WAS DEAD. The new remote-BuildKit step in
publish-images.yml sits at column 0 instead of indented under steps:, so the
whole file is invalid YAML and GitHub would refuse it. Their entire GHCR
distribution channel - which Start-ShopDBDev.ps1 and the compose files consume -
was broken at the source and clearly never run. Indented; all five workflows now
parse.
THE IMPORT'S APPROVAL CHECK COULD NOT FAIL. commit() built the plan and the
approval packet from the same upload and then asked the writer whether they
matched, so packet_matches_plan compared a hash against itself. The documented
contract - "a human approves a specific, frozen plan, and the writer can prove
nothing drifted" - did not exist in the web path: an admin could preview file A
and commit file B, gated only by the needs-create group NAMES they ticked. The
preview's planhash is now required at commit and checked against a freshly
rebuilt plan. Their tests committed without previewing, which is what let this
through; they go through the real flow now, and three new tests pin it - all
confirmed to fail with the check stubbed out.
endpoints.agentpackage QUIETLY GREW. It used to mean "wrap the staged exe in a
ZIP"; with the upload routes it also means "choose the executable every future
package installs and runs as SYSTEM". Before this, staging needed filesystem
access to the server. The permission description still said only "generate and
download a package", so an admin granting it could not see what they were
granting. Reworded.
NOTHING RECORDED WHAT WAS STAGED, OR BY WHOM. The manifest held version, size
and timestamp; re-uploading a version replaces the bytes in place under the same
label. A sha256 of the stored file is now recorded, and the upload is
audit-logged with it - after an incident there was previously no answer to who
put that binary on the fleet.
Also documented the nine new routes in docs/api-inventory.json (that gate cannot
run on the mirror, so it falls to us at promotion time again), regenerated the
spec and the project map, and kept our own docs where they conflicted: docs/ is
scrubbed from the mirror, so their DEV-BUILD-TEST.md still had the dev-be
channel mapping we corrected, and their PLUGIN-HOOKS.md was a new 45-line file
rather than our 737-line canonical one. Their journal section is appended to
ours.
Contract version comes across as 0.30.0 for a genuinely additive surface.
Suite 2369, frontend 324, gate green.
Scope: product
The USB kiosk had a feature flag and nothing else. A flag is a decision, not a
credential: with the kiosk on, anyone who could reach the app could move a drive
between checked-out and checked-in and sign it with a badge they guessed. Same
reasoning that closed the parts kiosk, smaller blast radius - which is why
Darkwing did not raise it and why it was worth asking about anyway.
Both kiosks now share one mechanism. The device token moved out of the
printedparts api wrapper into a request interceptor that attaches X-Kiosk-Token
by URL prefix, with one localStorage key per kiosk - they are separate screens
in separate rooms, and a shared token would mean provisioning one grants the
other. claimKioskToken() is shared too, so both views take the token from a
?token= URL once and strip it from the address bar identically.
Then the part that matters more than either fix. I told you to set the new
settings in the UI, and that was impossible: every plugin settings page carries
a hardcoded key list, and nothing had added them. printedparts_kiosk_enabled,
printedparts_kiosk_token and usb_kiosk_token were declared, seeded and read
correctly, and rendered by no page - changeable only by writing SQL.
tests/test_plugin_settings_reachable.py now fails when any declared setting
appears nowhere in its plugin's frontend. It found seven MORE that were already
in that state, all pre-existing:
collector_corporate_ranges computers_configpolicies_enabled
computers_machinelink_alerts computers_quietreporthours
geenforce_reportstaleminutes printers_dashboardpercent
warranty_expiringdays
All seven now have fields. I first parked them in a KNOWN_GAPS ratchet, then
fixed them and deleted the ratchet - a backlog list in a new check is a way of
agreeing the check does not apply yet. There is no backlog; the check is
absolute.
Two of my own mistakes on the way, both caught by tests rather than by reading:
the blanket rename of the USB guard also rewrote the call inside _kiosk_open, so
it called itself until the stack ran out; and the first version of the reachable
check skipped all 17 plugins because it looked for get_settings() when the
method is get_settings_defaults() - it passed by testing nothing, which is the
exact shape it exists to catch.
Suite 2336, frontend 316, build clean.
Scope: product
printedparts_kiosk_token declared valuetype 'secret'. The column documents
string, boolean, integer and json, and that was the only 'secret' in the
codebase - it worked only because Setting.get falls through to the raw string
when it recognises nothing.
Masking never depended on it: settings.py::_is_secret keys on the NAME
containing 'token', so the value is still hidden in API reads and in the audit
log. Changed to string, which is what it is.
Scope: product
Both from the second Darkwing scan. Every claim in it was verified against the
code before acting.
The kiosk was the product's only unauthenticated write, justified by a decision
record saying it was "physically rate-limited: it serves a touch screen on the
shop floor". Nothing in code enforced the physical half, and an HTTP route is
reachable from wherever the app is. What that left open:
- _kiosk_find_item falls back to matching bare digits against the numeric
tail of every active item, so 1, 2, 3 walk the catalogue without knowing a
barcode;
- a badge is a short numeric SSO, and the route answered 422 for an unknown
one and 200 for a real one - an oracle for sorting guesses into employees;
- the write then decremented stock and stamped the ledger, which IS the audit
trail, with whatever SSO the caller supplied;
- quantity was bounded only by what was on hand, so one request emptied a bin.
A badge is attribution, never authorization. Both kiosk routes now need the
site's kiosk flag AND a per-device token, and 404 rather than 403 without them
so a caller does not learn the door is there. The READ is gated too: gating only
the write would have left the enumeration oracle running against a kiosk a site
had switched off.
The token is per-DEVICE, not per-page, because a kiosk is a browser on a touch
screen and a token served inside the page is readable by anyone who can load it.
An admin opens the screen once as /parts-kiosk?token=..., the page moves it into
that browser's localStorage and strips it from the address bar.
Also: quantity is capped per take, JSON true no longer passes as 1
(isinstance(True, int) is True), and every badge failure on the anonymous route
returns one message. The gated restock and adjust handlers keep the specific
message - there the person is known and the detail is help, not an oracle. I
changed the wrong one of those three handlers first; caught by the test.
/api/dashboard/health formatted str(e) straight into the body of a route
PUBLIC_ENDPOINTS declares open because "monitoring has no account". A pymysql
OperationalError names the database account, the app host's address, the driver
and the failing statement. It is logged server-side now and the body carries a
coarse status. The version goes with it - it told an anonymous caller which
release to look up. Status stays 200 either way: a monitor keyed on the body's
status field is the contract here, and turning a degraded database into a
transport failure is a separate decision from not leaking the exception.
Nine tests. The three gate tests were confirmed to fail with the gate stubbed
out, rather than assumed to cover it. Suite 2325, frontend 316, build clean.
Scope: product
The previous commit said the link-resolving change had never met a real
filesystem. Run in the win11 VM against real fixtures, it turned out to reject
EVERY legitimate launch path.
ResolveDirectoryChain walked up looking for a junction and eventually asked
whether "C:\" was a link. That throws DirectoryNotFoundException, and the caller
treats a resolve failure as a rejection, so the first legitimate case came back
"cannot resolve the real path: Could not find a part of the path 'C:\'". The
agent would have failed closed on everything - an outage wearing a security
fix's clothes. It compiled with zero warnings and read correctly.
Stop at the volume root, and never let "cannot tell whether this is a link" turn
into a rejection: TryResolveLink returns null on any failure, and the assembled
path is re-checked against the roots regardless, so an unresolvable component
still cannot widen what is accepted.
One expectation in the probe was mine being wrong rather than the code. 8.3
short names used to be rejected because the lexical prefix check could not match
C:\PROGRA~1 against C:\Program Files - fail closed, but by accident of not
understanding the path rather than by intent. Resolution now expands it to the
real directory, which is inside the approved root, so accepting it is correct.
Verified on Windows, all ten cases:
accept a real exe under the approved root
accept 8.3 short name resolving into the approved root
reject exe reached through a junction pointing out of the root
reject alternate data stream on a file in the root
reject traversal back out of the root
reject UNC path
reject sibling directory sharing the root's prefix
reject wrong extension
reject file that does not exist
reject empty path
The probe is committed as agent/ShopDbEndpointAgent.PolicyProbe so this is
repeatable. It is NOT in CI and its README says why: junctions, alternate data
streams and 8.3 names do not exist on Linux, and a suite that skipped every case
it exists for would report green and mean nothing. This repo has been bitten by
that shape more than once already.
Scope: product
check to the file it actually runs
Two clusters the five-agent review found. Both are about a product that another
plant has to be able to install.
ADR-015: the shipped monitor bundle was the originating plant's fleet
convention frozen into product seed.
The business policy was seeded as "RPCA Shopfloor App Monitors". RPCA is that
plant's facility designator, the row is source='shipped', and the API refused
to rename it - so every adopting site would carry another plant's label at the
top of its policy list forever. 0007 now seeds the neutral name, and 0010
renames it on installs that already stamped 0007, guarded so it cannot collide
with a policy a site made itself. Downgrade is one-way on purpose: putting
another site's designator back is not a state worth reaching.
Which PC type the bundle attaches to was the literal 'Shopfloor PC'. A plant
whose type is "Shop Floor Workstation" silently got no shipped monitors at
all, and one using that name for something else got all of them. It is a
setting now, defaulting to the seeded name so nothing changes here, and a
blank value attaches the bundle to nothing - the right neutral answer for a
site that does not want it.
Shipped rows refused every edit, including isactive and launchpath. A plant
whose eDNC is on D:, or that runs no UDC, could not correct the path or switch
the monitor off, and then carried permanent false drift for it. ADR-015 allows
seed data BECAUSE a site can edit it afterwards. Paths and the active flag are
now editable; kind and displayname stay fixed, because that pair is the
identity 0007's bundle matches on.
Also corrected two false claims in 0007's docstring: the policy tables belong
to the computers plugin, not the core baseline, and a guarded no-op does NOT
reconcile later - Alembic stamps the revision either way, so the bundle would
simply never seed.
LaunchPathPolicy was purely lexical. GetFullPath is string normalization that
never touches the disk, so the root check constrained the NAME while File.Exists
and CreateProcess followed reparse points to somewhere else. A junction under a
writable directory in an approved root - creating one needs no privilege, only
write access to the parent, and vendor trees under Program Files are not
reliably locked down - turned WebUI monitor-edit rights into code execution as
the console operator. It now resolves the real target, leaf and directory chain
both, and re-checks THAT against the roots, rejecting if resolution fails.
Alternate data streams passed the whole policy on net8.0: an interior colon no
longer throws in GetFullPath, so "readme.txt:payload.exe" reported an .exe
extension, passed the root check, and existed. Rejected explicitly now.
Six tests added for the shipped-row and pctype behaviour. The Core project
compiles clean. The launch-path change has NOT been exercised against real
junctions, ADS or 8.3 names - that needs the win11 VM, and per the reviewer it
wants a planted-violation suite there rather than my say-so.
Still open from the review: monitor revocation does not take effect until agent
restart, ActionPicker is keyboard-inoperable, the lightbox has no focus trap,
one attached monitor grants generic any-target process control, and screenshots
are retained unencrypted in the user profile.
Scope: product
Findings the reviewers verified, fixed here. The larger cluster they found is
NOT fixed and is listed at the end.
Frontend. reportImageUrl is called from the template, so it ran on every render
with no in-flight guard: a report whose fetch was pending was re-requested, and
because each resolution reassigns the map the whole grid re-rendered and
re-fired the rest, costing up to N-squared requests for N thumbnails. Two blobs
resolving for one key orphaned the first object URL where revoke could not
reach it, and a failure was never recorded so a broken image retried forever.
Guarded, deduplicated, failures remembered.
Three CSS variables did not exist, so their hardcoded fallbacks always won:
--bg-alt (the "No screenshot" tile, near-black in light mode), and
--warning-text/-bg/-border (dark khaki on a dark modal). Replaced with real
tokens. The DEV badge's literal #c62828 is var(--danger).
Installer and launcher. The generated site installer ran docker compose with no
-p, so the project name defaulted to the package folder - which carries the
version. Installing the next version created a new project with new empty
volumes and the site came up looking wiped, old data stranded under the old
project. Pinned to shopdb-<channel>.
Start-ShopDBDev deleted any existing -TargetDir that lacked a .git, recursively
and without asking. For a script handed round as "save it anywhere and run it",
a mistyped -TargetDir pointed at Documents would have destroyed it. It now only
reclaims a directory that is empty or carries this launcher's own marker, and
refuses otherwise.
Dev secrets came from Get-Random, which is System.Random: a 32-bit seed, and all
four values off one stream, so recovering one recovers the rest. They sign JWTs
for a stack a pilot may point at real imported data. CSPRNG now, and the API
port binds to loopback like the DB rather than every interface.
Two of my own from earlier in this session. The operator-report size check sat
below request.form, and the first access to that parses and spools the whole
multipart - so it ran after the thing it prevents. Moved above. And the channel
remap missed build-channel-installer.ps1 entirely, where it set BOTH channels to
main: a "stable" package built from the trunk, the exact mislabel the remap was
for. stable now resolves the newest v* tag.
Also removed a zero-byte docs/ShopDB-Planner-Tasks-v2.csv that rode in.
NOT FIXED - needs a decision, and it is the biggest thing the review found.
Migrations 0006/0007 freeze the originating site's fleet convention into
product seed: a shipped policy literally named "RPCA Shopfloor App Monitors",
a hardcoded 'Shopfloor PC' type deciding who gets the shipped watch-list, and
shipped rows the API refuses to edit or deactivate - including their launch
paths. A site whose eDNC is on another drive cannot correct it and gets
permanent false drift. Those are stamped migrations, so this gets harder to
undo at every install. See the review notes before anyone upgrades a second
site through 0006-0007.
Suite 2314 passing, frontend 316, all three PowerShell files parse.
Scope: product
Six commits from wilm-import: the squashed import, route documentation, the
channel remap onto our tag-and-trunk model, the project map, parallel test
support, and the four smaller review findings.
Merged rather than cherry-picked: the published mirror rewrites hashes, so the
commits do not apply one at a time against our history even though the squashed
diff applies clean.
Scope: product
# Conflicts:
# docs/PROJECT-MAP.md
None blocked the merge; all four were mechanisms that did not mechanise.
_RUNNING_STATES existed twice, in api/routes.py and services/reconciler.py, with
a comment asking a human to keep them in lockstep. The panel and the reconciler
read the same agent snapshot, so a drift between the copies would paint a
service green and flag it as drift on the same page. One definition now, owned
by the reconciler and imported by routes - api -> services is the direction the
plugin layout already runs in, and the reconciler's import of routes is deferred,
so there is no cycle.
The operator-report upload checked its 10 MB cap AFTER upload.read() had already
pulled the whole body into memory, so the check could not prevent the thing it
was there to prevent. content_length is now an early out and the read itself is
bounded to cap+1, which is enough to know the cap was exceeded without taking a
body that lied about its length. The multipart envelope carries the other form
fields too, so the early out allows 64 KB of slack rather than rejecting an
image that is just under.
The same handler forced the extension to .png and the mimetype to image/png
whatever the bytes were, then served the file back under that type. The upload
headers (CSP sandbox + nosniff) make the document case inert, so this was never
the hole it looks like, but nothing is gained by storing a file we already know
is not what it claims. The PNG signature is now checked.
The lightbox bound Escape and the arrow keys to a div and never focused it, so
every one of those bindings was dead until the user happened to click the
backdrop. It takes focus on open and hands it back to the thumbnail on close.
On the two upload tests, honestly: the PNG one was confirmed to fail with the
check removed. The oversize one passes either way, because the old code did
reject an oversized image - it just did so after reading all of it. That test
pins the response, not the fix. Whether the read is bounded is not observable
through the response, so nothing here proves it; the change stands on reading
the handler.
Suite 2311 passing, frontend 316 passing, build clean.
Scope: product
The suite had grown to ~2300 tests and 5m30s. Profiling says there is no hot
spot to fix: the 25 slowest tests sum to about 60s of 331s, and the rest is
per-test fixture cost. The db fixture drops and recreates all 94 tables for
every test, measured at 0.058s a cycle, which is roughly 40% of the wall clock
on its own.
That shape parallelises almost linearly, so pytest-xdist buys more than any
rewrite would:
pytest -q 2309 passed in 331s
pytest -q -n auto 2309 passed in 103s (8 cores)
Deliberately NOT in pytest.ini addopts. Worker startup is dead weight on the
targeted single-file runs that make up most of a working day; the whole-suite
run before a push is where it pays. Opt in with -n auto.
A run distributed across 8 workers also has to be order-independent to pass at
all, so this is incidental evidence that the suite has no inter-test coupling.
Recompiling requirements-dev.txt to add xdist also moved idna, packaging and
pyjwt. That is the -c requirements.txt constraint doing its job rather than a
drift: all three ARE shipped, and the dev file had fallen BEHIND production
(idna 3.13 in dev against 3.19 in prod). CI was testing against older versions
than sites install, which is the exact failure the header comment on that file
was written about. Now matched.
Scope: product
The channel work assumed the branch model it was written against: upstream,
main is a release branch and dev-be is the trunk. Neither holds here. main IS
our trunk - 212 commits past v0.11.3 - and a release is a tag (ADR-007). dev-be
does not exist at all, so the edge half would never have fired and the launchers
would have cloned a missing branch.
Taken literally it would have published our unreleased trunk to every site as
:stable, which is the opposite of what the word promises to someone who has no
way to check.
Remapped throughout:
tag v* -> :stable the thing that was actually released
main -> :edge the trunk, DEV badge lit
publish-images.yml keys the channel on github.ref_type = tag rather than on a
branch name, so the mapping cannot silently drift back. Start-ShopDBDev.ps1,
dev-bootstrap.ps1 and build-channel-installer.ps1 resolve stable to the newest
v* tag via git ls-remote --sort=-v:refname, warning and falling back to main if
a repo has no tags yet. DEV-BUILD-TEST.md follows.
The push: trigger is commented out. Automatic publishing waits on someone
confirming the GHCR package exists under this owner and deciding who may pull
it; workflow_dispatch works meanwhile. Turning it on is one uncomment.
All three scripts parse clean under pwsh, and the tag-selection expression was
run against this repo rather than eyeballed. Per the standing rule, the runtime
paths still want a pass in the win11 VM before anyone leans on them.
Scope: product
Darkwing's reconnaissance flagged POST /api/printedparts/kiosk/take as an
unauthenticated write reaching a ledger commit and an outbound mail/webhook
trigger. Analysis dropped it, correctly: it is a declared exemption with a
decision record. Nothing to fix in the code.
The comment beside it was wrong, though. It described the USB kiosk writes as
open at the "same bar as the parts kiosk" while listing a gate the parts kiosk
does not have: usb_kiosk_enabled, off by default, every route a 404 until a site
sets it. grep -rn kiosk_enabled plugins/printedparts/ returns nothing.
So a site that adopts the product and has no parts kiosk still serves a live
unauthenticated inventory-decrement endpoint with no switch to turn it off, and
the registry that is supposed to be the record of these decisions said the
opposite. Whether printedparts should gain the same flag is a behaviour change
for a kiosk that is in daily use here, so it is left as a decision rather than
made silently; this only stops the comment misleading the next reader.
Scope: product
Found by the Darkwing scan of main (run 33654922078). Its own review dismissed
the finding as not remotely exploitable and it is right about that: nothing can
make a victim's browser attach a custom header, and deploy/windows/web.config
does no output caching. It still recommended the hardening, and the reasoning
for the dismissal is a property of OUR IIS config, not of the product - every
site runs its own instance behind its own proxy under ADR-004.
_batch_base_url() interpolated a caller-supplied host into the single-quoted
PowerShell -Uri literal of the .bat this route hands out. site_base_url ships
blank, so that fallback is the default path on a fresh install, not an edge
case, and GET /api/printers/install-batch is public.
Three rules now:
- the host is validated as a hostname before it is interpolated, so the quote
that would end the literal never reaches it,
- X-Forwarded-Host is only believed when TRUSTED_PROXY_COUNT declares a proxy
-- the rule shopdb/utils/clientip.py already applies to X-Forwarded-For.
That was centralised for this exact reason and did not cover the Host
variant,
- a request supplying neither usable header gets a relative base, which still
downloads correctly from the browser that fetched the script.
Three tests pin it, and all three were confirmed to fail with the old body
restored rather than assumed to cover it.
Scope: product
Two of the three things holding the wilm import out of main.
The monitor CRUD, the two resolver reads and the three operator-report routes
had no entry in docs/api-inventory.json, so test_api_inventory_parity failed.
The gate cannot run on the published mirror - the test ships but its data file
is scrubbed, and it skips on the file's absence - which is why the branch that
added the routes never saw it. Cost lands here at promotion time instead.
/api/endpoints is registered both with and without a trailing slash and both
forms are served, so both are described rather than guessing which one counts.
Spec regenerated: 353 paths, 504 operations.
plugins/endpoints/plugin.py had a contiguous run of continuation lines shifted
by one space. Python accepted it and git diff -w showed nothing, but it put
deletions in a diff that is otherwise purely additive. Restored; the file now
differs from the published main by additions only.
Scope: product
GE Aerospace now expects a Darkwing check on every repo. Two files rather than
one, because they answer different questions and only one of them is ours.
scan-darkwing-portable.yml is copied byte-for-byte from the ge-aero/darkwing
README so it keeps matching the sanctioned integration. Manual dispatch only.
scan-darkwing-pr.yml runs on pull requests and is ours to change. It exists
because the README workflow cannot simply be given a pull_request trigger: it
resolves its target as `inputs.target_ref || 'main'`, and a pull_request event
carries no inputs, so the expression falls through to 'main' and the job scans
the default branch instead of the code under review. That is a check that is
green because it never looked. This one resolves the PR head SHA explicitly.
Neither file gates a merge. Darkwing exits 0 whatever it finds, and its results
vary between runs on identical code, so a passing job means the scan ran and
nothing more. The SARIF-parsing step that would turn a finding into a failure is
written but commented out, to be switched on once the false-positive rate on
this codebase is known.
Scope: product
Everything on origin/wilm that is not on the published main, applied as one
change because the published history has rewritten hashes: per-commit
cherry-pick conflicts against our main even though the squashed diff applies
clean.
Contents: endpoint monitor catalog + policy attachment + warn-only drift
reconciler, operator issue reports from the tray, agent launch-path allowlist,
and the GHCR channel-image distribution path for dev workstations.
NOT READY FOR main. Open items:
- 10 new routes are absent from docs/api-inventory.json, so
test_api_inventory_parity fails. That gate SKIPS on the published mirror
(the test ships, its data file is scrubbed), which is why wilm never saw it.
- dev-bootstrap.ps1 and Start-ShopDBDev.ps1 hardcode the ge-aero GitHub org.
- plugins/endpoints/plugin.py carries a whitespace-only re-indent.
Scope: product
Eight fields across three asset types were shown on the detail page and
accepted by the API, but had no input on the form. A wrong value therefore
needed database access to correct, which is not a thing anybody should need
for a device category or a maintenance date.
The shape is the same in each case: a column was added, the detail card was
written, the API allowlist was updated, and the form was missed. Nothing was
broken - the values arrive by import or collector - so nothing complained.
PCs gain azureobjectid and azuredevicecategory. azuredeviceid was already
there, which is what made the omission easy to miss: the page showed two ids
and the form offered one. The two are not interchangeable and the pairing is
counterintuitive - the DEVICE id opens Intune, the OBJECT id opens Entra - so
each input says where it goes.
Machines gain the three maintenance fields. These matter beyond the immediate
fix: they are the columns a measuring instrument's calibration dates map onto
when metrology moves to machines, and until now there was no way to enter a
calibration date on a machine at all.
Printers gain iscolor, isduplex and isnetwork as a capability group.
isnetwork defaults to true in the form to match the API's create default; a
new printer claiming otherwise would be wrong for nearly all of them.
Checked before making any of these editable that nothing writes them
automatically - a hand edit silently reverted by the next collector poll would
be worse than no input at all. The printers collector reports queues and
supplies, not capabilities, and nothing writes machine maintenance dates,
which is exactly why they are all empty. The two genuinely collector-owned
computer fields, lastreporteddate and lastboottime, are deliberately still not
editable.
Scope: product
A CMM reported 100.0.0.1 as its address and never reported 10.48.255.210 at
all. Both halves of that were the same mistake.
THE BAY WAS CHOOSING. The collector schema took a single `ipaddress`, so the
script picked one NIC and dropped the others - preferring an address inside a
configured range, else whichever interface carried a default route. A bay
cannot make that judgement: it does not know the site's addressing. On this
CMM the Marvell AQtion 10Gbit link to its controller is a real hardware NIC,
up, and carries a default route of its own, so it satisfied every test the
script had. It won, and the corporate address was discarded before the request
was even built - which is why it is not in ShopDB even as a second row.
So the bay now reports EVERY connected hardware NIC with a routable IPv4, and
takes no view. The server picks the primary from collector_corporate_ranges, a
new per-site setting, and writes a Communication row for each of the others.
Naming the ranges is what turns the answer from a guess into a decision; with
none set it still falls back to the default-route flag and then to the first
reported, both guesses, both better than before only because nothing is thrown
away now.
Rows are keyed on the MAC where the bay sends one, because an address moves
between adapters and a MAC does not. A NIC that is down at check-in is left
alone rather than deleted - being down is not evidence the record is wrong,
and hand-entered gateway, DNS and serial settings live on those same rows.
Exactly one row ends up flagged primary, and only a row this report touched
can claim it, so a check-in cannot silently demote an address set by hand.
-AllowedRanges and the CollectorRanges registry value are now accepted and
ignored, logging a line that says where the setting moved. The parameter stays
because a scheduled task still passing it would otherwise fail on an unknown
argument.
A bay on an older collector sends one address and no interfaces; that keeps
working and is pinned by a test.
Scope: product
Twenty computer types existed and six had a machine on them. Three of the
empty ones were near-duplicates of live types - Shopfloor beside Shopfloor PC,
CMM beside CMM PC, Engineer Workstation beside Engineer - which is the kind of
list that grows because deleting from it by hand is nerve-wracking.
retype-by-role types a PC by WHAT IT CONTROLS. A metrology box has an active
relationship to a measuring_tool asset, so the fact is already recorded and
does not need inferring from a hostname or a tool's name. The imaging scope is
the authoritative source and stays so, but only 17 of roughly 292 PCs have
ever reported one, so it cannot answer for the fleet yet. Relationships can:
49 PCs drive a measuring tool, and 36 of them are currently typed as ordinary
shopfloor machines.
prune-types deletes a type only when nothing is on it AND the live pctype map
does not name it. The second guard is the one worth having: the map is what
the collector assigns from at check-in, so deleting a mapped type would leave
every future check-in logging 'mapped ComputerType not found' instead of
typing the machine.
Both report by default and write only with --apply, and prune prints why it is
keeping each type it keeps rather than only what it removes.
Also fixes `computers stats`, which counted the computer type named
'Shopfloor' - the EMPTY duplicate - rather than 'Shopfloor PC'. It has been
reporting 0 shopfloor PCs for as long as both rows existed, which is the same
duplication these commands clear up.
Scope: product
The Entra Device Info card showed two GUIDs and nothing to do with them.
Copying one into the portal by hand is the kind of errand a detail page exists
to remove.
Each link sits BESIDE the id it is built from rather than in a row at the top
of the card, because the pairing is counterintuitive: the DEVICE id opens
Intune and the OBJECT id opens Entra. A separate action row would have to
re-label which is which, and would still leave the reader matching them up.
BOTH URLS ARE SETTINGS WITH AN EMPTY DEFAULT, and that is the whole design.
The host is a tenant fact, not a product one - this site is on the US
Government cloud, portal.azure.us, where a commercial-cloud default would send
people to a page that does not exist. Microsoft also moves these blade paths,
and a site that hits that can fix it by editing a setting rather than waiting
for a release. Empty means no link and the id still renders as text, so a site
that configures nothing is unchanged. The example values live in each
setting's description, per cloud.
Same shape as the ServiceNow URL templates already in integrations, down to
the {token} placeholder, so there is one idiom for "deep link into a system we
do not own" rather than two.
Scope: product
An Azure device import died part-way with a foreign-key error, and the crash
was the lucky outcome.
The Azure adapter routes a device that already exists to its real asset by
writing that asset's assetnumber into the assetid column of the CSV it
generates, expecting the engine to look it up. The resolver instead takes any
all-digit string at face value as a literal id. This site numbers operations
0NNN, so numeric assetnumbers are ordinary here: `0704` was read as id 704,
which does not exist, hence the error. `0600` would have been read as id 600 -
a printer - and bound a PC to it silently, reporting success.
Two changes, both needed.
The resolver now tries the label first for an all-digit string and only falls
back to reading it as an id when nothing is named that. The face-value
behaviour exists for a real reason - exports round-trip and carry real ids, as
its original comment says - so it stays as the fallback rather than being
removed. Somebody writing `0600` in a column keyed on assetnumber means the
asset called 0600, and binding the row to an unrelated record is far worse than
failing.
_existing_hostname_map now hands over the assetid rather than the assetnumber.
Giving the resolver the id it is going to treat the value as makes the
passthrough correct by construction instead of correct by luck.
Verified against a restored copy of the production database: 234 devices, 221
matching an existing computer and updating in place, 13 new, no problems. The
same run previously failed on the foreign key.
Neither half was wrong alone. The passthrough dates from the original CSV
import and the hostname map from the wilm merge months later; they only meet
at a site whose natural keys are numeric.
Scope: product
"Metrics" promised numbers. The tab holds tracked services, tracked processes
and network interfaces - things being WATCHED and their current state, only
one of which is a measurement. Monitors is what the panels actually are, and
it matches the word the agent's own config already uses for them.
Moved to second, ahead of Software. What is being watched on this box is the
question asked right after "what is it", and well before its application
inventory - which is reference data somebody goes looking for deliberately.
Tab order is now Details, Monitors, Software, Activity.
Contract 0.29.0 keeps its number. It is unreleased, so the group vocabulary it
introduces is still being settled and nothing can yet depend on the old name.
Both the server and the client fall back to 'details' for a group they do not
recognise, so a site that deploys these halves out of step gets a misplaced
card rather than a missing one.
Scope: product
SOFTWARE OPENED COLLAPSED. The installed-software panel was collapsible and
started that way, because it used to be low-priority reference data near the
bottom of one long page. It is now the Software tab's whole reason to exist,
and somebody who clicked that tab has already said what they came for. Made
it neither collapsible nor collapsed.
TRACKED SERVICES VANISHED FROM METRICS. The group is decided by the SERVER, so
a new frontend against a server still serving the old vocabulary puts a panel
in a group the build does not render, and it is fetched, held, and shown
nowhere - with nothing logged. Moving services out of 'actions' is exactly that
shape. The client now falls back to 'details' for any group it does not render,
the same way the server does, so a version-skewed pair degrades to a misplaced
card instead of a missing one.
A TABLE PANEL WITH NOTHING IN IT HID ITSELF. panelVisible honoured `empty` for
list panels and ignored it for tables, for no reason anybody wrote down. A
declared empty message is an answer - "No installed software reported." - where
an absent card is a question. It matters more now that a panel can be the only
thing in a tab, because the tab disappears with it.
THE TAB STRIP HAD A VERTICAL SCROLLBAR. Per the overflow spec, when one axis is
not `visible` the other computes from `visible` to `auto`, so `overflow-x: auto`
alone made the strip scrollable vertically too and the -1px margin the tabs use
to sit on the border was enough to trip it.
Also moved .detail-full into the global stylesheet next to .content-grid, per
frontend/CLAUDE.md: it is detail-page layout, and the other four detail pages
need it when they adopt tabs.
Scope: product
Four changes from looking at the built page.
ACTIONS HELD ONE CARD. Tracked services was the only thing in it, and it was
the wrong split anyway: a service row reads as state first and a Start/Stop
second, so separating the buttons from the state they act on made a tab that
answered no question on its own. Services move to Metrics, beside the rest of
what the endpoint reports, and the group is gone from the contract.
WIDE PANELS SPAN, THEY DO NOT SCROLL. Network interfaces carries adapter,
address, MAC, speed and state; at half page width in the two-column flow it
either squeezed every cell to three lines or offered a horizontal scrollbar.
A scrollbar for data that just needs room is a layout failure. Table panels
now take column-span, which is the multi-column layout's own answer, so short
fact cards stay two-up around them. The wrapper keeps overflow-x only as a
floor for a genuinely narrow viewport.
SOFTWARE AND ACTIVITY RUN FULL WIDTH. Both are single long lists, and a list
split into two balanced columns reads as two unrelated lists with no way to
tell where the first ended. Details and Metrics keep the two-column flow
because they are short cards, which is what that flow is good at.
Contract 0.29.0 keeps its number: it is unreleased, and the group vocabulary
it introduces has simply lost one value before anybody could depend on it.
Scope: product
The page had grown to roughly ten cards in one flow, and three more surfaces
were wanted on it: installed software, live metrics, and the service controls.
Stacking those into the same column would have buried what the page is for.
Tabs were mocked and rejected on 2026-08-28, on the grounds that the strip
duplicated the sidebar and cost hero space. That objection is answered by
where the strip sits: the hero and the triage band stay above it and never
move, so "is this machine OK" is still answered without a click, and the strip
partitions the detail underneath rather than navigating anywhere.
WHY A PANEL DECLARES ITS OWN GROUP
Most cards on a detail page are not the page's. They arrive through the
ADR-010 get_asset_panels hook, so a page that sorted panels into tabs by
knowing their ids would defeat the hook it is built on. Panels now carry an
optional group instead, and the page only knows the five group names.
Resolution happens server-side, in the asset-panels route, so that an absent
or misspelled group becomes 'details' before the client ever sees it. The
client renders known groups only; an unrecognised value that survived to it
would be a panel that exists and is displayed nowhere. Contract 0.28.0 ->
0.29.0, additive - every existing plugin keeps working untouched, and the
endpoints status panel is deliberately left ungrouped so the default path is
exercised by a shipped plugin and not only by tests.
A tab is offered only when something fills it. An empty tab reads as broken
rather than as "this site does not collect that", so the strip is built from
the groups that actually returned data, and a tab that disappears on refetch
falls back to Details rather than leaving the page blank.
AssetDetailTabs is core, not a PCDetail component: machines, printers,
network devices and measuring tools all have the same shape, and the shared
asset-detail shell they need starts here.
Scope: product
A link is built by formatting that protocol's own linktemplate, and the
templates are rows in accessprotocols that a site can edit. So no scheme
is a fixed fact about the product, and writing one down as though it were
misleads whoever reads it next.
Two places did. The guest-grant docstring said the detail route resolves
"working rdp:// and vnc:// links", which is wrong for the row we actually
ship: VNC is com.realvnc.vncviewer.connect://{host}:{port}, the RealVNC
handler, not vnc://. And the protocol admin form offered vnc://{host}:{port}
as its placeholder, in the one screen where somebody authors a new template,
so the example contradicted the seeded data it sits next to.
Neither changes behaviour. _computer_access_links already reads the
template from the database and the frontend only ever renders the resolved
link, so nothing in the code assumed a scheme - only the prose did.
Scope: product
The sidebar box was drawn for anyone signed in, on the reasoning that
offering it to a signed-out reader gives them a control that can only
answer 401. That stopped being the right test when the guest role became
able to hold assets.view: the box was hidden from exactly the visitor the
grant was reopened for, and a guest could reach search only by typing
/search?q= by hand.
The client cannot decide this. Its hasPermission getter reads the signed-in
user's permission list and a guest has no user at all, so asking it about a
guest answers no for everyone. get_navigation already had the problem and
already solved it by letting the server decide, so capabilities does the
same thing and reports what the CALLER holds.
It reports booleans about the caller and never what the guest role holds in
general, so it tells an anonymous reader nothing the rendered page does not.
Public because it is the pre-login call that decides whether the box is
drawn at all.
Members who lack assets.view now lose a box that only ever 401'd for them.
On a failed call the box stays hidden rather than falling back to the old
signed-in test, which would put it back in front of those same members.
Scope: product
Inverting the grantable rule from the .view suffix to explicit declaration
dropped every permission that had qualified by how it was spelled. Two of
them were shipped guest features with grants already in the field.
guides.view: the nav item and list_guides both reason in comments about a
signed-out reader holding it, and the route is written around one - drafts
are filtered out, search matches the published revision so draft wording
cannot leak through a hit, and the author branches cannot open for a caller
with no user id.
tools.view: without it a guest holding tools.secrets has permission to use
Secure Send and no way to reach the page, because tools.view is what puts
the entry in the nav. The tools themselves run in the browser and touch no
data.
The failure was worse than a missing checkbox. update_role refuses the
WHOLE request when any submitted permission is undeclared, and the role
grid resubmits what the role already holds, so a site that granted either
of these under the old rule could no longer save the guest role at all -
including to tick assets.view, which is the grant the shop floor needs to
look up an asset number without signing in.
The test asserts both stay grantable, so narrowing the set again means
looking at what is already granted in the field rather than only at what
the catalogue declares.
Scope: product
Global search is gated on assets.view, and that was the whole check.
Behind it the route queried twelve domains: applications, the knowledge
base, guides, the staff directory, assets, measuring tools, USB devices,
printed parts, custom fields, notifications, hostnames, and vendor and
model reference data, plus addresses and subnets for an IP-shaped query.
None of them consulted their own permission.
So anyone who could look up an asset number could also read the staff
directory out of the HR database, the USB checkout history and who holds
which drive, and the network inventory. The SEARCH_DOMAINS settings look
like they bound this but they are site feature toggles, not
authorization.
Each domain now answers only to a caller who holds its permission.
Assets, custom fields and vendor/model stay on assets.view because all
three emit asset rows routed to asset pages.
caller_holds rather than require_permission: refusing the whole request
because one domain is off-limits would make search useless to anyone
holding less than everything. A domain the caller cannot see contributes
nothing and the rest still answers.
This is what makes assets.view safe to grant the guest role again. It
left the grantable set in 3474028 because of the leak above, and that
took the shop floor's asset lookup with it - a signed-out visitor at a
machine could no longer search for the asset number in front of them.
Now the grant reaches assets and nothing else.
The first version of the leak test passed with the gates removed. The
staff directory is an external HR database and is unreachable under
test, so the domain silently returned nothing and the assertion proved
nothing. It patches _employee_rows now, and fails when the gate goes.
Scope: product
(cherry picked from commit 0a01ddcf25)
Rows sorted on iscritical then on printer NAME. Critical is everything
at or below 5%, so a cartridge at 5% and one at 0% tied on the first key
and the alphabet decided which came first.
Wrong order on its own, and worse than it looks: the dashboard queue
keeps only the first few rows per source, so a printer that was
genuinely empty was truncated off the card while a fuller one held its
place. Reported from the floor as "starts at 5% and hides one at 0%",
which is exactly the two bugs compounding.
remaining is already normalised for waste cartridges - a full waste bin
reads as low remaining - so one ascending sort is worst-first for every
supply type, and iscritical falls out of it rather than needing its own
key. Name only breaks ties.
worstremaining is carried on the row because the supplies list is
rendered to text ("Black 0%") before it reaches the client, so the
number the sort needs is gone by then.
The toner forecast report was already worst-first (it sorts on daysleft)
and is unchanged.
Scope: product
(cherry picked from commit 6644a07ae1)
The export rsyncs the WORKING TREE, so .gitignore does not apply to it:
anything ignored-but-present reaches the publication repo unless it is
named as an exclude. devnotes/ is declared never-committed and holds
review writeups and chat handoffs, and it went straight through.
The scrub gate stopped it, which is what that gate is for, and nothing
was committed or bundled. But a gate failure is a stop, not a filter,
and the fix a hurried person reaches for is deleting their notes to get
a bundle out.
Same shape as the stray .env.bak that reached this tree once before, so
the rsync now carries a note saying why excludes live there rather than
in .gitignore.
Scope: product
(cherry picked from commit f1136d920d)
Delivery descriptors for run_script and update_app are resolved when the
agent claims a command, not when an operator queues it, and the command
row recorded no digest: the enqueue path resolved the descriptor purely
to validate it and threw the result away.
So what an operator approved and what an endpoint runs were two separate
lookups against mutable content, minutes or hours apart. Anyone able to
replace a script body or an installer package could do so in the gap.
The claim then re-resolves, the descriptor faithfully describes the new
bytes, the agent verifies correctly, and it runs as LocalSystem. The
audit trail names the operator who queued the original.
The gate on the verb and the gate on the payload were also different
grants: pushing needs endpoints.deploy, but replacing what gets pushed
needs only applications.edit.
pinnedsha256 records the digest as it stood at enqueue, and the claim
refuses a descriptor that has moved, failing the push with an audit row
instead of shipping it. Null for actions that deliver no content, and
for rows queued before this migration, which claim as before. A
descriptor that vanished entirely is not treated as tampering: that path
already sends the command through undescribed so the agent fails closed.
Separately, a result may now only finalize a command that is actually
running. That needs no agent change and closes two things: reporting on
a command that was never claimed, and overwriting a recorded failure
with a success.
The agent now sends its hostname on the result callback so the server
can check it owns the command it is finalizing. The shared collector key
does not say who is calling, so without it any agent can post a result
for any command id. The server treats the parameter as optional for
agents that predate this and enforces it when present, so there is no
flag day. This is a floor, not a fix: only a per-endpoint credential
makes the caller's identity provable, and that is tracked separately.
The .NET side is unbuilt here and needs a build plus a win11 VM run.
Scope: product
(cherry picked from commit d64b7122ab)
client_ip() read the first X-Forwarded-For hop unconditionally. The
login rate limit and the account lockout are keyed on that value and the
audit log records it, so rotating the header per request bought an
unbounded budget and an untraceable trail.
This walked around TRUSTED_PROXY_COUNT, which exists to decide exactly
this. The app leaves ProxyFix off by default and says why, ten lines of
comment away: ProxyFix trusts X-Forwarded-*, a client can send those
itself, and the login limiter is keyed on the address. Then client_ip
read the header anyway, so the stated protection never applied.
Resolution is now from remote_addr, which ProxyFix has already filled
from the header wherever a proxy is genuinely configured. Both cases end
at the same trustworthy source, and the port still comes off because ARR
forwards clientip:port and ProxyFix passes it through verbatim.
geenforce and tools had each hand-rolled the same mistake for their own
limiters. Rather than duplicate security-relevant logic a third time,
client_ip and strip_port join the shopdb.api surface (contract 0.28.0).
geenforce's _trusted_client_ip, which already refused the raw header for
its IP allowlist, is untouched and was the model for this.
The stability table was also missing rows for 0.26.0 and 0.27.0.
INSTALL-WINDOWS-IIS.md never mentioned TRUSTED_PROXY_COUNT, so a site
installed from it runs behind IIS with the setting at 0: every request
looks like it came from IIS, the whole site shares one rate-limit
bucket, and the audit log records the proxy for every action. It now
says so, and says not to set it when nothing is in front.
Three test modules simulated separate callers by setting
X-Forwarded-For, which is the header a real attacker rotates to escape
the limit. One of them, test_client_ip_prefers_forwarded_first_hop_
stripped, asserted the defect outright. The suite could not have caught
this. They now drive REMOTE_ADDR, with a regression guard that forging
the header buys nothing.
Scope: product
(cherry picked from commit 7260244b7e)
There is no token denylist in this product, so the JWT user loader is
the only place a deactivated account can be stopped, and it did a bare
lookup by id. Somebody deactivated at 09:00 kept a working access token
until it expired, and a refresh token that minted fresh ones for thirty
days.
/api/auth/refresh and the API-token path already checked isactive. This
makes the loader consistent with them rather than inventing policy.
Scope: product
(cherry picked from commit 9c1848fc61)
guest_grantable_permissions() opened any permission whose name ended
'.view' or '.read' to the guest role, on the reasoning that reading is
the safe default. That is true of a permission gating a list of asset
numbers and false of one gating bytes, and a suffix cannot tell them
apart. Twenty-four of eighty-five qualified and none had been decided:
applications.view reached the SYSTEM-context PowerShell bodies and
the installer binaries pushed to bays
backups.view returned the full payload that the separate
backups.download permission exists to protect
assets.view gated a global search that queries the staff
directory, USB history and network inventory with
no per-domain check of its own
audit.view usernames, addresses, the before/after of every
settings.view change, and every setting row - both admin
category, opened by naming alone
users.view gates nothing at all; list_users is
require_role('admin')
Every entry is now declared, by CORE_GUEST_GRANTABLE or by the plugin
that owns the permission, having read what its routes actually return.
Adding one is a code change, which is the point: it puts the decision
in a diff rather than in a suffix.
The two routes serving script bodies and installer packages move to a
new applications.download permission, which is not grantable, so a site
can still open the catalogue metadata to its floor without the bytes
going with it.
The declared set preserves the grants this site had already chosen:
applications.view, computers.view, endpoints.view, reports.view,
tools.secrets, warranty.view.
Two known gaps are recorded on the hooks that expose them rather than
silently carried: the computer serializer publishes every column plus
resolved rdp:// and vnc:// links, and the per-asset endpoint panels are
keyed on a small integer so walking ids reconstructs much of the roster
that endpoints.fleet exists to protect. Neither is widened here.
A shipped test was granting guests audit.view and passing.
Scope: product
(cherry picked from commit 347402896c)
A multi-agent review of the merged branch produced 29 findings that survived
adversarial verification. This clears the blockers and the four worth fixing
before it reaches main. The design held up well: path-traversal defence on all
four serving routes is clean, the agent's SHA-256 check fails closed, enqueue
authority is proportionate and audited, and no site literals or credential leaks
in the new C#.
THE TESTS NEVER RAN
pytest.ini sets testpaths = tests, so plugins/endpoints/tests/ was collected ZERO
times. Fifty-five lines of delivery assertions were dead code, and run directly
five of them failed. plugins/printedparts/tests/ had been invisible for longer.
That is the worst shape a gap can take: the suite is green, the coverage is
imaginary, and the collector auth on a download route could be deleted without
anything going red.
Both files moved under tests/. tests/test_no_hidden_tests.py now fails if a test
file appears anywhere pytest will not look; it caught a staged copy inside the
installer bundle on its first run, and was verified by planting a violation.
The five failures were stale expectations, and they split by cause. His: the
applications panel made six panels where his test said five, and 'applications'
belongs in KNOWN_PAYLOAD_KEYS now that it is persisted - his own test contradicted
his own feature. Ours: we renamed the events card to activity, moved the
fleet-wide cards to endpoints.fleet, and made the list route a bare route('')
which Flask rejects unless the blueprint is mounted under a prefix.
FOUR FIXES
The snapshot table's key was a 32-bit autoincrement on a table rewritten
wholesale every check-in, so it climbs at (endpoints x apps) per report and
exhausts on MySQL. Widened while the table is unreleased. Widening it broke every
insert on SQLite, which only autoincrements a column typed exactly INTEGER
PRIMARY KEY, so it carries a dialect variant - and nothing caught either fault,
because no test inserted into that table at all. One does now.
Ingest truncated displayname but not displayversion or publisher. On strict MySQL
an over-long registry value fails the whole check-in transaction, so one machine
with a malformed Uninstall entry goes silent entirely. SQLite does not enforce
lengths, so the suite would never have shown it.
update_app's version: branch pinned AppVersion.sha256, but that route resolves the
version to its APPLICATION and serves application.installpath - so the digest
described an artifact the route never sends, and where it was null the agent had
nothing to verify and failed open. It now pins the file actually served.
The same branch ended the delivery URL at the id, so an MSI arrived with no
extension and could not be routed through msiexec. It carries the filename now.
Agent packaging moved off require_role('admin') to a new endpoints.agentpackage
permission. The generated ZIP carries a live collector key, so this is a heavier
grant than deploying to one endpoint - and a role cannot be given to one person
for one job the way a permission can.
Full suite 2231 passed (2198 before, plus the 22 that never ran and 11 new),
frontend 314.
Commit 0c2b816 added Script to shopdb.api so a plugin can read the ShopDB-hosted script library that backs run_script delivery, but left __contract_version__ at 0.26.0. This bumps it to 0.27.0 so a plugin can pin core_version against the widened surface. Additive name, minor bump, matching the existing convention. The docs/PLUGIN-HOOKS wiki page (not tracked in-repo) still needs the Script entry.
Reverses everything the installer and site-agent generator create: stops and deletes the service, kills agent/tray processes, removes the tray Run key and the HKLM\SOFTWARE\GE\ShopDB credential hive in both registry views, drops the Defender path exclusion, and deletes the install directory and ProgramData tree. Every step is best-effort and idempotent so it is safe to re-run and safe on a partially installed host.
Terminate TLS in gunicorn using a host-generated cert mounted read-only under ./certs, so the browser sees a secure context. Secure-context-only web APIs (navigator.clipboard and others) refuse to run over plain http on a non-localhost origin, which the FQDN-accessed test box hits. Healthcheck now probes https with -k to match the TLS listener. A site swaps in its own cert or fronts this with its own IIS/nginx proxy.
Bumps the agent to 1.4.1. run_script fetches a ShopDB-hosted script by digest and runs it in the script's configured context (LocalSystem or the interactive logged-on user), replacing the local scriptpaths allowlist; user-context runs wait for exit and report the real exit code instead of fire-and-forget. update_app resolves an app-level or version-level install source, fetches the pinned bytes, verifies sha256, and runs silently. Both stage into a fixed Defender-excludable folder (C:\ProgramData\ShopDB\EndpointAgent) instead of the LocalSystem temp path. Adds the DeliveryClaim contract for the claim payload.
Adds the two delivery actions to the endpoints command allowlist, each gated on a new endpoints.deploy permission ON TOP OF endpoints.command so pushing software is a heavier grant than a routine toggle. Enqueue validates update_app appversionid ownership and installer presence before queueing. New collector-authenticated download routes let the agent fetch installer packages and script bodies by digest rather than pre-staging them or sending bodies over the command wire. Wires the scriptsApi client, the siteagent generator routes, and a temporary dev-only Delivery test bench (view/route/nav) for exercising the path while the production UI is built.
AppVersion gains sha256 and silentargs so the endpoint update_app push can pin exact installer bytes and carry per-version silent switches; Application gains installersha256 for the app-level install source. New ShopDB-hosted script library (Script model plus scripts API) backs run_script delivery, with the body fetched by digest on push rather than pre-staged or sent over the command wire. Two idempotent migrations with real downgrades. Script added to the shopdb.api contract surface for plugin use.
The command queue is at-most-one-in-flight, but nothing ever expired a stuck command. When an agent claimed a command (RUNNING) then died before reporting a result - killed, uninstalled, or offline mid-flight - the row stayed in-flight forever and every operator action for that endpoint returned 409 'already in flight'. Reap in-flight rows past a staleness ceiling (15m for RUNNING, 60m for PENDING) to a new STATUS_EXPIRED at enqueue time, inside the same transaction, so the queue self-heals lazily the next time someone queues work. No background job and no schema migration - status is free-text.
A misconfigured BaseUrl (plaintext http:// against the TLS port) made the command long-poll fail at connect in milliseconds. The worker misread that fast failure as a normal idle hold-window close and re-polled with no delay, spinning into a request storm. Gate the idle-close path on the poll having stayed open at least MinIdlePollDuration, so a sub-second transport failure takes the transient backoff instead; add a MinPollInterval floor so no path can exceed a safe request rate. Also reject a non-loopback http:// BaseUrl at startup with a single clear error, turning a silent runtime flood into a fast fail. Loopback http stays valid for local dev.
Several days of endpoint-agent work left DB dumps, sample datasets, captured command output, generated maps, vuln scan output, merge-kit exports, and staged agent publish trees in the tree. None are product source. Ignore them so status shows only real work and they cannot be committed by accident. exports/ stays on disk for reference. agent/.gitignore also reserves dev-bin/ and _stage/ for delivery-build scratch.
Tracking docs/openapi.json (so the app can serve the spec at runtime) makes docs/ exist on the published GitHub mirror, which defeated the skipif(not DOCS.is_dir()) guard in the docs publish/version tests: they then ran against a prose-less checkout and failed on CI. Key the skip off docs/*.md prose instead, matching the canonical published-checkout definition in scripts/check-gates-ran.sh.
Also: the OpenAPI generator emitted a trailing-slash path (/api/docs/), which a strict proxy like IIS treats as a redirecting resource a subpath-mounted client cannot follow; normalise trailing slashes in _norm_path and correct the committed spec. And allow docx (python-docx) as an undeclared import: it is used only by scripts/docs/generate_docx.py, a developer tool that never runs on a deployed site.
The endpoint agent already reports an installed-software list (SBOM), but ingress dropped it: the collector schema had no entry for it and there was nowhere to store it, so operators could not see what is installed on a PC.
Persist it as a per-asset snapshot in a new endpointapplications table, replaced on every report (current inventory is what operators ask for, not install history). Add applications to the collector schema and KNOWN_PAYLOAD_KEYS so ingress stops discarding it, a read route for the raw collection, and a collapsed-by-default Installed software panel to surface it. appid is nullable groundwork for a future Managed Applications link; no matcher is wired yet.
Standalone one-shot enrollment tool that creates a ShopDB PC + Endpoint entry using only the endpoints collector key, reusing the existing collector adopt path. Collects identity, WMI hardware, and installed-software SBOM; requires admin. Server-side packaging route attaches to the existing endpoints blueprint; no core contract change.
The agent now reports installed applications from both Uninstall registry hives (64-bit and WOW6432Node) as {displayname, displayversion, publisher}, mirroring the Assess-Orchestrator applications collection. Registry-only, no Win32_Product/WMIC self-repair and no AppX. The server logs it as an unpersisted key until the SBOM store is built.
The endpoint agent now sends disk, bios, memory, processor, osinfo, and domain telemetry. The server does not persist these yet, so ingest logs any unpersisted payload keys with a compact preview. This makes incoming telemetry observable in the logs before any UI or schema work, so we can confirm the agent is sending it and enhance the UI later.
wj held nothing main lacked - git cherry marks every one of its commits as
already present - but promotion cherry-picks, so the histories diverged and a
fast-forward is not possible. Merging rather than resetting keeps the branch
recoverable and needs no force-push.
Levelling first so wilm's patches apply against current product code.
# Conflicts:
# CONTRIBUTING.md
# frontend/src/views/Dashboard.vue
# tools/wj-status.sh
Asked twice what "All assets" covered, which is the signal a label is
underspecified rather than a reader being slow.
The endpoint already counted every status to build its rollup and then returned
two of them, so a reader could see In Use and had no way to learn where the rest
of the total went. It now returns them all, and the panel lists them under the
total - indented and quieter, because they break the total down rather than
adding to it.
Three things this makes visible that were not guessable:
Retired assets are IN the total. Retired is a status on an active record, not
the same as deactivated, and seven of them sit inside this site's 610.
There is an "Inventory" status. I would have hardcoded In Use, In Repair and
Retired and silently dropped it. The rows are built from whatever the API
returns, sorted by count, so a status a site adds appears rather than
disappearing - which matters for sites adopting with their own seed data.
Rows held back by isactive are in NO status bucket, so they cannot be found by
adding the buckets up. That gap is exactly what makes a dashboard total
disagree with a list page, so the count is reported and captioned.
The caption states the counting policy - active records only, dual-bay pairs
counted once - and the dual-path clause renders only when that site toggle is
on, since asserting it unconditionally would be false at a site that has it off.
Full suite 2182, frontend 314.
The panel had no group label before it was one panel, so grouping it needed a
heading and it got 'Estate' - RMM idiom, and the word the leadership documents
already used for the whole of what a site owns.
Wrong register for this surface. The sidebar already groups exactly these things
under a section named assets, so the dashboard had invented a second word for a
group the app already names, and the panel's own total row says 'All assets' -
the heading and the total disagreed inside one panel. 'Estate' is also
enterprise-IT and largely British idiom; it earns its place in an executive
summary and not on a screen read at 6am.
The scoped classes follow the label, so the markup does not keep a word the UI
stopped using. The documents keep 'estate', where it is doing real work for a
leadership reader.
Scope: product
The bundle became one file per branch, but the frontend dists did not: they are
still frontend-dist-subpath-{ops,shopdb}, fixed paths, and update-prod-server.cmd
robocopies the shopdb one onto the production server with /MIR - a full mirror -
with no idea which branch built it.
So --dist from a site branch would have replaced production's frontend with
unreleased work at the next deploy, silently, and nothing downstream could have
told. Worse than the bundle collision it was sitting beside, because the dist is
what actually lands on the server.
Branch-qualifying the directories would only move the trap to whoever pointed a
deploy at the wrong one. Production runs the product, so the export now refuses
and says which commits to promote instead.
Scope: product
Promotion cherry-picks, which gives the commit a new sha on main. wj-status
compared by sha via main..wj, so after promoting ten commits it still listed all
ten as unpromoted - and would have invited promoting the same work twice.
git cherry compares patch content instead and marks an already-present commit
with a leading '-'. When nothing is genuinely outstanding it now says so, counts
what has already landed, and shows how far the branch is BEHIND main with the
rebase to run. Behind was the state that mattered here and the tool could not
express it at all.
Scope: product
Recorded that wilm no longer existed, on the evidence of the local gitea clone
where it was merged and its ref deleted. That was wrong. It lives on the
corporate GitHub and the other site is developing the .NET endpoint agent on it.
The dev box cannot see that branch - no account there - so its absence locally
was never evidence of anything, and fetch-wilm-branch.cmd on the share exists
precisely to bring it back for review. Says so now, so the next reader does not
draw the same conclusion from the same missing ref.
Scope: product
The work-PC instructions began with 'git checkout <branch>', which fails the
first time a branch goes over because it does not exist in that clone yet -
exactly the moment somebody follows them most carefully. Nothing to check out,
and nothing to merge into either.
Now prints both: fetch then checkout -b then push -u the first time, and the
checkout/fetch/merge/push cycle afterwards.
Scope: product
(cherry picked from commit 1fe788a3ed)
promote-to-main.sh checks out main. The tooling lives only on the site branch,
so that checkout removes tools/promote-to-main.sh while the script is still
running - and bash reads a script lazily, by file offset, so it would carry on
executing whatever bytes happen to land there. While manipulating branches.
Not theoretical: main does not have tools/ right now, so the first real use
would have hit it.
Both versions now copy themselves to a temp file and re-exec from there. The
bash copy unlinks itself immediately, since the open handle keeps working. The
Windows copy has the same hazard for the same reason - cmd.exe re-seeks the
batch file after every command - and two extra traps worth naming: the copy runs
from %TEMP%, so the repository root has to travel with it rather than being
derived from %~dp0, and %errorlevel% inside a parenthesised block is expanded
when the block is PARSED, before the call it is meant to report on.
Scope: product
(cherry picked from commit 6d42110a91)
A guest got a heading and nothing else.
THE STRUCTURAL FAULT WAS MINE
Moving the Estate panel into DashboardCards' rail slot put it behind that
component's root v-if, which was `liveCards.length > 0`. A guest is permitted
zero cards, so the whole component disappeared and took the slot with it. Before
that move the tiles were top-level and always rendered. The wrapper no longer
vanishes with the cards.
TWO LIES THE EMPTY STATE WAS TELLING
"Nothing needs attention" is now shown only to somebody who can see the checks.
A reader permitted none of them is not looking at a healthy fleet, they are
looking at nothing, and saying otherwise is the same fault as reporting Low
Supplies 0 while the monitoring is down. The count of PERMITTED cards is now
tracked separately from the count that currently hold rows.
The Estate panel no longer renders zeros for a caller who cannot read totals.
/api/dashboard/summary needs assets.view, and the failure was swallowed into an
empty object that drew a tidy column of zeros - an estate of nothing, stated
with confidence.
WHAT A GUEST GETS INSTEAD
Where they can go: the same permission-filtered navigation the sidebar uses,
which the server already filters to the guest role for a signed-out visitor. So
it cannot offer a link that dead-ends at the router guard, and this page names no
plugin it had to know about. Verified against the running instance - a guest is
offered Guides and Tech Tools, matching guides.view and tools.view.
ALSO: the measuring tools row
totalassets counts five types - machines, PCs, network, printers AND measuring
tools - but only four had rows, so the total stopped matching the addends above
it wherever gages exist: 610 of visible rows under a stated 729. That is exactly
the fault this panel was built to remove, and it looked right here only because
this database has no measuring tools. Every counted type now has a row.
Frontend 314 passed.
(cherry picked from commit 0f2da9beaf)
The export wrote a single shopdb-flask-pub.bundle whatever it published. That
was fine while only main was ever published, and became a trap the moment it was
not: exporting a site branch silently overwrote the bundle holding main's work,
and the work PC's documented "git fetch <bundle> main" then failed because that
ref was no longer in the file.
It had already happened here - two wj exports overwrote a main bundle from the
same morning. The failure is at least loud on the work PC rather than pushing
the wrong branch, but the main bundle has to be regenerated.
Now shopdb-flask-pub-<branch>.bundle. Two branches cannot share one filename.
Scope: product
(cherry picked from commit 1311dea113)
The branch checkout landed after the rsync, where git checkout --force reverts
the tree to the branch's committed state and discards everything rsync had just
written. The export then found no changes and published nothing, while still
printing a bundle path and looking like it had worked. A silent no-op is the
worst shape for this: the next person believes the export ran.
Scope: product
(cherry picked from commit 06c584240e)
export-github.sh syncs the WORKING TREE, not a branch, and then committed
whatever it found onto the publication repo's main and printed
"git push origin main".
So running it while checked out on a site branch published that site's work AS
THE PRODUCT, to the repository every other site consumes, and said nothing. The
branch model made that reachable in one wrong command, from a checkout that
looks completely normal.
It now reads the current branch, mirrors it in the publication repo (creating it
from main the first time, as in the dev repo), bundles that branch, and prints
work-PC instructions naming it. Publishing anything other than main also prints
a reminder that other sites read main and these commits are not there yet.
The publication checkout uses --force, which is safe only because that tree is a
mirror: the rsync has already replaced it, so there is nothing local to lose. It
would not be safe anywhere else.
Also cleans up the publication repo: origin pointed at a bundle in a scratchpad
from a session that ended long ago, and two orphaned remote-tracking refs
(github/main, wilm/review-fixes) survived the removal of their remotes. None of
it broke the export, which never pushes, but a `git fetch` in there failed with
a confusing path.
Scope: product
(cherry picked from commit 81df9607e7)
The convention existed only in the scripts' comments and in one conversation,
which is not where somebody joining looks. CONTRIBUTING is.
Records the two branches, the Scope trailer, the promote flow, and the point
that most work is product: ADR-015 puts site differences in settings, seed data
or scripts/site_imports/<site>/, so a site branch is about timing rather than
locality.
Also names the failure worth watching. Promoting something site-specific by
accident is caught by the ADR-015 gate on the way into main. The real risk is
product work stranded on a site branch because nobody decided, which is what
turned wilm from a merge into a port.
Scope: product
(cherry picked from commit 1d4ac11572)
The scripts could move a commit to main but nothing said which commits should
go, and that decision does not survive being deferred: months later it is a list
of subject lines and a guess.
So it is recorded at commit time, as a trailer:
Scope: product the rest of the world should have it
Scope: site this site alone, keep it here
tools/wj-status.sh reads them and prints main..wj in three groups, with the
promote command for the ones marked product.
The path fallback is deliberately weak. Under ADR-015 the only site-specific
home in the tree is scripts/site_imports/<site>/ - everything else is meant to
be product, with settings and seed data carrying what differs between sites. So
"touches only site_imports" is the one confident site verdict paths can give,
and everything else is reported UNMARKED rather than guessed into a bucket. A
tool that quietly sorted commits by directory would be wrong often enough to
stop being read.
The unmarked message says which way to lean, because the failure mode here is
not promoting something site-specific by accident - the ADR-015 gate catches
that on the way in. It is product work sitting on wj forever because nobody
decided, which is how the wilm branch became a port rather than a merge.
Scope: product
(cherry picked from commit 0400a58786)
Same two operations as the bash pair, for driving git from Windows rather than
from the development box.
CREDENTIALS ARE HANDLED DIFFERENTLY, ON PURPOSE
The bash version reads a token out of secrets.md and feeds it to git through
GIT_ASKPASS, because that box has no credential store. Windows has one: Git for
Windows installs Git Credential Manager and enables it by default, so `git push`
prompts once and Windows Credential Manager keeps it.
So the token handling is deliberately NOT ported. Copying a token into a script,
an environment variable or a remote URL on a machine that already has a
credential store is strictly worse than the thing it would replace, and the
comment in the file says so, because "make it match the other script" is exactly
the tidying somebody would otherwise do later.
THE PROMOTE SCRIPT REFUSES TO RUN WITHOUT BASH
scripts/check-naming-and-style.sh is a shell script, and it carries the ADR-015
site-literal check - the one gate that makes a site branch survivable rather
than a slow leak into the product. Skipping it on a machine without bash would
defeat the entire point of the script, so it stops and says how to fix it
instead of promoting without it.
CRLF
.gitattributes already pinned *.sh to LF so Git Bash can run them; *.cmd is now
pinned to CRLF for the same class of reason in the other direction. cmd.exe
parses a lone LF inconsistently around labels and goto, so an LF-only batch file
can run its happy path and then jump nowhere on the error path - which in a
promote script is the path that matters.
(cherry picked from commit 49a898fc5a)
wj holds this site's work; main is the product every other site sees and the
publication mirror exports. Two scripts, one for each direction.
tools/git-push.sh pushes a branch without leaving the token behind. The way this
was being done by hand - rewrite remote.origin.url to embed the token, push,
rewrite it back - writes the token into .git/config in plaintext on every push,
and leaves it there if the push fails or the terminal closes mid-way, where
`git remote -v` will happily print it. Putting it in the URL on the command line
is no better: argv is visible in `ps` to every user on the box. The token now
goes in through GIT_ASKPASS, so it never reaches the config or the process list
and there is nothing to clean up when the script dies.
tools/promote-to-main.sh cherry-picks NAMED commits onto main, rather than
merging the branch. Merging takes the site-specific work along with the product
work, which is how a site branch quietly becomes the product; naming the commits
keeps that decision explicit and per change.
It runs main's gates after the pick and before any push, and does not push at
all - it prints what landed and leaves that to a human. The ADR-015 site-literal
check is the one that earns its place: it catches a hostname or share path
riding along in a commit that looked generic, which is the failure that makes a
site branch dangerous rather than merely awkward. It caught this very commit,
which named the site in a comment.
Both refuse a dirty tree. Pushing one pushes the last commit rather than what is
in front of you, and cherry-picking from one mixes uncommitted work into the
promotion. On failure the promote script aborts the pick and returns you to the
branch you started on, with main untouched.
(cherry picked from commit 6d9954c6f9)
The work-PC instructions began with 'git checkout <branch>', which fails the
first time a branch goes over because it does not exist in that clone yet -
exactly the moment somebody follows them most carefully. Nothing to check out,
and nothing to merge into either.
Now prints both: fetch then checkout -b then push -u the first time, and the
checkout/fetch/merge/push cycle afterwards.
Scope: product
promote-to-main.sh checks out main. The tooling lives only on the site branch,
so that checkout removes tools/promote-to-main.sh while the script is still
running - and bash reads a script lazily, by file offset, so it would carry on
executing whatever bytes happen to land there. While manipulating branches.
Not theoretical: main does not have tools/ right now, so the first real use
would have hit it.
Both versions now copy themselves to a temp file and re-exec from there. The
bash copy unlinks itself immediately, since the open handle keeps working. The
Windows copy has the same hazard for the same reason - cmd.exe re-seeks the
batch file after every command - and two extra traps worth naming: the copy runs
from %TEMP%, so the repository root has to travel with it rather than being
derived from %~dp0, and %errorlevel% inside a parenthesised block is expanded
when the block is PARSED, before the call it is meant to report on.
Scope: product
A guest got a heading and nothing else.
THE STRUCTURAL FAULT WAS MINE
Moving the Estate panel into DashboardCards' rail slot put it behind that
component's root v-if, which was `liveCards.length > 0`. A guest is permitted
zero cards, so the whole component disappeared and took the slot with it. Before
that move the tiles were top-level and always rendered. The wrapper no longer
vanishes with the cards.
TWO LIES THE EMPTY STATE WAS TELLING
"Nothing needs attention" is now shown only to somebody who can see the checks.
A reader permitted none of them is not looking at a healthy fleet, they are
looking at nothing, and saying otherwise is the same fault as reporting Low
Supplies 0 while the monitoring is down. The count of PERMITTED cards is now
tracked separately from the count that currently hold rows.
The Estate panel no longer renders zeros for a caller who cannot read totals.
/api/dashboard/summary needs assets.view, and the failure was swallowed into an
empty object that drew a tidy column of zeros - an estate of nothing, stated
with confidence.
WHAT A GUEST GETS INSTEAD
Where they can go: the same permission-filtered navigation the sidebar uses,
which the server already filters to the guest role for a signed-out visitor. So
it cannot offer a link that dead-ends at the router guard, and this page names no
plugin it had to know about. Verified against the running instance - a guest is
offered Guides and Tech Tools, matching guides.view and tools.view.
ALSO: the measuring tools row
totalassets counts five types - machines, PCs, network, printers AND measuring
tools - but only four had rows, so the total stopped matching the addends above
it wherever gages exist: 610 of visible rows under a stated 729. That is exactly
the fault this panel was built to remove, and it looked right here only because
this database has no measuring tools. Every counted type now has a row.
Frontend 314 passed.
The export wrote a single shopdb-flask-pub.bundle whatever it published. That
was fine while only main was ever published, and became a trap the moment it was
not: exporting a site branch silently overwrote the bundle holding main's work,
and the work PC's documented "git fetch <bundle> main" then failed because that
ref was no longer in the file.
It had already happened here - two wj exports overwrote a main bundle from the
same morning. The failure is at least loud on the work PC rather than pushing
the wrong branch, but the main bundle has to be regenerated.
Now shopdb-flask-pub-<branch>.bundle. Two branches cannot share one filename.
Scope: product
The branch checkout landed after the rsync, where git checkout --force reverts
the tree to the branch's committed state and discards everything rsync had just
written. The export then found no changes and published nothing, while still
printing a bundle path and looking like it had worked. A silent no-op is the
worst shape for this: the next person believes the export ran.
Scope: product
export-github.sh syncs the WORKING TREE, not a branch, and then committed
whatever it found onto the publication repo's main and printed
"git push origin main".
So running it while checked out on a site branch published that site's work AS
THE PRODUCT, to the repository every other site consumes, and said nothing. The
branch model made that reachable in one wrong command, from a checkout that
looks completely normal.
It now reads the current branch, mirrors it in the publication repo (creating it
from main the first time, as in the dev repo), bundles that branch, and prints
work-PC instructions naming it. Publishing anything other than main also prints
a reminder that other sites read main and these commits are not there yet.
The publication checkout uses --force, which is safe only because that tree is a
mirror: the rsync has already replaced it, so there is nothing local to lose. It
would not be safe anywhere else.
Also cleans up the publication repo: origin pointed at a bundle in a scratchpad
from a session that ended long ago, and two orphaned remote-tracking refs
(github/main, wilm/review-fixes) survived the removal of their remotes. None of
it broke the export, which never pushes, but a `git fetch` in there failed with
a confusing path.
Scope: product
The convention existed only in the scripts' comments and in one conversation,
which is not where somebody joining looks. CONTRIBUTING is.
Records the two branches, the Scope trailer, the promote flow, and the point
that most work is product: ADR-015 puts site differences in settings, seed data
or scripts/site_imports/<site>/, so a site branch is about timing rather than
locality.
Also names the failure worth watching. Promoting something site-specific by
accident is caught by the ADR-015 gate on the way into main. The real risk is
product work stranded on a site branch because nobody decided, which is what
turned wilm from a merge into a port.
Scope: product
The scripts could move a commit to main but nothing said which commits should
go, and that decision does not survive being deferred: months later it is a list
of subject lines and a guess.
So it is recorded at commit time, as a trailer:
Scope: product the rest of the world should have it
Scope: site this site alone, keep it here
tools/wj-status.sh reads them and prints main..wj in three groups, with the
promote command for the ones marked product.
The path fallback is deliberately weak. Under ADR-015 the only site-specific
home in the tree is scripts/site_imports/<site>/ - everything else is meant to
be product, with settings and seed data carrying what differs between sites. So
"touches only site_imports" is the one confident site verdict paths can give,
and everything else is reported UNMARKED rather than guessed into a bucket. A
tool that quietly sorted commits by directory would be wrong often enough to
stop being read.
The unmarked message says which way to lean, because the failure mode here is
not promoting something site-specific by accident - the ADR-015 gate catches
that on the way in. It is product work sitting on wj forever because nobody
decided, which is how the wilm branch became a port rather than a merge.
Scope: product
Same two operations as the bash pair, for driving git from Windows rather than
from the development box.
CREDENTIALS ARE HANDLED DIFFERENTLY, ON PURPOSE
The bash version reads a token out of secrets.md and feeds it to git through
GIT_ASKPASS, because that box has no credential store. Windows has one: Git for
Windows installs Git Credential Manager and enables it by default, so `git push`
prompts once and Windows Credential Manager keeps it.
So the token handling is deliberately NOT ported. Copying a token into a script,
an environment variable or a remote URL on a machine that already has a
credential store is strictly worse than the thing it would replace, and the
comment in the file says so, because "make it match the other script" is exactly
the tidying somebody would otherwise do later.
THE PROMOTE SCRIPT REFUSES TO RUN WITHOUT BASH
scripts/check-naming-and-style.sh is a shell script, and it carries the ADR-015
site-literal check - the one gate that makes a site branch survivable rather
than a slow leak into the product. Skipping it on a machine without bash would
defeat the entire point of the script, so it stops and says how to fix it
instead of promoting without it.
CRLF
.gitattributes already pinned *.sh to LF so Git Bash can run them; *.cmd is now
pinned to CRLF for the same class of reason in the other direction. cmd.exe
parses a lone LF inconsistently around labels and goto, so an LF-only batch file
can run its happy path and then jump nowhere on the error path - which in a
promote script is the path that matters.
wj holds this site's work; main is the product every other site sees and the
publication mirror exports. Two scripts, one for each direction.
tools/git-push.sh pushes a branch without leaving the token behind. The way this
was being done by hand - rewrite remote.origin.url to embed the token, push,
rewrite it back - writes the token into .git/config in plaintext on every push,
and leaves it there if the push fails or the terminal closes mid-way, where
`git remote -v` will happily print it. Putting it in the URL on the command line
is no better: argv is visible in `ps` to every user on the box. The token now
goes in through GIT_ASKPASS, so it never reaches the config or the process list
and there is nothing to clean up when the script dies.
tools/promote-to-main.sh cherry-picks NAMED commits onto main, rather than
merging the branch. Merging takes the site-specific work along with the product
work, which is how a site branch quietly becomes the product; naming the commits
keeps that decision explicit and per change.
It runs main's gates after the pick and before any push, and does not push at
all - it prints what landed and leaves that to a human. The ADR-015 site-literal
check is the one that earns its place: it catches a hostname or share path
riding along in a commit that looked generic, which is the failure that makes a
site branch dangerous rather than merely awkward. It caught this very commit,
which named the site in a comment.
Both refuse a dirty tree. Pushing one pushes the last commit rather than what is
in front of you, and cherry-picking from one mixes uncommitted work into the
promotion. On failure the promote script aborts the pick and returns you to the
branch you started on, with main untouched.
Splitting "cannot see the cartridges" from "nothing is low" was right, but it
treated two unlike cases as one. isconfigured means enabled AND a URL AND a
token, so its absence means the site did not ask for supply monitoring at all -
and a permanent "Supply monitoring is not configured" line is a card reporting
nothing wrong every day, which is how a board teaches people to stop reading it.
It matters for sites adopting ShopDB without Zabbix, who would otherwise inherit
a warning about a feature they declined.
Not configured now returns a plain empty and the card hides. Only configured and
unanswerable reports itself, because that is something meant to work that is not
working.
Also withdraws an unfounded inference. A dev box has no route to Zabbix by
design, so "not reachable" there is the expected state and says nothing about
what production can see. The value of this change is not that it revealed a
production fault; it is that when production's Zabbix does go down, the board
will say so instead of displaying "Low Supplies 0".
Four tests pin the three cases apart: unreachable reports, unconfigured stays
quiet, and a real zero from a reachable Zabbix is not flagged as blindness. The
fourth holds the envelope's rows, since carrying a reason changed the response
shape. Verified they are not vacuous by restoring the original branch in place -
the unreachable test fails against it.
Full suite 2182 passed.
TWO CONTROLS, ONE BOARD
Severity answers "how bad". It could not answer "what kind", and on a fleet
where eight endpoints go quiet at once, being able to set those aside and look
at the four crashes is the difference between a board and a list.
Chips per contributing check, coloured by its severity, composing with the
severity rollup above them. Both narrow what is already loaded rather than
refetching, so no combination can disagree with the counts. Picking a severity
that excludes the active chip clears the chip: two controls lit above an empty
queue, neither obviously wrong, is worse than losing the narrower one.
A chip counts every row, not the rows the queue had room for. A chip reading 5
above a "3 more" line would be its own small lie.
"LOW SUPPLIES 0" WAS NOT TRUE
Asked what happened to toner replacement, and the answer was worse than the
layout question that prompted it.
_get_low_supplies_data returned the same empty result for three different
situations: Zabbix not configured, Zabbix unreachable, and every cartridge
full. The dashboard strip therefore displayed "Low Supplies 0 / Critical 0"
while the monitoring was dead - and this instance is exactly that case, with
zabbix_enabled true against a URL nothing answers on.
So the strip removed in the previous commit was not information. It was
reassurance, and it read identically whether the printers were fine or nobody
could see them. That is the failure this codebase already names elsewhere: an
empty card and a broken card must not look the same.
The endpoint now distinguishes the three, and returns an envelope rather than a
bare list so a card can say WHY it has no rows. cardVisible shows such a card
whatever its empty rule, because "I cannot tell" is not "nothing to report".
It renders as a notice line, not a queue row: there is no asset to act on, and
burying "the monitor is dead" among the things it failed to find would be the
same mistake in a new place.
Note what this does NOT fix: the Zabbix URL on this instance is a dev
placeholder. The board will now say so instead of implying the cartridges are
full, but somebody still has to point it at a real server - and it is worth
checking what production points at, since nothing would have shown there either.
Frontend 314 passed (13 new), full suite 2178. The response shape changed from
a list to an envelope; no test depended on the old shape.
The board was one card per source in a fixed grid, and both halves of that hurt.
Every source got the same small box whatever it had to say. Warranties showed 5
of 50 behind "and 45 more"; the crash card truncated at 4; a one-row card sat
beside a four-row card and left a hole. Nothing had room, which is the complaint
that started this: not enough space to relay the data.
And a box per source flattened priority. An enforcement failure and a warranty
notice looked equally urgent because each owned a card, while the largest type
on the page belonged to counts that are true every day and need nobody.
WHAT THE PAGE IS NOW
A severity rollup, one worst-first queue of everything that needs a person, and
a row of context beneath it.
The queue merges every critical and warning row and sorts them, so reading order
is work order. It is full width, which is where the room came from. Rows are
capped per source, because fifty expiring warranties must not bury two crashed
PCs, and what did not fit is named per source with a link to it - a bare "and 45
more" that leads nowhere is worse than saying nothing.
The rollup counts ROWS, not cards, and filters the board in place rather than
navigating away, so the counts and the rows cannot disagree.
Severity, position and permission were already declared per card and already
sorted on. The old layout threw that away; this only stops it.
WHAT WENT, AND WHY IT SHOULD NOT COME BACK
Four asset tiles plus an "All assets" tile: machines + PCs + printers + network
IS all assets, so the page showed a total beside its own addends at the same
weight. It is now a line under a rule, where a total belongs.
The printer strip. It carried the printer count a third time - the tile row and
"Total Printers" were already the same number - and "Online" equalled total, so
three tiles held one fact. A genuinely low cartridge already reaches the queue
through the printers-supplies card, which is the moment anyone wants it.
Three cards whose whole job was to display 0.
Recent Devices: the largest block on the page, listing five machines that were
all "In Use" with business unit "TBD". Recency is not a reason to look at
something.
CONTRACT 0.26.0: 'shortname'
Cards may declare a short label for their rows in the merged queue. Their
'title' is written as a heading - "Endpoints with crashes or shutdowns" - and as
a row badge it ran wider than the hostname it qualified, putting the badge
rather than the asset at the start of every line. CSS-truncating the title was
tried first and was worse: four consecutive rows reading "ENDPOINTS WITH CR..."
say nothing at all. Optional and defaults to the title, so a plugin that does
not declare one is unaffected. Additive field, minor bump.
ALSO: "silent 1 days"
A meta suffix is written for the plural case and the card cannot know the value.
One row saying it was a shrug; the merged queue puts every quiet endpoint on one
screen, so it said it five times in a column and read as broken. Now singularised
at exactly 1, on the first word, keeping what follows: "1 event (48h)", "silent 1
day". The test asserting ' ms' survives is what caught the first version turning
it into ' m'.
DashboardRow.vue is extracted because the queue and the context panels render
the same row, and the map tooltip, link fallbacks and chips are too easy to get
subtly different in a second copy.
Verified on screen against live data. Full suite 2178 passed, frontend 301.
The triage band now answers the common case above the fold - what an asset
serves, or what drives it - so the full edge list below is reference rather than
the first thing to read. On a machine showing two relationships, both already
appeared in the band and the banner, and the card was a third copy.
Collapsed by default, using the same caret, roles and keyboard handling as the
collapsible panels on the same page rather than a second, slightly different
one.
The card is NOT redundant with the band and is not removed. It is the only
surface with Add Relationship and the per-row delete, and it keeps each edge's
exact type where the band deduplicates by asset. The printer page is the proof:
four rows there - usesprinter and defaultprinter, for each of two machines -
collapse to two band chips, and which machine DEFAULTS to that printer is a
distinction the band deliberately drops. Fine for triage, wrong for management.
THE COUNT IS OF WHAT IS RENDERED, NOT OF EDGES
A collapsed card that just says "Relationships" cannot be told from an empty
one, so the header carries a count. The first version counted raw edges and was
wrong on the first machine it was shown: a symmetric Dualpath link is stored in
BOTH directions in the legacy shape, so it is two rows in outgoing+incoming and
one "Connected" entry on screen. The badge read 3 above a card that shows 2 when
expanded - exactly the mismatch the badge exists to prevent. It now counts
rendered items.
Add Relationship expands the card before opening the modal, so a new row cannot
land behind a closed caret and read as the add having failed.
Verified on screen: machine reads 2 and matches, printer reads 4 and matches.
Frontend 280 passed.
TWO STRIPS OF CHROME SAYING THE SAME KIND OF THING
The machine page opened with a banner - "Dual-bay machine - sibling bay: 2022" -
directly above a hero whose band said "Controlled by". Both answer "what am I
looking at", and splitting them across a banner and a band cost a strip of
vertical space to say so.
The sibling is now a chip in the band beside the controlling PC. The band takes
an `extraitems` prop rather than learning what dualpath is: the pairing is
resolved by the machines plugin, not from the relationships endpoint, so core
cannot derive it, and a dualpath-shaped prop on a core component would be the
machines plugin leaking upward. A page passes [{label, links:[{text, to}]}] and
the band renders it with its own markup, which also keeps the chips visually
identical to the ones it derives itself.
The label stays "Dual-bay sibling" rather than "Dualpath". The banner spelled
the meaning out because a dualpath pair is ONE physical machine in two bays; a
chip reading "Dualpath" would have dropped that.
THE GAGE PAGE FILTERED FOR A COUNTERPART THAT CANNOT EXIST
MeasuringToolDetail was wired counterparttype="machine" labelled "Used on".
Wrong on both counts. The import synthesizes one measuring-tool asset per
metrology PC and a 'controls' edge PC -> tool (stage_metrology), so a gage's
counterpart is a COMPUTER, exactly as on the machine page. Filtering for a
machine peer meant the band rendered nothing on every gage that has a PC.
It looked merely empty rather than wrong because this dev database holds one
measuring tool with no relationships at all, so there was nothing to be missing.
A hand-held gage genuinely has no such edge and correctly shows nothing; a
metrology station now shows the PC that drives it.
The relationships card stays. It is the only surface with Add Relationship and
the per-row delete, and it lists types the band filters out - the band is
triage, the card is management. It does now repeat what the band shows, which
is worth revisiting separately.
Verified on screen. Frontend 280 passed.
Three defects broke the loader's own documented resume guarantee. All three
were silent: nothing raised, the run finished cheerfully, and fewer rows
arrived than anyone counted.
THE PAGE SIZE WAS CLAMPED, SO THE 409-RESOLVE SEARCHED THE FIRST 100 ROWS
_upsert asked for ?per_page=10000 and read one response. The server clamps any
page size to MAX_PAGE_SIZE, which is 100 - not an error, not a warning, just
100 rows. Anything past that resolved to None, so the id never reached the
crosswalk and the run quietly dropped a model's link on every machine that used
it, or an application's installs.
The target already holds more than 100 models and more than 100 applications,
so this fired on any re-run or resume - exactly the case the loader documents
as safe.
_resolve_existing now tries the exact natural-key filter the API has exposed
all along (?modelnumber= on models, ?appname= on applications, ?vendor= on
vendors) and then pages properly, bounded. An endpoint without such a filter
ignores the unknown parameter and the paged walk finds the row.
stage_metrology repeated the same mistake in its own way: it skipped 409s
entirely, so a resumed run left the tool out of the crosswalk and the Controls
edge unmade - metrology PCs driving nothing. stage_printers already had the
resolve branch; that stage was written later and missed it.
THE CROSSWALK WAS NOT BOUND TO THE DATABASE IT DESCRIBED
idmap.json maps this site's legacy ids to assetids in ONE database, and those
assetids mean nothing anywhere else. Keeping it across a drop-and-rebuild left
warranties and printer assignments attaching to whatever asset now holds that
id: wrong data, no error. The runbook's two "drop and rebuild" steps never
mentioned the file.
The loader now writes a uuid marker into the TARGET's own settings table, so it
dies with the data, and stamps it into the crosswalk. A crosswalk built against
a different database is refused before a row is written. The database name
alone would not do - a rebuild keeps the name. A file written before this guard
is adopted rather than refused, so no existing crosswalk is stranded.
Also adds --idmap, so one checkout can drive more than one target.
THE SOURCE DATABASES ARE NAMED, NOT HARDCODED
'shopdb_src' appeared at 29 call sites, so aiming the loader at the prodscratch
restore meant editing run.py. Those names are now LOGICAL and resolved in
Source.db() from SOURCE_DB_MAIN / SOURCE_DB_USB / SOURCE_DB_EMPLOYEES, with
host, port, user and password overridable too. The call sites are untouched -
resolving centrally avoids missing one, and the literals now read as
documentation of which source a stage draws from. The run prints every source
it redirected, so "it imported nothing" can be told apart from "it read the
wrong database".
This is what stood between the loader and the prodscratch run.
TWO SMALLER ONES
_source_password() raised IndexError rather than its intended message whenever
the DATABASE_URL user is not root, which it will not be on a real server.
main() returned 1 on failure into a caller that discarded it. A run that logged
200 endpoint errors exited 0 and read as a success in a deploy log; it now
exits non-zero.
TESTS AND DOCS
The loader had no tests at all. Sixteen now cover the resolve, the crosswalk
binding and the source aliasing - the logic that can run without a MySQL
source, which is the part that was wrong. Verified they are not vacuous by
running the old algorithm against the same fixture: it returns None where the
new one returns the row.
The README claimed 15 stages "complete" with counts from 2026-07-13 that two
later routing changes invalidated, and the runbook told operators to verify a
fresh import against them. Both corrected, with the three gaps NOT fixed here
recorded plainly: warranties, notifications and knowledgebase POST without a
duplicate check so resuming across them doubles rows; there is no --dry-run or
target guard; and the verify stage counts target assets against source machines,
which is apples to oranges.
Full suite 2178 passed.
THE RECIPROCAL LINKS BUG
A PC's page showed "Serves machine 7803" at the top. The machine's page showed
nothing equivalent, though the same edge was right there in the relationships
card at the bottom. That asymmetry read as missing data and was reported as
such.
It was neither missing data nor an API fault. The endpoint returns both
directions and always did; AssetTriageBand already walked outgoing AND incoming
and deduped, with a comment explaining that production holds reciprocal rows.
It then discarded every peer whose asset type was not 'machine', and hardcoded
the label "Serves machine". So the component could only ever work from the PC
side, and it was mounted only on the PC page.
The counterpart type and label are now props, defaulting to the old values so
the PC page is unchanged. The machine page mounts the band with
counterparttype="computer" and the label "Controlled by".
Eliminated with evidence rather than assumed, because this bug had already been
misdiagnosed twice: the endpoint runs two queries and returns both arrays; the
generic card renders both; that route has no pagination to clamp; and a single
permission gate cannot admit one direction and not the other.
THE RELATIONSHIP TYPE WAS ALSO HARDCODED
Not every asset pair is linked by 'controls'. A machine reaches a printer
through usesprinter and defaultprinter, so the band was blank on the printer
page - the one page where the link is not obvious from the asset itself. The
types are now a prop, and the printer page passes its own.
THE CHIPS WERE DEAD LINKS
Found while adopting the band more widely. It linked to /assets/<assetid>.
There is no /assets/:id route and no catch-all, so every chip in the band was
dead - and this change would have propagated that to four more pages.
AssetRelationships.vue already resolved this correctly and privately. That
function is now frontend/src/utils/assetRoute.js and both components use it, so
a third caller cannot get it wrong again. Two things it gets right that are easy
to miss: the detail routes key on pluginid, not assetid, so passing assetid
opens the wrong record; and the lookup is now case-insensitive, because
csvimport.py seeds 'Computer' and 'Machine' capitalised while this site's
database holds them lowercase - a case-sensitive match produced dead links at
whichever site seeded the other way.
The band now carries pluginid and assettypename on each chip, which the
resolver needs.
TESTS
assetRoute had none and neither did the band. Seven cover the resolver,
including the assetid-vs-pluginid confusion and the seeding-case difference.
On the backend, nothing asserted that the TARGET of a directional edge receives
it in `incoming` - only that rows carry isdirectional. That missing assertion is
this bug's exact shape, so it is now pinned: the endpoint was right all along,
and the next such report should not cost anyone a re-diagnosis of the API.
Verified on screen: the machine page shows "Controlled by G6K76CW3ESF" in the
hero, the PC page is unchanged, and a page with no such link renders no empty
band. Full suite 2162 passed, frontend 280 passed.
docs/PLUGINS.md listed sixteen of the seventeen bundled plugins. The missing one
was endpoints - the newest, and the one another site is most likely to ask
about, since it is the Fleet console and the live panels on a PC detail page.
The row records the two things that are not guessable from the name: it depends
on computers because an endpoint is telemetry ABOUT a PC asset rather than a
second record of it, and its four permissions are split deliberately. Read is
grantable to guests by the .view suffix rule; fleet, command and restart are
not, and the plugin declares no override, so the command channel cannot be
opened to anonymous callers by ticking a box.
Ingest is called out as NOT being a route here - agent reports arrive on the
generic collector route under a service token scoped collector.ingest.endpoints.
Somebody looking for the ingest endpoint in this plugin will not find one.
applications/ stays absent on purpose: it has no manifest and no plugin.py, so
per the manifest-less rule it is core frontend surface that always ships, not a
plugin a site can choose.
requirements.txt carried a hand-written header explaining why the lockfile is
universal and hashed, what each flag buys, and a warning not to hand-edit it
because waitress and tzdata were once added by hand and vanished on the next
compile.
Running `uv pip compile` to upgrade two packages wiped all of it, including the
warning about hand-edits vanishing. The header had been documenting its own
failure mode and still lost.
The explanation now lives in requirements.in, which is the file people edit and
which a compile never rewrites. The header is restored in requirements.txt for
anyone reading that file directly, but it is no longer the only copy.
Also records the --upgrade-package form for moving one package without moving
everything else, and the scan_sbom.py check to run before a release.
scan_sbom.py reported 50 advisory matches against shipped components, 16 of them
at or above HIGH. This clears all of them: the scan now reports 0 shipped
findings and the gate passes.
pyjwt 2.12.1 -> 2.13.0
idna 3.13 -> 3.19
axios 1.13.2 -> 1.20.0 (29 advisories)
postcss 8.5.6 -> 8.5.26
dompurify 3.4.11 -> 3.4.14
nanoid 3.3.11 -> 3.3.18
follow-redirects 1.15.11 -> 1.16.0
form-data 4.0.5 -> 4.0.6
Every one is an in-major bump inside a range the manifests already allowed, so
this is a lockfile refresh rather than a migration. The Python side was compiled
with --upgrade-package for each name so nothing else moved; the diff is two
version lines and their hashes.
nanoid is worth recording: it is transitive under postcss at ^3.3.11, and its
advisories are fixed on both the 3.x and 5.x branches. 3.3.18 is the fix that is
actually reachable.
The declared floors for axios and dompurify were raised to the versions in the
lockfile. They were ^1.6.0 and ^3.4.11, which still nominally admitted the
vulnerable range - npm resolves to the highest match so an install was never
going to pick one, but the manifest should state the minimum safe version rather
than rely on that.
axios 1.20 adds optional proxy-agent support, which pulls https-proxy-agent and
agent-base into the tree. Both are Node-only and neither reaches the browser
bundle; the build was checked for them. The main chunk grew 256kB to 271kB
(85.7kB to 90.5kB gzipped) from the rest of the axios delta.
The PyJWT entry in the allowlist was removed rather than kept. It stopped
matching the moment the upgrade landed, and an allowlist that accumulates
entries for advisories that no longer apply is one nobody rereads. The reasoning
is preserved as the worked example in docs/DEPENDENCY-SECURITY.md.
Verified: full suite 2161 passed, frontend 273 passed, build clean,
npm audit --omit=dev reports 0 vulnerabilities.
The rollup took the highest fix version across a component's advisories. That
is wrong whenever an advisory is patched on more than one release branch.
nanoid is the case that exposed it. Both advisories against it are fixed on two
branches - 3.3.16 and 5.1.16, 3.3.18 and 5.1.6 - and the rollup recommended
5.1.16 with a MAJOR warning. But nanoid is transitive, pulled in by postcss at
`^3.3.11`, a range no 5.x release can satisfy. The recommendation was not merely
a bigger job than necessary, it was unreachable: the actual target is 3.3.18, a
patch inside the existing range.
Per advisory the target is now the lowest published fix above the version in
use; per component it is the highest of those, since one upgrade has to satisfy
every advisory against it. Fixes on older branches are skipped rather than
offered as a downgrade - axios advisories list 0.30.x fixes that would have been
a downgrade from 1.13.2.
This also gives the MAJOR label meaning. It now fires only when no fix exists on
the current branch, rather than whenever some branch happened to be higher.
START-HERE routes by the question somebody arrived with, and 'are we exposed to
this CVE' had no entry - the SBOM was only mentioned in the Windows update doc,
which is not where anybody asking that would look.
The SBOM was generated and installed onto every server, and nothing read it.
"Are we exposed to this advisory" was still a person going and looking, which
meant it happened when somebody remembered to.
scripts/scan_sbom.py queries OSV.dev by purl - no account, no API key, no agent,
no vulnerability database to maintain on a box that cannot reach the internet.
The SBOM is already the input OSV wants, so this is a lookup against what
shipped rather than a second scan of the tree that could disagree with it.
Three decisions worth recording:
Shipped and build-only findings are both reported, but only shipped ones can
fail a build. Two thirds of the components here are build tooling the SBOM
already marks scope 'excluded', and the highest-severity match in the whole tree
today is a CRITICAL against the test runner. A gate that fails a release over
that trains everybody to bypass it, and then it catches nothing.
Severity is taken from the advisory, never derived. Where there is only a CVSS
vector the report says UNKNOWN, which is shown and never gates. A guessed score
is worse than an honest gap because somebody would act on it.
The allowlist is a record rather than a mute button: each entry carries the
reason, the accepter, the date, and what would invalidate the reasoning. The
first entry is a worked example - a PyJWT algorithm-confusion advisory that
needs both mixed algorithm families and a raw JSON JWK as the key, and this
application configures neither.
The report ends with a per-component upgrade rollup, since that is the part
somebody acts on. A target on a different major version is labelled rather than
listed beside patch bumps as though the two cost the same.
Current state: 8 shipped components have fixes available, none of them
exploitable through a path this application opens today.
Three colour faults on the PC page, each hiding the next.
ACTIVITY BADGES WERE ALL GREY. The list renderer reads a badge colour from a
ROW FIELD, and the activity panel declared none, so every severity rendered
identically. The feed's whole job is to make a critical event visible and it
was painting it the same as a logon. The route now emits severitycolor - info
grey, warning yellow, critical red - and the panel declares it.
PAUSED AND THE TRANSITIONAL STATES WERE RED. statecolor was green-or-red-or-
grey, so a paused service looked like a stopped one. They are different: a
paused service was paused deliberately, and one stuck in Start Pending is
failing to start on a retry cycle - which is exactly what raises event 7011,
the thing behind the critical counts on the Fleet page. Both are amber now.
NAMED COLOURS ALWAYS GOT WHITE TEXT. readableText only parsed hex and returned
white for anything else, but panels emit NAMES. Green, red and grey are dark
enough that white worked by luck; amber is not, so the fix above shipped as
white-on-yellow and could not be read. Names resolve to hex before the contrast
test, so this holds for any named colour anywhere in the app rather than these
two panels.
colorStyle had no test file at all. It has one now.
Every one of these was found by looking at the rendered page. None of them
would fail a test that checked markup.
Full suite 2131 passed, vitest 273 passed.
Reported from the Activity card: the header dot was red while every visible row
was info, because the critical event was older and sat further down the list.
The dot and the list appeared to contradict each other, and the only way to
find the critical was to scroll.
The dot is computed from all rows the panel received, so a red dot always means
a critical IS in the data. The chips are computed from that same array, which
means the count sits directly under the dot and the two cannot disagree by
construction: the card now reads 'Critical 2 Warning 1 Info 12', and one
click surfaces them wherever they sit in time.
Derived from the panel's EXISTING status declaration rather than a new one per
panel. A card that shows a health dot has already said what it dots on, so
Tracked Services and Tracked Processes get chips with no plugin change, and any
future list panel with a status rollup does too. Two declarations of the same
thing would eventually disagree.
Details that matter: a colour-keyed panel is labelled by meaning, so services
read Critical and OK rather than Red and Green; chips are ordered worst first
rather than alphabetically; and no selection means show everything, not hide
everything - a card that emptied when you deselected the last chip would read
as broken. Filtering to nothing says so, and gives the hidden count.
vitest 267 passed, including the exact reported case.
A site running external mode keeps its devices and check-in/out history in a
separate cmmc_usb database. There was no way to bring that in, so the choice
was to keep the second database forever or lose the history. This adds
'flask usb import-external' - dry run by default, --commit to write - which
reads through the same connection settings the routes already use and never
writes to the source.
Two things it has to reconcile, and they are why it is a command rather than a
few INSERTs.
The schemas disagree about SHAPE. External is an event log, one row per action;
self-hosted is a paired row carrying both times. Pairing walks each device in
time order, and real logs after years of use are untidy. The rule is that
nothing is invented and nothing vanishes: a drive still out becomes an open
checkout and is NOT flagged (that is just a drive somebody has), a return with
no matching issue is skipped and named, and two issues in a row leave the first
open rather than inventing a return time nobody recorded.
Identity is the other. usbdevices.serialnumber is unique and NOT NULL and the
external devices table has no serial, so device_id becomes it. Blank or
duplicated ids stop that device rather than inventing a key. external
device_owner has no column here, so it is appended to the device notes rather
than dropped.
Also adds usbcheckouts.scannedviruses, the one field with nowhere to land.
waswiped already means what external calls sanitized - selfhosted.py was
mapping it - but scanned_viruses was HARDCODED None in the log output: the API
promised the field and had nothing behind it, so an imported history would have
rendered blank. It now reads and writes properly.
Nullable, never defaulted to false. An unrecorded scan is not a failed scan,
and for removable-media records those are very different claims.
Full suite 2131 passed; seven new tests pin the pairing rules.
Reported from production: health showed '418 critical' while the ten most
recent events were all info. Both were true, which is the problem.
The health rollup counts critical events in a 48h window; the activity feed
shows the fifteen most recent events of ANY severity. A box that logged 418
service timeouts overnight and ordinary logons since produces exactly that
pair - a page contradicting itself with no way to tell why.
Two fixes.
The rollup now names the DOMINANT event instead of totalling: 'Service timeout
x418' rather than '418 critical'. Counting by event id costs nothing (one more
GROUP BY column) and turns a number into a lead. This is the same reasoning
that made health a phrase rather than a score; it stopped one step short.
The activity feed now fetches the newest criticals separately and merges them,
and keeps up to five of them when trimming to the cap. They hold their place in
time but survive the cut, so what health is reporting always appears in the
feed that exists to explain it.
Verified against the reported shape: 20 service timeouts two hours old under 12
newer logons. Before, the feed was fifteen logons; now it carries five critical
rows and health reads 'Service timeout x20'.
Worth noting the underlying signal is real - several hundred criticals on one
endpoint in 48h is a box crash-looping or a service failing on a retry cycle.
The display was hiding it, not inventing it.
Full suite 2124 passed, naming and parity gates green.
Select endpoints, then Collect now or Restart across the selection. Restart is
gated on endpoints.restart, both are re-checked server-side.
The confirmation names the MACHINES, not just the hostnames:
Restart 2 computer(s)?
GJX9B2Z3ESF, HOST-7
This STOPS the machines these PCs serve:
DT Office
A dialog listing eight hostnames tells an operator nothing about what they are
about to halt, and that is the whole consequence of the action on a shop floor.
Where no machine is recorded it says so, because 'none listed' and 'we did not
check' are different things at 6am. Only possible now the Machine column exists.
A 409 is reported as a SKIP, not a failure. The queue permits one command in
flight per endpoint, so a bulk action across eight PCs where two are already
busy says '6 queued' and '2 skipped - a command is already in flight'. Folding
those into an error count would make a working button look broken.
Selections clear on every reload: the rows they referred to may not be on the
new page, and acting on an invisible selection is how bulk tools surprise
people.
No per-row buttons. They were built first, and a tenth column on a nowrap table
pushed itself off the edge - selecting one row costs a single click and leaves
one command path instead of two.
Full suite 2124 passed, vitest 260 passed, naming gate green.
The tiles were 6px while .card is 0.25rem, which left Fleet marginally rounder
than every other page. The global card treatment is a separate decision and has
been deferred; until it is made, this page should look like the rest of the app
rather than half-way to a mockup.
Four things the mockup showed and the first cut did not have.
FLEET
- A 'with critical events' tile, from the stability route that existed and
nothing called. Clicking it narrows to rows whose health is not ok - client
side, because the health rollup is assembled per row after pagination, so
it narrows the PAGE rather than the fleet. Worth knowing before the count
is read as fleet-wide.
- PC type and Location filters. The list route already accepted both; there
was simply no control. Both use listAll, not list: perpage is clamped to
100 server-side, so a long location list would lose its tail in a dropdown
while still looking complete.
- The machine cell shows number and name together where they differ, so
'3105 Okuma' rather than a bare number.
PC DETAIL
- A Policy card, contributed by the computers plugin. It matters because the
effective policy already narrows the command allowlist, so without it the
reason a control is unavailable on a given PC is written down nowhere.
Provenance, so it sits in the reference column, not the triage band.
Its route returns an EMPTY field list rather than the feature-gate 404 when
policies are disabled. The panel contract hides a card with no fields, so
it disappears on sites that have not enabled the feature - a 404 would have
rendered a broken card on most of them. That was flagged in the design
review as a snag; this is the fix.
- Refresh Status and Restart Computer moved out of the page header into the
triage band. They act on the MACHINE; Print Label, Edit, Retire and Back to
List act on the RECORD, and mixing them put the most consequential button
on the page - it halts whatever machine the PC serves - between printing a
sticker and editing a form. In the band it sits beside the liveness chip
and the machine it would take down.
The band takes them through an 'actions' slot rather than knowing about PC
verbs, so a measuring tool can put 'Calibrate' there without the core
component learning anything about computers.
Full suite 2124 passed, vitest 260 passed, gates green, spec at 333 paths.
Two changes to the PC page, both about what a responder meets first.
LIVE STATE FIRST. The contributed panels moved to the head of the card flow.
The grid is a two-column FLOW (column-count, with display:contents on the
column wrappers), so a card cannot be pinned to a side and only document order
decides where it lands - putting the panels first fills the left column with
endpoint status, services, processes, interfaces and activity, and leaves
identity, hardware, network and location to the right. Panels still self-order
among themselves by declared position and still self-hide when no plugin
answers for the asset type.
ONE ACTIVITY FEED replaces the Recent events card. Events and commands answer
the same question - what happened to this box - and reading them as two lists
meant interleaving by eye. GET /api/endpoints/asset/<id>/activity merges them
newest-first, capped at 15. Both are keyed by assetid, so it is two indexed
reads and a merge sort rather than a join, and it renders through the existing
ADR-010 list renderer with no bespoke frontend work. A failed command is marked
critical, because that is the row somebody needs to notice. Not collapsed by
default, unlike the card it replaces.
The core AuditLog is deliberately NOT a third source: its rows carry the action
as the entity name and the asset only inside a details string, so including it
would mean parsing prose to find out which machine a row belongs to.
Requester names are withheld from callers who cannot themselves issue commands,
via caller_holds. A read-only viewer - a guest, on a site that grants this
panel - gets the history without a roster of who restarted what, and the lookup
is skipped entirely rather than fetched and discarded. requestedby holds a user
ID, so names resolve through one batched query, not one per row.
Verified on dev telemetry: a queued command shows as 'Collect now / Pending by
admin' above the Windows events, ordered by time.
Full suite 2124 passed, vitest 260 passed, naming/contract/parity gates green,
spec regenerated at 332 paths.
Adds a shared triage band to the PC detail hero: is it alive, what stops
without it, how do I reach it.
'Serves machine' is the reason it exists. On a shop floor a PC matters because
a machine stops without it, and that fact was buried in the generic
relationships card at the bottom of the page. Both edge directions are read,
deduplicated, and filtered to assets typed 'machine' - production holds
reciprocal rows, so reading one direction shows the link on half the pages and
not the other half.
The band is a CORE component, not a PCDetail block. Five asset detail pages
already rebuild the same hero and grid independently, which is how they drifted
apart; machines, printers and network devices can drop this in rather than
growing a sixth copy. Liveness is passed in rather than fetched, because only a
plugin that collects telemetry can answer it and a site may not have one.
The status chip's verdict comes from the SERVER: /api/endpoints/<id> now
returns the same reporting/quiet/never it computes for the fleet list. The
quiet window is a site setting, so a threshold guessed in the browser would
disagree with the Fleet list about the same endpoint the moment a site changed
it.
Two cards removed from the body:
- Remote Access duplicated the band's chips. It is what a responder clicks
first, so it belongs above the fold, once.
- Status was worse than duplication. It reads the COMPUTER RECORD, while the
endpoint panel beside it reads live agent telemetry, and on an
agent-managed PC the record fields sit empty - so the page rendered 'Last
Reported: Never' a few inches from live telemetry saying otherwise. It now
renders only when that record actually holds something, which is the case
on sites running the asset-report script instead of the agent.
Verified against dev telemetry: band shows 'Reporting, 1 h ago', the machine it
serves, and its access chips; both duplicate cards are gone.
vitest 260 passed, naming/contract/parity gates green.
Two colour bugs on the same page, both invisible in light mode.
A button does not inherit color the way it inherits font, so the plain
Endpoints total rendered in the browser default black on the dark theme's card.
The coloured tiles set their own color, which is why only that one showed it.
The coloured ones were wrong too, just less obviously: dark mode redefines
--success and --warning to bright variants but leaves --success-dark and
--warning-dark tuned for a white background, so those numbers and the health
pills came out dark green and dark amber on a dark blue card.
Scoped to the stamped [data-theme="dark"] rather than a prefers-color-scheme
query, matching the note in style.css: the theme store always stamps the
attribute, and a bare media query leaks dark styling into light mode on a
dark-OS machine.
Twelve endpoints routes shared one permission, which made a reasonable policy
impossible to express. A site that wants signed-out visitors to see live status
on a PC page had to hand them the entire fleet - including every logged-in
username on the floor - because the per-asset panels and the fleet-wide list
were gated on the same string.
endpoints.fleet now gates the list and the three dashboard routes.
endpoints.view keeps the per-asset reads: status, services, processes,
interfaces, events, command history.
The name is load-bearing. guest_grantable_permissions() opens any '.view' or
'.read' permission to the guest role, so calling this one 'endpoints.fleet'
rather than 'endpoints.fleetview' is what keeps the roster off the public
surface while per-asset telemetry stays grantable. Verified both ways.
Also: the services panel emitted Start/Stop row actions to every reader,
because row_actions_for() never consulted the caller. Clicking was always
refused server-side, so nothing was exploitable, but showing a read-only viewer
controls over a shop-floor PC is the wrong signal - and it stops being cosmetic
the moment a site grants that panel to guests.
That needed a contract addition rather than a local check: caller_holds() on
shopdb.api answers whether THIS caller holds a permission without refusing the
request, which is the complement to require_permission for a surface that
serves everyone but should only offer what the caller can act on. Guests are
included, since a signed-out visitor holds whatever the guest role was granted.
Reaching into shopdb.utils.authz instead would repeat the ADR-002 violation the
contract gate caught earlier in this branch. Contract 0.24.0 -> 0.25.0.
DEPLOYMENT: endpoints.fleet is a new permission. Run 'flask seed permissions'
and grant it to the roles that should keep seeing the Fleet page. Until then
that page is admin-only, since admins bypass every check - a safe direction,
but not an automatic one.
Full suite 2124 passed, naming and contract gates green, map and spec
regenerated.
The fleet list showed hostname, user, uptime, CPU, memory and last-reported,
sorted by last-reported descending. Two of those columns have never held a
value, and that sort floats the healthiest endpoints to the top and buries the
ones somebody needs to act on.
What a shop-floor fleet view is actually for is which MACHINE stops if a PC is
dead, so that is now a column. Both edge directions are checked, deduplicated,
and constrained to assets typed 'machine' - without that last filter the column
filled up with office locations that happen to carry a controls edge.
Also here:
- Sorted WORST FIRST: never reported, then quiet, then reporting. Expressed
as a sortable SQL rank rather than a label applied after the fact, so the
order survives pagination.
- Health rollup as a PHRASE, not a score: 'eDNC stopped', '2 critical'. A
percentage does not tell an operator what to do.
- Filters for status, PC type, location and business unit; a sort parameter
(the route accepted none); and search widened from hostname alone to asset
number and the machine number a PC serves, which is how people actually
look a bay up.
- CPU and Memory columns dropped. The agent declares those fields and has
never assigned them, so they rendered '-' on every row.
- A summary strip wired to the dashboard routes, which existed and which
nothing called.
Two bugs found while building it:
The health rollup first read a 'running' boolean, but service rows carry a
STATE STRING and the vocabulary varies by source, so every service would have
reported as stopped. The panel and the rollup now share one _service_running()
helper; two copies of that rule would drift.
The summary counted quiet as total - reporting, which folded never-reported
endpoints in with quiet ones. Those are different failures - an enrolment or
key problem versus a machine that went down - so 'never' is counted separately.
Per-page batching for machines and critical-event counts: three queries per
page rather than two per row.
Full suite 2124 passed, vitest 260 passed, naming and contract gates green.
get_pagination_params reads 'perpage' and only falls back to 'per_page', and
this client has always sent 'perpage', so the interceptor's duplicate never did
anything except make every list URL carry a parameter twice.
Removing it also clears a live production problem. The endpoints list route used
to answer a 308, which is a PERMANENT redirect: the browser caches it against
the exact URL including its query string. Clients that hit the old route keep
that rewrite after the server is fixed, because the request no longer leaves the
machine - nothing deployed can reach it, and it fails as mixed content when the
cached target is http. Confirmed in the field: the page failed on load with the
default query string while a search, whose query string differs, worked in the
same session.
Changing the query string changes the cache key, so an affected client misses
the poisoned entry and reaches the corrected route without the user having to
clear site data.
Plugin views that pass per_page explicitly still work - the backend fallback is
untouched. vitest 260 passed.
An endpoint reported telemetry correctly for hours while the Fleet page showed
"No endpoints found". The PC detail page displayed its panels the whole time, so
ingest was plainly working; only the list was empty.
The list route was the one route in 331 paths registered with a trailing slash:
@endpoints_bp.route('/', methods=['GET']) -> /api/endpoints/
The browser client calls it without one, so Flask answered a 308 redirect to an
ABSOLUTE url. On dev that resolves and nobody notices. Behind the IIS subpath it
cannot: with no ProxyFix, Werkzeug builds that Location from the scheme and host
it believes it has, so it emitted http:// and dropped the /shopdb prefix. A
browser will not follow an https page to http, the request failed, and the
page's catch block rendered the empty state.
Every other route in the plugin worked throughout, because none of them carries
a trailing slash - the detail panels call /endpoints/asset/<id>/status and were
never redirected. Detail works, list empty is the signature of this bug.
Registered as '' now, so the rule is /api/endpoints. That is what
CONTRIBUTING.md's API section spells and what every other list route in the
product already did. The inventory and the generated spec are updated to match.
Also here:
- tests/test_core/test_no_redirecting_routes.py fails the build if any /api
rule is registered with a trailing slash and strict slashes. Verified by
planting the violation: it named the offending route, then passed once
reverted. It takes the shared app fixture rather than building its own
application - doing that rebinds SQLAlchemy and leaves the NEXT module
without tables, which showed up as three unrelated "no such table: settings"
failures visible only in a full run.
- The Fleet page now renders "Could not load endpoints (HTTP nnn). This is a
failed request, not an empty fleet." A list that reports a 403 and an empty
fleet identically is what made this take as long as it did.
- TRUSTED_PROXY_COUNT (default 0) applies ProxyFix when set. Off by default on
purpose: it makes X-Forwarded-* trusted, and a client can send those itself,
while the login rate limit and account lockout are keyed on the client
address. Set it to 1 on an instance behind IIS.
Full suite 2124 passed, vitest 260 passed, naming and docs gates green.
The export died with no output and no dists, exit 1, having committed nothing.
grep exits 1 when it matches NOTHING, and under set -euo pipefail that failing
pipeline takes the script with it. Once every internal reference had already
been scrubbed, the gitea grep matched nothing, and the SUCCESS case became the
abort. It printed not one line explaining itself, which is what made it cost an
afternoon: the run looked like it had worked.
All three scrub greps are guarded now. This is the third failure of this shape
in this file, after the two fetch/push tag omissions.
Also excludes a stray .env copy from the sync. A hand-made .env.bak-<date>
reached the publication tree, because rsync walks the WORKING TREE and
.gitignore does not apply. The scrub gate caught it, which is what that gate is
for, and it never reached a commit or a bundle - but a file of live credentials
should not get that far. --include '.env.example' comes first so the example
still publishes.
Both verified by planting a violation: a no-match grep under pipefail, and a
planted .env.bak file that the sync now leaves behind.
The endpoints plugin declared a section of its own, so the sidebar rendered a
one-item 'Fleet' group at the bottom, below Information, far from the PCs the
view is about. Its own docstring said it belonged just after PCs, and its
position (16, against PCs at 15) already said so too; only the section
disagreed.
Filed under assets, which puts it directly under PCs. The now-empty fleet
section is removed from NAV_SECTIONS - an unused section is an invitation to
file something else there.
The printers observedqueues migration failed with error 1071 on a 5.6 dev
server whose file format was still Antelope. The unique index it needs is
(hostname, queuename), two utf8mb4 VARCHAR(255) columns, which is 2040 bytes
against a 767-byte cap.
The compiler hook in shopdb/utils/mysql_charset.py was already emitting
ROW_FORMAT=DYNAMIC, which raises that cap to 3072. What is easy to miss is that
Antelope makes 5.6 IGNORE the row format and build a Compact table with a
warning rather than an error, so the tables look created and keep the old limit.
docs/DEPLOY.md has required innodb_large_prefix plus Barracuda for pre-5.7
servers all along; nothing pointed a developer hitting 1071 at it.
So the troubleshooting row that said 'use MySQL 8 for dev' now gives the fix,
names the silent Compact downgrade, and says such tables stay capped until they
are rebuilt with ALTER TABLE ... ROW_FORMAT=DYNAMIC.
DATABASE_URL also gains ?charset=utf8mb4, which DEPLOY.md already requires and
this environment did not have.
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.
What arrives:
endpoints plugin - a ShopDB agent reporting endpoint state, telemetry,
services and interfaces, with a command channel the agent
long-polls. Ingest stays on the generic collector route
(ADR-006); the agent key is a UI-managed single-scope
token whose revocation is an instant kill switch.
computers policies - Configuration and Business Policies (RMM-style), scope
trees validated at write time, actions checked against
the endpoints command allowlist. Feature-gated: every
route 404s while the setting is off, so enabling it can
never half-expose the surface.
asset CSV import - templates, a collector script carrying the site path, and
a validate-then-commit import. A plugin can own a virtual
column plus a post-write row hook, which is how a PC's
machine number becomes a controls link without a
hand-built relationships sheet.
agent/ - the Windows agent and its installer.
Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.
MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).
Review fixes carried on top:
- six policy GETs were jwt_required(optional=True), which authenticates a
caller without checking anything; they require computers.view now, and six
redundant jwt_required decorators are gone
- the agent installer took a default BaseUrl and CollectorKey, so a mistyped
run silently pointed an endpoint at the wrong instance; both are mandatory
and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
- a committing CSV import wrote no audit row
Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.
Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
Every job on GitHub is warning:
Node.js 20 is deprecated. The following actions target Node.js 20 but are
being forced to run on Node.js 24: actions/checkout@v4, actions/setup-node@v4
Working today because the runners shim it, and a hard failure the day that shim
goes. checkout@v5, setup-node@v5 and setup-python@v6 are the releases built for
Node 24, so the warning goes away rather than being suppressed.
Nine references across five jobs. The comment above them says why, so nobody
pins them back to the v4 generation on the assumption it is the safer number.
The org's actions policy allows GitHub-owned actions with no SHA pinning
required, so the tag change needs no allow-list update.
Two faults with one cause: docs/ is excluded from publication because the prose
becomes the wiki, and two things under docs/ are not prose.
shopdb/core/api/docs.py reads docs/openapi.json AT RUNTIME to serve /api/docs.
Excluding it meant every site installed from the published repo shipped the
ReDoc page, loaded it, and got {"error":"openapi.json not generated"}. The spec
now publishes - it is a product file that happens to live under docs/. The prose
still goes to the wiki and nowhere else, and docs/api-inventory.json stays out,
because nothing reads it at runtime.
The second fault is the reason this was worth chasing. Three gate modules skip
themselves when the file they read is absent, which is correct in a published
checkout and silently wrong on GitHub, which runs against exactly that checkout.
The modules skipped, CI reported green, and an outside branch added nine
undocumented routes and passed. A gate that stops running is worse than no gate,
because nobody looks at it again.
scripts/check-gates-ran.sh asserts each gate ran, with two lists: the ones that
must run everywhere, and the ones needing files publication deliberately leaves
behind - required here, allowed to be absent there. A `gates` job runs it in CI.
Proven in both directions: green in each checkout, and red when the file a gate
reads is removed.
Also purge instance/plugins.json from the publication tree. It was never
published - it is untracked there - but an old copy had sat in that tree for
months, protected from --delete by its own exclude, and it made the published
repo fail its own suite locally: 13 plugins enabled where the code expects 16.
A publication tree that cannot pass its own tests cannot be verified before
shipping. That is the exact trap the comment at the top of the script warns
about, which had claimed a second victim.
Also: concurrency on the workflow, so a second push supersedes the first rather
than queueing behind it holding a self-hosted runner.
A link on its own invites being pasted onto text that is already in the chat
box, and every chat client turns the selection into the link's caption. The
secret then travels as the visible message with the secure link wrapped around
it, protecting nothing.
This is not hypothetical. Six LAPS passwords went out that way in production -
machine names and passwords as the caption, the correct /s/<token>#<key> URL
underneath, doing nothing at all.
Copy message puts a whole sentence on the clipboard: what the link is, how many
opens it has, when it expires, and then the URL. A sentence has nowhere to
attach itself - pasted into an empty box it is a message, pasted over a
selection it replaces it rather than decorating it. Plain text on purpose; a
rich-text flavour is exactly what would let a secret ride along as markup.
The success card now says to delete the password from the chat box before
pasting. The existing hint explains that the key cannot be recovered, which is
true and does not stop anybody pasting the password itself.
Nothing about the link was wrong and nothing about it changed: the URL carries
the token, the key rides in the fragment the browser never sends, the plaintext
is dropped the moment it is sealed, and redeeming stays a POST so a preview
cannot spend it.
Sending somebody a label meant sending a screenshot and a list of settings to
retype, and they would get one of them wrong. Copy share link hands them a URL
that reopens the generator with the content, the label, the copy count and
every layout value that was actually changed.
The whole payload rides in the fragment. A fragment is never transmitted, so it
does not reach Flask, it is not in the IIS log, and the proxy between them never
sees it - which matters because the content is usually an asset number or a
serial, and those have no business scattered through logs. It is NOT secrecy:
the link sits in a chat message and in browser history, and anyone holding it
can read the content. Secure Send exists for the case where that matters, and
the note under the box says so.
Only values that differ from a first-time visitor's defaults are carried, so
the link stays short enough to paste into a chat box; a shared link then wins
over the recipient's own saved settings, because somebody following one is
asking to see THAT code rather than their own last one. An unknown key is
ignored rather than refused, so a link made by a newer version still opens on
an older one. The picture is never carried - a data URL runs to megabytes.
Values are coerced back to the type the field holds. Everything arrives from a
URL as a string, and a nudge of 0 plus '2' is '02'.
Proven end to end: a link made in one browser, opened in a clean context with
no saved settings, rendering the same code.
It struck the heading and left everything underneath looking untouched, which
is the opposite of what somebody ticking "this section is done" means. The
strike now covers every block under that heading, down to the next heading of
the same or higher rank - so a top-level section takes its sub-sections with
it.
That needed the rank, and the HTML importer was throwing it away: h2, h3 and h4
all arrived as a flat 'heading', so a parent heading had no children to cross
out. It was the visible half of a real loss - a converted article kept its
words and lost its shape. Headings now carry their level and render at it, and
a guide written before this reads as flat, exactly as it did.
Code blocks, tables and images inside a finished section fade but are not
struck. A line through a command makes it unreadable, and re-reading the
command is usually the reason to look back at a step already done.
The five imported guides were re-imported so they carry their structure.
Every tool on that page is usable without an account, on purpose - the routes
carry no requiresAuth and say so - and Secure Send is the one write this
product offers a guest. The sidebar entry still never appeared for them: it
named no permission, so the filter that decides what a signed-out visitor may
see had nothing to go on and dropped it.
The plugin had no read permission to name, only tools.secrets, so it gains
tools.view: reaching the page at all. The tools themselves run in the browser
and touch no data, which is why this is about whether the entry appears rather
than about what anybody can read.
The item names either right. Somebody granted only Secure Send needs the entry
that leads to it, and a manager who holds neither still does not see it.
The sidebar hides what the caller cannot reach - that is what makes it useful
to a guest, and what stops a member with a sparse role being offered links that
can only answer 403. It reads the permission off the nav item to decide.
Only four items ever declared one. Machines, PCs, Printers, Network, USB,
Knowledge Base, Notifications, Calendar, Dashboard, Map, Applications and
Reports declared nothing, so the filter had nothing to go on: granting the
guest role machines.view opened the API and left the sidebar empty, which reads
as the grant not working.
Every item now names its read permission - in the plugins, in the core
navigation endpoint, and in the frontend's fallback list, which is what renders
when the endpoint is unreachable and would otherwise disagree with it.
A sweep of the URL map found 165 GET routes that answered anybody, with no
token at all: the whole asset inventory, every machine, PC, printer and network
device, the USB checkout history and who is holding which drive, the staff
directory with its photographs, every core report. Guest access had been argued
about one feature-flag at a time while the front door stood open.
Each of those now carries the read permission it belongs to. That also makes
the guest role mean something: a site can open exactly the reads it chooses,
and nothing follows by accident.
Thirty-three endpoints stay public, and each is named with its reason rather
than left to be inferred: the app shell and its bootstrap, the settings
allowlist (the handler already filters an anonymous caller down to branding and
a few keys), images fetched by <img src> which carries no Authorization header,
what a display kiosk renders before anybody signs in, the parts and USB kiosks,
the client scripts that run as SYSTEM before a user session exists, and a
secure-send link whose unguessable token IS the authorisation.
tests/test_core/test_public_read_surface.py is the standing gate, the companion
to the mutating-route sweep that has guarded writes for months. A new public
read fails CI until somebody adds it to that list in front of a reviewer. Four
further reads need a login but no permission, because the handler decides from
who you are - your own user record, your own API tokens, the role catalogue the
grid renders - and the same test proves each still refuses an anonymous caller.
Verified not vacuous by ungating one route and watching the sweep name it.
The employees plugin gained employees.view; the directory is names, SSOs,
photographs and reporting lines, and it had no permission of any kind.
Around ninety tests asserted payload shapes against endpoints that answered
anybody. They are about what the answer says, not about who may ask, so they
are signed in now rather than the gate being loosened. The ones that assert an
anonymous caller is refused were left exactly as they were.
Only checklist items could be ticked, which answers "what have I done" but not
the question a long procedure actually raises: where was I. Somebody looks away
to do the thing the guide asked for and looks back at a wall of text.
Every step and every heading now carries a checkbox while reading. Ticking one
strikes it through and fades it rather than hiding it, because somebody
scanning back up the page has to still be able to read what they did. The
number is struck through with it, but the step's own "Step 3" label is not -
that is the address of the thing, not part of what was done.
Headings count, not just steps. A guide converted from an HTML article often
has fifty headings and no steps at all, and those are exactly the guides long
enough to lose your place in. That also means the card's "N steps to tick"
stops reading as a broken feature on a guide that is entirely tickable.
Reading only, as the checklists already were: in the editor a tick would be an
edit, and an author clicking around their own draft would quietly change what
the next reader sees as done. Progress stays per browser and per version -
shopfloor PCs share a login, so one technician's ticks must not greet the next
person on that bay, and step 4 of version 3 is not step 4 of version 2.
Whether somebody with no account may see something was being answered one
feature at a time: a securesend_allow_guests setting, an enable flag per kiosk,
and very nearly a third for guides - each with its own settings page, its own
vocabulary and its own hand-written branch inside a route. Every one of them
answered a permissions question, and this app already administers permissions
per role.
So a caller with no token is not nobody: they hold the seeded `guest` role.
require_permission consults it when there is no identity, which is the entire
mechanism. The role is seeded holding NOTHING, so an upgrade opens no door; a
site opens one by ticking a box in the same grid it uses for every other role,
and closes it the same way. Revocation is read on every request rather than
cached, because taking guest access away is what somebody does the moment they
realise what they opened, and it has to bite immediately.
The redundant @jwt_required() above require_permission is gone from 184 routes.
It was belt-and-braces before; now it is actively wrong, because it refuses a
request with no token before the permission check can have an opinion, so a
grant would appear to do nothing.
What a guest may hold is bounded. Any .view or .read permission is grantable.
A WRITE has to be blessed by whoever owns it, through the new
get_guest_grantable_permissions hook (contract 0.21.0), and Secure Send is the
only case that exists: sending a password to a vendor is the one write this
product means to offer somebody who never signs in. Everything else is refused
by the API and greyed out in the grid rather than warned about - a checkbox
that quietly makes deleting an asset anonymous is not a warning anybody reads
twice. A guest write records no creator: the trail says when and from where,
never who, and that is the reason for the bar.
GE-Enforce and Slides gained read permissions so they can be opened to guests
without handing out the manage right. Their read routes accept either through
require_any_permission, so nothing an existing manager could reach has moved.
The navigation endpoint now returns only what the caller can actually reach.
Without that a signed-out visitor gets a sidebar of the whole app where every
link bounces off the login page, which reads as a broken site rather than as a
closed door. A nav item may name several permissions; holding any one is
enough, which is what GE-Enforce and Slides need now that a viewer and a
manager hold different strings.
Two things this found on the way: current_user is a LocalProxy, so
`current_user is None` is False for a request with nobody behind it - harmless
under a required JWT, an AttributeError the moment verification became
optional. And an image pasted into a guide before its first save was never
attached to it, so it was visible to editors and invisible to every reader;
saving now attaches it, and `flask guides fix-images` repairs what is stored.
The guides guest setting written earlier today is gone. Granting guides.view to
the guest role replaces it, which is the point.
Guides are written on one site and needed on all of them, and the sites that
matter are air-gapped from wherever the writing happens. Until now a procedure
was stranded in the database it was typed into: the HTML importer could rebuild
one from its original article, but anything authored in the editor had nowhere
to go.
flask guides export writes one .guide.json per guide, with every image the
blocks reference carried inside it as base64. One file on purpose - a bundle
that needs its images copied alongside it is a bundle that will eventually
arrive without them. flask guides import reads them back, a directory at a
time, and refuses anything that is not a bundle before it writes a row rather
than half-importing a stray .json sitting in the same folder.
What travels is the content: title, summary, tags, blocks, images. What does
not is ids, revision history, authorship and publication state, because those
belong to whichever database issued them - an imported guide starts as a draft
at revision 1, owned by whoever ran the import. The SLUG does travel: it is the
address somebody pastes into a work order, and the same guide should answer to
the same name on every site.
--replace updates the guide already carrying that slug instead of creating a
second one, which is how a corrected guide is re-shipped. It drops the stored
images first: the incoming bundle is the whole truth for that guide, and a
guide re-shipped monthly would otherwise accumulate orphans nothing points at.
Bundles refuse a version they are too old to read rather than importing the
parts they recognise. A newer site's guide is not a guide this one can be
trusted to render.
Proven by round-tripping a five-guide export back into the same site, including
one carrying two PNG screenshots, and reading it in a browser.
A site adopting this app arrives holding procedures already written: styled
HTML pages that get mailed around, opened off a share, and edited by whoever
still has the file. Those pages are exactly what a guide is for, and retyping
one into the editor is how a good procedure stays on a share forever.
flask guides import-html reads the page and produces blocks. It recognises the
vocabulary those pages actually use - a step box with its own number, an alert
in one of three colours, a code block with a copy button, a checkbox list, a
table with a header row - and maps each to the block that already renders it,
lifting embedded images into guideimages so a document stops carrying its
screenshots as base64. Page furniture goes: the header, the footer, and the
table of contents a guide builds for itself from its own headings.
The conversion is lossy on purpose. Colour, fonts, width and nested layout are
dropped, because storing blocks rather than markup is what makes dark mode,
printing and tickable steps work at all. So an import is always a DRAFT and the
default is a dry run that reports what it would produce and writes nothing.
Blocks go through the same validator the API uses: an import must not be able
to store something the editor could not have produced.
The parser is tolerant, because real articles are not well-formed. An unclosed
<p> is normal HTML and the browser closes it at the next block element; without
the same rule one unclosed tag swallows the rest of the article into a single
paragraph, which is what it did before the test caught it.
Proven end to end against four real articles, including one carrying two PNG
screenshots, read back in a browser.
POST /api/reports/email has always taken any {columns, rows} from any caller,
but the Email report button existed on the toner and warranty pages only - not
because those reports are special, but because they happen to be their own
routed pages and the reports page had no button at all. It is now beside Export
CSV for every report the page can build a table from, which is all of them:
plugin reports through their declared columns, core reports through the flat
list they return, and the asset inventory summary by flattening its three
tables into one with the grouping named in a column. Presentation-only fields -
a status colour, a link id - are left out; a column of hex codes is worse than
no column. The shapes live in a module with tests rather than in the view.
The button is now hidden without reports.export on all three pages. Offering
somebody a button that can only answer 403 is worse than not offering it.
Separately, on the USB kiosk a long drive name pushed Take or Return out
through the side of its card, leaving a drive nobody could press. The name is a
flex item and so defaulted to min-width:auto, keeping its full intrinsic width;
it now yields, wraps mid-word for the serial-style names that carry no spaces,
and clamps its description to two lines so one long name cannot make every card
in the row tall. The locker line is never clipped - it is how somebody finds
the drive.
A cabinet of thirty drives was one long column to read through. The free list
now filters from a search box and lays out in as many columns as the panel is
wide - four on a wall display, two on a tablet, one on a phone - while the
steps that hold a single control stay narrow, because stretching one input
across a 24 inch screen reads as a broken page.
A drive is found by its id, by what is written on it, or by where it lives. The
words can arrive in any order, and "locker" is part of what is searched, so
"locker engineer" finds the engineering locker - that is how people say it out
loud. Filtering is client-side: every free drive is already on the screen, and a
round trip per keystroke on a wall panel would be worse than no filter at all.
The matcher lives in its own module with tests rather than inside the view.
The box takes focus as the picker opens so somebody can simply start typing,
and it is cleared when the kiosk resets - the next person at the cabinet must
not inherit a filter and conclude there are no drives left.
Also: both this view and the guides blocks coloured muted text with a
`--textmuted` variable this app has never defined. The browser dropped the
declaration, so every hint, subtitle and locker line was rendering at full
strength. The variable is `--text-light`.
The reports page ran a hardcoded switch over five core report ids and fell
through to `default: console.error('Unknown report')`, and its results panel was
a matching chain of per-id templates ending in a `JSON.stringify`. So a plugin
could register a report, watch it appear in the catalogue, and get a wall of
JSON when it was opened - which is where the three printedparts reports have
been since they were added.
Any report that declares an endpoint is now fetched from it, and any report
returning {columns, rows} renders as a table. That shape is already the
convention on the server side, so a plugin needs no core code at all to have a
working report. The per-id templates stay for the core reports whose payloads
are not tabular.
Export CSV goes through the api client rather than window.open. A new window
carries no Authorization header, so every report behind a real jwt_required -
which is every plugin one - answered the export button with a 401. The core
reports are jwt-optional, which is the only reason the old way ever appeared to
work. It also followed /api/reports/<id>, which does not exist for a plugin
report, instead of the endpoint the report itself came from.
A report that fails now says so instead of leaving an empty panel, and the page
offers the time window the endpoints already accepted - printedparts-consumption
was permanently stuck on its 30-day default because nothing could send one.
Verified in a browser against both plugin report families and the core reports.
CMMC removable-media tracking is an audit obligation, and answering "who had
this drive, and when" meant paging through the check-in/out list by hand.
GET /api/usb/reports/audit returns the events - a take and a return are separate
rows, because a count cannot answer the question an audit actually asks - with
summary figures riding along for the people who only want the shape of it, and
a CSV for the people who want it in a spreadsheet.
The window is named rather than a day count: today, last month, six months, a
year, all time. Nobody should have to work out that a year is 365.
Both directory modes answer in one voice. The external path reads the
cmmc_usb checkinoutlog; self-hosted expands one usbcheckouts row into its two
ends, so a returned drive is distinguishable from one still out, and the window
filter runs in SQL against either end of the loan - a drive taken before the
window and returned inside it is a check-in the window has to show. Rows come
back from the mode-specific half; the CSV, the summary and the response shape
are built once in the route.
Ordering is total in both modes. Timestamp alone ties for two events in the same
second, and the same query then returns them in a different order each run,
which reads as the log being unstable.
The kiosk was a single column of full-width rows top-aligned on the page, so a
cabinet of a dozen drives meant scrolling a wall panel, and the page itself
scrolled - which takes the "Your drives" heading off screen and leaves somebody
looking at a list with no idea whose drives they are. The lists now sit in a
grid that fills the width and scroll INSIDE themselves against a bounded
height, with the panel centred as one block the way the parts kiosk beside it
already is. Two kiosks on the same floor that sit differently on their screens
read as two different products.
Drive ids sorted as text, which puts USB-10 ahead of USB-3. Compared with a
numeric collator so the list runs in the order the labels on the drives read.
fetchUser() existed and was called from nowhere. The signed-in user - including
the permission list every v-if in the app reads - was written once at login and
cached in localStorage, and nothing ever refreshed it. So an admin granted
somebody guides.create, the person was told they had it, and the New guide
button stayed hidden until they happened to log out and back in. Nothing on
screen explained why, and the natural conclusion is that the permission system
is broken.
Called on app boot, so a permission change lands on the next page load.
Reproduced before and after: button hidden, permission granted mid-session,
plain reload, button there.
Only a 401 logs out now. The old handler logged out on ANY failure, so an API
restart or a network blip would have thrown people out of sessions they still
held - a cached user is stale, not invalid.
The kiosk page tells an administrator to enable it in Settings under USB. That
place did not exist - the setting was reachable only through the API or a shell,
which is the same trap the backups plugin shipped with. A settings page and a
catalogue card, so the instruction leads somewhere.
Machines, PCs, printers and measuring tools all carry Print Label / Edit /
Retire / Back to List in a page header. A network device had no header at all:
Edit was tucked inside the hero title row, and Print Label and Delete sat in a
bar at the very bottom of the page, past every card. Managing one meant looking
somewhere different from every other asset type.
They move to the standard header. The bottom bar is removed rather than left as
a second copy, and the two delete handlers collapse into one - the route is a
soft delete, so it is called Retire here as it is everywhere else, and the
confirm says what is kept.
Four things people kept tripping over.
The move controls were a literal ^ and v character. They are arrows now.
Sections can be dragged. The handle is a grip in the block bar, not the block
itself: making the whole container draggable swallows text selection inside its
own textarea, so you could no longer select a word to bold it.
The type palette lived once, at the bottom of the entire guide. Adding a step in
the middle meant adding it at the end and walking it up one press at a time.
There is now an insert point in every gap - and at the end, so there is one way
to add a section rather than two.
Linking to another guide or a KB article meant knowing its slug or id by heart.
A picker searches both by title and writes the markdown link at the cursor. It
offers published guides only - never send a reader to a draft - and it stores an
in-app path rather than an absolute URL, so the link survives the site being
reached by another hostname or mounted under a subpath. The knowledge base is a
separate plugin that a site may not have installed; its absence costs nothing
but its results.
The section headings were derived from position ranges: 10-29 became Assets,
30-49 became Information. A number is for ORDER; it was never a statement about
meaning, and using it as one meant a plugin landed in whichever bucket its
number happened to fall in.
That is how USB Devices - an asset - came to sit under Information beside
GE-Enforce and Tech Tools, and 3D Parts with it. Nobody chose that. It also
meant two plugins picking the same number silently reordered each other:
Warranties and Guides both held 8, the tie broke on plugin load order, and
Guides rendered under Warranties rather than under Slides where it was asked to
go. Warranties moves to 9.
Each nav item now names its section and the renderer groups by what was
declared. An item with no section stays in the unlabelled group at the top,
which is the safe default: it appears, it is simply not filed.
Assets gains USB Devices and 3D Parts. GE-Enforce and Tech Tools move to a new
Fleet section - they are neither assets nor reference material, and calling them
Information was only ever an artefact of their position numbers.
Also: sorting used `position || 99`, and 0 is falsy in JavaScript, so Dashboard -
the only item that uses 0 - sorted to the end of its own group.
The kiosk route lives outside AppLayout, so nothing linked to it - a full-screen
page with no way in from the app is a page nobody finds. It joins the Displays
section beside Parts Kiosk, TV Slideshow and the Shopfloor Dashboard, gated on
the route existing so a build without the USB plugin does not advertise it.
The people who actually take USB drives are standing at the cabinet, and most
of them have no account here. Check-out and check-in were gated behind a login
that was not doing much anyway: both modes already attribute the action to a
scanned BADGE rather than to the signed-in user, so the login was gating the
screen, not the record.
/usb-kiosk is a full-screen touch page modelled on the parts kiosk, off unless
a site sets usb_kiosk_enabled - every route 404s until then, rather than 403,
so a disabled kiosk does not advertise that there is a door here. It works
against the self-hosted tables or an existing cmmc_usb database.
THE LIST IS THE TRADE. These drives carry no barcode, so unlike the parts kiosk
there is nothing to present and the screen has to offer a list - which exposes
more than a scan-first design, where you had to already hold a drive to learn
anything. So the list is not anonymous: an SSO comes first and must resolve to
a real person before any drive is named, which is the same bar taking a drive
already had to clear. It is a POST so the badge stays out of proxy access logs.
The main screen shows the caller's own drives, what is free, and a COUNT of
everything else; holders of other people's drives appear only behind a second
deliberate tap, so nobody reads the whole cabinet off a wall panel in passing.
A badge that resolves to no employee is REFUSED, unlike the signed-in route
which creates the user row. A checkout nobody can trace to a person defeats the
tracking, and at a kiosk there is no operator standing there to vouch for it.
Returning somebody else's drive asks first and then records the badge that was
SCANNED, so the log says who actually handed it back. That is the honest record
and it needs no change to a schema this app does not own. Refusing a mismatch
outright is a setting but not the default: a drive left behind by somebody on
leave would otherwise be unreturnable at the kiosk, which just moves the problem
to a person with a login.
The authz sweep caught these writes, twice - once for the two mutations and
again for the two list endpoints. Both are exempted with the reasoning written
out beside the parts-kiosk entry rather than silenced. The list endpoints are
POSTs that change nothing.
Verified by driving the real page: SSO on the keypad, tap Take, database moves.
.env is committed git-crypt encrypted. Without the key the checked out
file is the raw encrypted blob, and python-dotenv fails reading it as
UTF-8 with "invalid start byte". The migrations job already provides
DATABASE_URL, SECRET_KEY and JWT_SECRET_KEY through env:, so removing
the file is enough. Previously masked by the job dying earlier on a
missing mysql client.
A service container is reachable on 127.0.0.1 only when the job runs
directly on the runner host, which is how GitHub-hosted runners work and
how the old host-executor runner worked. Under a container-mode runner
the job is itself a container on the services network, published ports
land on the runner host rather than the job's own localhost, and the
service answers to its name.
Fixes: pymysql.err.OperationalError (2003, "Can't connect to MySQL
server on '127.0.0.1' ([Errno 111] Connection refused)").
Three failures, all from the workflow inheriting whatever the runner
happened to have rather than declaring what it needs.
- backend ran pytest without installing it. pytest is in
requirements-dev.txt, which nothing installed, so the step died with
"No module named pytest".
- Both Python jobs used the runner's system interpreter. The old host
executor had a matching Python by accident; a container image has
3.10, where alembic additionally requires tomli. requirements.txt is
hash-pinned and compiled on 3.14, so tomli has no hash and pip refuses
the whole install under --require-hashes. 3.14 matches the Dockerfile,
the README and the .github workflow.
- The utf8mb4 step shelled out to the mysql client, which is not in
every runner image. pymysql is already a runtime dependency.
The comment explaining the concatenation fix spelled the example hostname out
in full, which tripped the publication scrub - the same trap the surrounding
comment warns about, since the scrub greps this file for the very strings the
patterns are assembled from. Reworded with a neutral example.
Re-verified after the reword by planting a violation and watching the gate fail.
Guides was shipped without an entry in PLUGINS.md or START-HERE, so a site
adopting the product had no way to find out it exists.
BACKUP-KINDS.md described a host's badge as coming from the most recently
confirmed kind. It has not worked that way since the fleet report was reworked:
the freshest revision per (host, kind) is taken, and then the STALEST of those
kinds decides, because reporting the freshest would let a healthy NTLARS backup
mask a part-marker capture that stopped a month ago. Stalest among kinds that
can be JUDGED - a kind never confirmed is unmeasurable, not stale, and treating
it as the worst value made a host with a working backup show no badge at all.
The backups row also still advertised UDC and CMM kinds that do not exist.
api-inventory, openapi.json and PROJECT-MAP regenerated.
Unlike its twenty siblings it ran a bare add_column plus create_index, so
`flask plugin upgrade-all` failed with "duplicate column name: gagelabtag" on
any database whose tables were built by create_all and stamped nowhere - and a
chain stops at its first error, so 0004 never applied either.
The index is guarded by COLUMN SET rather than by name: create_all names indexes
by SQLAlchemy's convention and the migration by this repo's, so a name-keyed
guard would have added a second index over the same column instead of skipping.
The test that should have caught it was restricted to plugins whose whole chain
is a single baseline, on the reasoning that a longer chain meets a second
problem - an ALTER whose column is already present - that guarding a create_table
does not fix. True, but the conclusion was wrong: the answer is to guard the
ALTERs too, not to stop testing the chains that have them. It now covers every
post-cutover chain, and was confirmed to fail against the unguarded migration
before the guard went in.
The NTLARS/DNC registry carries an FTP password in cleartext and it was stored
verbatim. That put it in the backuprevisions table, on the revision panel which
renders values, and in the download route which hands back a ready-to-import
.reg - so anyone holding backups.view could read it off a web page.
Redaction happens at PARSE, so the secret never reaches the database at all.
Redacting on display would have left it at rest and one query away.
What is deliberately NOT done is dropping the value silently. This codec already
refuses to discard an unparseable value, precisely because a backup that looks
complete and restores a machine with a setting missing is the worst outcome. A
redacted value keeps its name, is listed in the projection, and renders as a
COMMENT naming what must be set by hand. Emitting a placeholder as the password
would produce a machine that looks restored and cannot connect, and the
technician would go debugging the network.
scripts/redact_backup_secrets.py scrubs revisions recorded before this, dry run
by default. It leaves contenthash alone: the hash identifies the configuration a
bay reported, so rewriting it would make every bay's next post look like a
change and insert a duplicate revision across the fleet.
Separately, sourcehostname is now normalized at ingest. samesource() maps '' to
IS NULL for the query while the insert stored the raw '', so rows written by a
client posting an empty hostname could never match their own next post: every
post inserted, the table grew without bound, and the history filled with
identical entries nobody could read. Whitespace did the same thing more quietly.
Every domain catches broadly and returns no rows so one broken table cannot take
the whole search box down. That part is right. On its own, though, it makes a
DOWN domain indistinguishable from an EMPTY one: the technician sees no results,
is offered a spelling correction, and concludes the record does not exist. It
does - the search just could not look.
Failed domains are now named in the response and stated above the results.
The suggestion is deliberately still offered on a degraded run. Suppressing it
was tried first and is worse: a chronically unreachable domain, an external HR
directory say, would silently disable spelling help for every search for ever
with nothing saying why. Naming the failure is the honest fix; withholding a
second useful signal is not.
Guides search also moves to the published revision. It correctly limited itself
to published GUIDES, but matched and displayed title/summary/tags/blocksjson
from the Guide row - which is the WORKING copy. A published guide with
unpublished edits matched on wording that only exists in the draft, and the
reader opened it to find the word absent.
The audit log is append-only and nothing ever removed a row, so it grows for as
long as a site runs. This site reached tens of thousands of pages, which is past
the point of being useful: nobody pages through that to find anything.
Retention is OPT-IN and the default keeps everything. An upgrade that silently
started deleting a site's audit history would be indefensible, so
audit_retentiondays defaults to 0 and nothing is removed until somebody sets a
number. The Site & Facility card shows the log's size and exactly how many
entries a purge would delete BEFORE anything is pressed.
Deleting happens in committed batches. That is not an optimisation: one DELETE
over a million rows builds a single enormous transaction and holds locks across
the table while it runs, which on this server means blocking the collectors
posting inventory. The API caps its batch count so the request cannot outlive a
proxy timeout, and a run that stops early reports how much is left rather than
claiming it finished - a purge reporting success over 900k remaining rows would
be the same silent half-failure this codebase keeps getting bitten by.
`flask audit stats` and `flask audit purge` for the server, the latter reporting
by default and deleting only with --apply, with no batch cap because it is not
behind a proxy and a purge that stops halfway leaves the operator guessing.
audit.purge is a separate permission from audit.view. Reading the log and
destroying it are different acts, and plenty of people need only the first.
Chunking selects a page of ids and deletes those rather than using
DELETE ... LIMIT, which is MySQL-only while the suite runs on SQLite.
The patterns are assembled from fragments so the published script does not
contain the strings it greps for. They used the apostrophe-escape form
('a'\''b'), which EMBEDS a literal apostrophe, instead of the concatenation form
('a''b'), which joins them. Every pattern therefore carried a stray quote in the
middle and could never match a line of real code, so the check passed everything
from the day it was written.
Verified the only way that means anything: plant a violation, watch it fail,
revert. Fixed, it immediately found 25 real hits.
Among them, in seven places, the default ServiceNow URLs - one organisation's QA
tenant, shipped with the integration ENABLED. Any site that had not configured
ServiceNow still rendered ticket links, pointing its technicians at a tenant
they cannot reach; an adopting site would have been pointed into somebody else's
instance entirely. There is no sensible cross-site default for "which ServiceNow
is yours", so the URLs ship blank and the integration ships off, matching the
Zabbix block that got this right.
A named SaaS tenant is as site-specific as a hostname, so it joins the patterns.
Colocated .spec.js files are excluded the same way tests/ already is: sample data
naming a facility is not a shipped literal.
Images were a broken icon for every user. The stored URL was bound straight
to <img src>, and a plain image request carries no Authorization header while
this app keeps its JWT in one, so every fetch was a 401. They are now fetched
through the API client, which also fixes the stored '/api/...' path under a
subpath mount where the API is not at /api at all. A failed image says so in
words rather than rendering as a broken icon, and the editor's own thumbnails
go through the same path - an author needs to see the screenshot they pasted.
Revision and image routes gated on their parent guide. Both were gated on
guides.view alone, and revision ids are sequential, so being able to read one
guide was enough to walk the ids and read every unpublished draft on the site -
the opposite of what docs/GUIDES.md promises. An image in a PUBLISHED guide
still serves to any reader; one that only appears in a draft does not.
The guide list, the tag chips and search now match the PUBLISHED text rather
than the working copy. Matching the working copy handed readers a hit on
wording that is not in the page when they open it.
The slug no longer follows the title. It was regenerated on every save that
carried a title and the editor autosaves every three seconds, so fixing a typo
in a heading silently moved the URL and broke exactly the pasted links the
stored slug exists to protect.
publish() now aborts when the pre-publish save failed. It ran regardless, so
the author saw "Published version N" while readers got the previous revision.
Link rendering: text is escaped before formatting, so a captured URL arrives
carrying entities and was escaped a second time - every multi-parameter
ServiceNow and SharePoint link opened with &amp; in it. Emphasis could also
reach inside an href, and '//host' passed as an in-app link when the browser
reads it as off-site.
Tests are now HTTP-level. The draft-invisibility rule lived entirely in routes
that no test executed, which is how three of them shipped serving drafts.
Five of the seven asset types had no way to take a record out of ShopDB from
the interface. Network devices and 3D printed parts had a button; machines,
PCs, printers and measuring tools did not, so a scrapped tool stayed in every
list and report until somebody went to the API.
The endpoints were already there and already correct. Every one of these
DELETEs sets asset.isactive = False and nothing more, so the record leaves the
lists while its calibration history, backups and relationships stay exactly
where they were.
Called "Retire" rather than "Delete" for that reason: the two existing buttons
say Delete, which reads as destruction of a record that is in fact kept and can
be restored by setting isactive back. The confirm says so too, because the
question somebody needs answered before clicking is whether the history goes
with it.
The button is guarded on the record having loaded, on the two pages whose
action row renders before the fetch resolves - measuring tools showed Retire
beside a hidden Edit while the tool was still loading, which would have called
delete with an undefined id.
printersApi had no delete method at all; added.
USB is deliberately left out. It retires through a status column in the
external CMMC database rather than isactive, has its own /retire endpoint, and
needs its own decision about what retiring means there.
Every detail page read "West Jefferson ShopDB - Machine". Two tabs on two
different machines were indistinguishable, a bookmark said nothing about what
it pointed at, and a link shared into Teams previewed as the section rather
than the thing.
The router cannot fix this alone: afterEach runs before the view has fetched
anything, so it only ever knows the KIND of page. So a detail view names its
record once the record arrives, and both writers go through
composables/pageTitle.js.
THE ORDERING IS THE WHOLE PROBLEM. The router awaits the site settings to build
the "<facility> ShopDB" prefix, so on a cold load it finishes AFTER a view that
fetched quickly and would overwrite the good title with the generic one. The
route's fullPath is claimed before that await, and the generic write stands
down when a record title already belongs to the route being landed on. The
reverse case is handled too: a view that names its record BEFORE the facility
name arrives composed against the placeholder prefix, so the late resolve
rewrites it rather than leaving "ShopDB - CSF16" on a site called West
Jefferson. Both are pinned by tests, because both fail only under timing.
Covers machines, PCs, printers, network devices, subnets, measuring tools, USB
devices, printed parts, applications, employees, knowledge base articles and
guides. Printers reuse the page's own displayTitle, which already discards the
literal name "NONE" a discovered queue can carry, so the tab and the heading
agree.
A record with no usable name, or one that failed to load, keeps the section
name: "Machine" beats "Machine 4192" for somebody scanning a row of tabs.
Long names are cut at a word boundary - knowledge base articles are named with
a whole sentence of description, which a browser otherwise truncates mid-word.
Reports gained meta.title, which the router already supported and nothing used.
That also fixes "Pc Relationships", which the name-derived spelling produced.
Two faults the guides plugin shipped with, both caught by gates that were
already there.
deploy/site-profile-universal.json never listed `guides`. That file is the
profile the released Windows installer is built from, so the wizard would not
have offered Guides at all - and worse, `flask plugin prune-schema` drops the
tables of any plugin absent from the profile at provisioning (ADR-014). A site
installing the next release would have had its guides table dropped.
plugins/guides/models/guide.py imported shopdb.core.models.base directly.
Plugins reach core only through shopdb.api or shopdb.plugins.base (ADR-001),
and shopdb.api already re-exports BaseModel, so this is the same symbol by the
supported path.
The suite was red on main because of these two; it passes again.
Global search has per-domain toggles (search_<type>_enabled), enforced
server-side and edited from Settings. Guides answered the search box but had no
toggle, so it was the one domain nobody could turn off - and the omission would
only have surfaced when somebody went looking for the switch and could not find
it.
Missing key still means enabled, so nothing changes until somebody decides
otherwise.
The repo already enforces that every served route appears in
docs/api-inventory.json, and the guides endpoints did not - so the parity test
was failing. They are now listed with auth, params and a working example each,
beside knowledgebase because the two get read for the same reason.
docs/GUIDES.md covers what the inventory cannot: the block schema and why it is
blocks rather than HTML, the id rule that makes a tick rememberable, what the
server refuses and why validation happens at the door, the working-copy /
revision / published-pointer split that stops a live procedure changing under
somebody following it, and a worked example of creating and publishing a guide
from a script.
It also states plainly what does not exist yet - no importer, orphaned images
never pruned - and what converting existing documents will actually be like:
structured HTML and Markdown map onto block types nearly one for one, while a
plain-text runbook lands as paragraphs and needs marking up by hand afterwards.
Generated ids have to stay stable across re-imports, or every re-import resets
readers' progress.
Guides moves to position 8 in the sidebar, beside Slides, rather than down among
the reference lists in Information. A guide is something somebody goes and does
something with, not a catalogue they look things up in.
A guide was invisible to global search. Core search has a hardcoded block per
domain rather than a hook, so a new plugin stays unreachable until someone adds
one - which is the same shape as the gap this test file already documents for
USB devices and printed parts: records with a list page, labels in the world,
and no way to find them from the search box.
Guides are searched by title, summary, tags AND body. The body matters more here
than elsewhere: people look for a procedure by something written halfway down it,
not by a title they never read. PUBLISHED ONLY - a draft is deliberately
invisible to readers, and search is exactly where that rule leaks if nobody
writes it down. The test proves it with a draft sharing a tag and a body word
with a published guide, so a missing filter would show up as a ride-along.
PROOFING catches what a spellchecker cannot, because none of these are spelling:
a smart quote in a command so it will not run, a checklist item with no text so
a step is silently missing, a heading left with nothing under it after a
reorganise, an image with no alt text, a TODO published by accident. Warnings,
never blocks - somebody publishing at 14:40 with one empty bullet should be
told, not stopped, and a procedure that exists imperfectly beats one nobody
wrote.
The smart-character issue reports once per block with a one-click fix, rather
than once per offending character: an opening and a closing quote are one
problem, and listing both makes the panel look like it is nagging.
The character table itself is written as escapes. The style gate forbids
non-ASCII in source, and a file whose job is detecting smart quotes cannot
contain any - which the gate caught, correctly.
Seven changes, all aimed at the same thing: think about the procedure, not
about blocks.
Live preview beside the editor, rendered with the SAME component readers get,
so what an author sees is what will be published - a separate preview
implementation would drift within a month.
Formatting buttons wrap the selection, so nobody needs to know the markup is
Markdown. New blocks and pasted images land AFTER the block being worked on
rather than at the bottom of the document, which removes the scroll-back that
made adding a screenshot mid-guide annoying.
Enter in a list makes the next item and focuses it, the way every other list in
every other tool behaves; backspace on an already-empty item removes it and
steps back up. Typing a ten-item checklist no longer needs the mouse.
Templates, because the blank page is the real barrier to somebody writing a
procedure down at all. The shapes are lifted from the KB articles the team
already writes, so guides written here read like the ones written before them.
Autosave on a three-second debounce with an honest status line, safe precisely
because the server dedups on content - an autosave that changed nothing writes
no version. Leaving with unsaved work now asks first.
CLIPBOARD: the code block no longer writes to the clipboard. A page that
programmatically copies a shell command is the exact shape of a ClickFix attack,
where malware plants a command in your clipboard and talks you into pasting it
into Run - and uBlock Origin blocked ours, correctly, because it was writing
PowerShell on click. The button now SELECTS the code and says to press Ctrl+C:
same convenience, decision back with the person, and nothing for security
tooling to object to.
AUTOCORRECT is on for prose and off for code and table cells. A code block that
autocorrects produces a guide telling somebody to run a command that does not
work - smart quotes replacing a straight one, an autocapitalised cmdlet - and it
still LOOKS right to the author. The person at the bay gets the syntax error.
Printing a dark-mode page lays a full page of toner down for the background and
knocks the text out of it in white. On a shopfloor printer that is a cartridge
for a handful of guides, and the result is harder to read than the plain
version. Somebody would have found this the expensive way.
The theme tokens are redefined for print only, in an unscoped block because
overriding them has to reach :root. Every guide element already renders through
those variables rather than literal colours, so this is a dozen lines instead of
a rewrite - the same discipline that made dark mode work without dark-mode code.
Backgrounds are dropped rather than printed grey: a callout is identified by its
rule and its heading, which cost nothing. Checkboxes print EMPTY even where they
are ticked on screen, because a printed guide is a worksheet somebody carries to
the bay, and one person's progress is not part of the procedure. The footer
gains the print date beside the version, so a stale copy pinned to a noticeboard
can be spotted.
Also corrects the model and migration found by running this against a real
database rather than SQLite: BaseModel contributes createddate, modifieddate and
isactive to every table and the migration never created them, so the first read
failed on unknown columns; the model had also declared duplicate timestamps on
top, now removed with the API field names kept stable. And the slug is 191
characters, not 300 - it is UNIQUE-indexed, MySQL caps an index key at 767
bytes, and utf8mb4 is 4 bytes per character.
The knowledgebase plugin is a link registry - a description, a URL and some
keywords pointing at a document held somewhere else. That leaves the documents
themselves as HTML files on a share: no search, no history, no way to know
whether the copy pinned to a noticeboard is current, and no way for a technician
to tick off where they are.
A guide is stored as an ordered list of typed blocks rather than a blob of
HTML, because three of the four things this feature exists for depend on it. A
tickable step needs a stable id to be remembered; dark mode needs semantic
markup rather than pasted colours; and print needs to know where a step begins
so it does not split one across a page. Free HTML gives none of that and lets
pasted styling fight the theme.
Inline text is a restricted Markdown subset - bold, italic, inline code, links,
line breaks - escaped BEFORE formatting, so nothing an author pasted can reach
the DOM as markup. No fonts, sizes or colours: emphasis is semantic and the
theme decides how it looks. That is what makes a guide written next year match
one written today, and dark mode work without an author thinking about it.
DRAFTS AND VERSIONS. The guide row is the working copy; every save that changes
something writes an immutable revision; and a separate pointer says which
revision READERS get. Without that pointer, editing a live procedure changes it
under a technician who is halfway through following it. Publishing promotes an
existing revision rather than snapshotting, so a publish with unsaved edits
cannot ship a version nobody reviewed. Restoring writes a new revision with old
content, so the record of what people were told in between survives.
Saves dedup on a content hash covering title, summary, tags and blocks, so an
autosave that changed nothing writes no version - otherwise the history fills
with identical entries and hides the three edits that mattered.
Progress lives in the browser, not the database: shopfloor PCs share a login, so
per-user ticks would show one technician's progress to the next person on that
bay, which reads as "already done". Keyed by guide and version, because step 4
of version 3 is not step 4 of version 2.
Five permissions, and the split between edit and publish is the point: a
technician can write a procedure up while somebody else decides it is fit for
the floor. create-without-edit works too - an author may edit their own guide
until it is published, after which other people are following it.
CI caught this: judging a host by its stalest kind treated a never-confirmed
revision (lastseenat NULL) as the worst case, so it won. The verdict for a NULL
confirmation is tri-state None, which renders as NO BADGE - so a host with a
healthy ntlars backup and one backfilled row showed nothing at all. Hiding a
working backup is the one thing that badge must never do.
A NULL confirmation is not stale, it is unmeasurable. The stalest kind is now
chosen among kinds that can actually be judged, which is the rule staleness.py
already applies: a chain becomes measurable the first time its PC posts and says
nothing until then. If every kind is unmeasurable the host still reports which
kind it has, with a None verdict and no badge.
The test that caught it pinned the old newest-wins behaviour, which part markers
made wrong; it now pins stalest-measurable-wins and keeps its NULL clause, which
was right all along and is what found the bug.
Also worth recording how it was missed: the change was verified against
test_geenforce_reporting.py and test_backups.py, and this assertion lives in
test_geenforce_publish_gate.py, which was never run.
The publication scrub gate rejects internal references, and a real site
hostname in a test is still one in the published repo. The rewrite does not
care which server it is.
Snapshots written before the client was fixed stored "W:\backup\partmarker\..."
as their location. That mapping exists only inside the SYSTEM logon session,
only during an enforcement cycle, and only on the bay that wrote it, so the path
recorded against a marker resolved to nothing for the person most likely to open
it.
The script rewrites backuprevisions.sharepath and nothing else - no revision is
created, deleted, re-filed or re-hashed, and no file on the share moves, since
the bytes are already where the UNC will point. Dry run by default.
It matches on the drive letter alone and leaves everything else untouched,
including paths that are already UNC and paths on another drive. This repairs a
specific known mistake rather than normalising whatever it can reach, which is
the version of this script that would eventually rewrite something it should
not have.
rewrite() is kept pure so the decision of WHICH paths to touch is testable
without an app context; the tests cover the already-UNC, other-drive, empty,
case-difference and trailing-separator cases.
The plugin shipped five settings and no page for any of them. The core settings
catalog is a hardcoded list a plugin cannot extend, so backups_shareroot and the
collection interval were reachable only through the API or a flask shell on the
server - which is not a configurable setting in any useful sense, and it is how
today went: the setting simply was not there to set.
The page is contributed the ADR-010 way rather than by editing core - a card
from get_settings_cards plus a route in the plugin's own routes.js, both of
which disappear when the plugin is disabled. Filed under Machines, next to
Machine Types, because that is where the assets these back up live.
It also states the two things the settings do not say for themselves: that
health is judged on when a backup was last CONFIRMED rather than when it last
changed, and that retention prunes database revisions only - files on the share
stay, so enabling it on a share-backed kind leaves orphaned snapshot
directories.
Snapshot paths are now shown as full UNC text in both places: a Snapshot row on
the Marker Contents card, and the existing path on the history page, which
gains a copy button. Deliberately NOT a link - browsers refuse to navigate from
an https page to file://, so a clickable path looks actionable and silently
does nothing, which is worse than no link. Copy, then paste into Explorer.
The path comes off the revision, never composed from the configured share root:
a snapshot written before the layout changed still lives where it was put.
Until part markers, a PC reported exactly one backup kind and "newest revision
wins" was the same thing as the truth. A marker bay reports two: ntlars for the
machine's DNC settings, partmarker for the controller's patterns.
With newest-wins, the daily ntlars post keeps the badge green while the marker
backup has been failing for a month. The failure shows only on that marker's
asset page, which is not where anyone looks to find out something stopped - and
"is this still being checked" is the one question this badge exists to answer.
So a host is green only when everything it backs up is current, and the badge
names the kind that is not. Freshest revision per kind first, because
lastseenat only ever moves on the newest revision of a chain; judging every row
would report a healthy, frequently changing config as permanently stale.
Known consequence, and the right direction to err: a device retired from a PC
leaves its last revision behind, so that host stays stale until the revision is
pruned. It complains about a backup that genuinely is not running rather than
staying quiet about one.
The dashboard card needed no change - it already lists every stale chain
independently, so a stale partmarker chain was never hidden there.
A 'share' kind keeps its bytes on the file share and gives ShopDB only a hash
and a UNC path. That is right for an opaque vendor file, but it means the app
cannot answer the question a tech actually arrives with - what does this part
marker hold, and which pattern disappeared last month. Reading the share from
the app server is not an option: the download routes refuse to mount it on
purpose, because a permissions slip there becomes an unexplained empty panel.
So the collector now accepts an optional manifest alongside the hash: the
index, not the content. Names, sizes, per-file hashes, and the marking system
each pattern was authored for. Advisory throughout - a client that sends none
behaves exactly as before, and dedup still keys on contenthash.
The marker asset gains a Marker Contents card built from it: counts by type,
the controller family, and a diff against the previous snapshot in the same
source chain. Removed files are named rather than counted, because "1 removed"
only tells someone to go and look. Files edited in place are named too - the
file set can be identical while the content hash moved, and reporting no
change there would contradict the reason the revision exists.
The manifest is deliberately NOT covered by contenthash, which attests the
bytes on the share. A client that starts describing itself therefore posts an
unchanged hash, so a no-op backfills a manifest the row does not yet have -
otherwise the card would appear only on markers that had recently changed, and
stay hidden on every healthy one. The same no-op refreshes sharepath, so a
snapshot that moved on the share stops being pointed at where it used to be.
Manifests are validated at ingest rather than at render: a malformed one would
otherwise become the latest revision and fail every view of the asset page
until the content next changed, outliving the client bug that caused it.
sharedir() for the marker kind is overridden to the layout the site actually
uses, so what ShopDB displays and validates matches what the collector writes.
Canonical names chosen by the site: Dell Inc., HP, Zebra. The rule is the same
in each case - keep the spelling that holds the entries - it just lands on a
different form per company. Dell Inc. carries 310 references and 33 models
against 1 apiece for Dell and DellInc.; HP carries 51 and 24 while HP Inc. has
none; Zebra carries 7 and 2 while Zebra Technologies has none.
So HP Inc. and Zebra Technologies are gone from the shipped catalog. Both were
empty rows that only ever offered a second way to spell a vendor already
present, and neither exists in this site's database at all - they would have
arrived as duplicates at a NEW site and nowhere else.
Also removed UnscopedLiveVendor, which is a test artifact. It reached the
catalog because export_catalog.py reads the live development database, so a row
left behind by a lab run was published as reference data and has shipped to
every site seeded since.
Vendors 51 to 48. Verified nothing referenced any of the three before removing
them, and the catalog still has no duplicates by any spelling.
The seed catalog is what a new site inherits, so its duplicates propagate.
Removed the four space-stripped twins it carried: vendor 'DellInc.', model
'OptiPlex7080', and operating systems 'MicrosoftWindows10Enterprise' and
'Windows10', each of which had a correctly spaced original already present.
'OptiPlex5000' was NOT a duplicate - no spaced twin existed - so it was
renamed to 'OptiPlex 5000' rather than dropped, matching every sibling in the
list. Its vendor moved to 'Dell Inc.', which this site treats as the one
canonical Dell.
Vendors 53 to 51, models 128 to 127, operating systems 14 to 12. Verified
afterwards: no exact duplicate rows, no near-duplicates, no orphaned
model-to-vendor or supply-to-model reference.
The legacy data carried space-stripped twins: 'DellInc.' beside 'Dell Inc.',
'Windows10' beside 'Windows 10', 'OptiPlex7080' beside 'OptiPlex 7080'. They
rode into the shipped seed catalog, so every site that ran `seed catalog`
inherited them, and they stay invisible until somebody opens a vendor picker
and finds the same company listed twice.
Not a code fault. The collector strips only leading and trailing whitespace,
matches with ilike, and treats the OS list as lookup-only, so it can neither
have created these nor be creating more. They arrived with the classic-ASP
import, which holds the same three pairs today.
`flask catalog dedupe` merges only whitespace-and-punctuation twins, because
those are one name typed two ways. 'Dell' and 'Dell Inc.' normalise apart and
are left alone: that is a naming choice and only a person can make it. The
surviving row is the most referenced, tie-broken toward the spaced spelling,
which is the one somebody pressed the space bar for.
`flask catalog merge` makes that judgment call explicitly, and refuses to
invent a target that does not exist, so a typo cannot move rows into a new
vendor. `flask catalog rename` handles the case neither can reach: a stripped
name with NO twin, where there is nothing to merge into.
All three report and change nothing unless --apply is passed, and dedupe is
idempotent so it is safe in a deploy script.
References are discovered from the LIVE schema rather than listed in code.
There are 15 foreign keys into these three tables across 7 tables, and
machines.controllervendorid and printerdrivers.modelnumberid are not ones
anybody would enumerate by hand. Every reference is repointed before the row
is deleted, so nothing is orphaned, and a plugin adding a table is covered
without editing this command.
Running `python scripts/export_catalog.py` on a deployed site died with
1045 "Access denied for user 'root'@'localhost'", naming a user that site
does not have.
Nothing loads .env for a plain script run. wsgi.py calls load_dotenv() for the
served app and the flask CLI loads .env for itself, so both reach the real
database; a bare `python scripts/...` gets neither. DATABASE_URL then stays
unset, FLASK_ENV defaults to development, and config.py falls back to a
hardcoded root@localhost URL.
The read-only failure above is the harmless version. Six of these seven
scripts WRITE - backfill, reclassify, retype, collapse, refile, import - and
on any box where those fallback root credentials happen to resolve, they would
have silently rewritten the wrong database with no error at all. That is also
why this survived: the dev .env points at root, so the fallback matched what
the developer expected and nothing ever looked wrong here.
All seven now load .env from the repo root, resolved from __file__ rather than
the working directory, since these are run by hand during a deployment and
rarely from the repo root.
tests/test_maintenance_scripts.py pins it for any future script: every script
that calls create_app must call load_dotenv, before create_app rather than
after it, and with an explicit path rather than a cwd-relative search.
PLUGINS.md listed 14 of the 15 bundled plugins. `backups` had no row: the
per-asset configuration backups the collector takes off shopfloor machines,
kind-pluggable, with view and download as separate permissions because a .reg
is a restorable artifact. Added, with the pointer that separates it from
BACKUP-RESTORE.md, which is the database and a different subject.
PLUGIN-LAB-PRINTEDPARTS said the finished implementation lives on branch
feat/printedparts-plugin. That branch was deleted in the v0.8.0 cleanup; only
the 17 lab-stage tags survived, and they are enough. Says so now, and warns
that the tags are a record of contract 0.11-0.13 rather than today's
conventions. Its manifest example carried core_version >=0.12.0 against a
shipped >=0.16.0.
The printer-assignment proposal still said the client script is not built. It
shipped and released, across v0.11.3 and v0.12.0, and the collector half that
contradicted the assigned-vs-observed split was retired in adc7ef4.
PROJECT-MAP regenerated for the test count.
`flask plugin new` wrote views to frontend/src/views/<name>/ and a route file
to frontend/src/router/routes/<name>.js. Every one of the 16 bundled plugins
keeps its frontend in plugins/<name>/frontend/ instead, staged into
.plugins-staged by stage-frontend.mjs and aggregated into routes.gen.js.
It was not visibly broken, which is why it lasted: the router still globs
./routes/*.js during the Phase 4 dual-location transition, so a scaffolded
plugin's pages loaded. What a new plugin author silently lost was ADR-013 lean
builds. Pruning works off the staged tree, so a frontend in the core tree
cannot be excluded from a site build, and deleting the plugin strands its
views behind.
The templates for the correct layout already existed; only the output paths
were wrong. Also fixed the imports they carry: core imports now use the @
alias (@/api, @/components/...), which resolves both before and after staging,
where the previous ../../api was correct only in the old location. Sibling
views stay relative and travel with the route file.
frontend_dir is gone from scaffold_plugin. There is no second output tree to
point at, and a plugin developed in its own repo now gets the same frontend as
a bundled one rather than being skipped.
Proven end to end: scaffolded a throwaway plugin, confirmed it stages, builds
real chunks, and that SITE_PLUGINS=network prunes it, which is what the old
layout could not do.
A NOTICE result linked to /notifications, so clicking the row search had
already identified dropped the reader on the list page to find it again by
hand. The id was in the payload the whole time; only the URL ignored it.
`/notifications/<id>` is the existing notification-detail route.
The subnet searcher had the same defect two functions away: an IP search
resolves which subnet the address falls inside, then linked to /network, the
hub. It now links to /networks/<id>, the subnet detail route.
Every other search result type already carried its id. These were the only
two that did not.
The printer feature rests on one rule: a collector report is OBSERVED state and
never ASSIGNED state. If a drifted bay's own state can become what it is told to
install, enforcement stops meaning anything, because a bay that installed the
wrong printer makes itself right simply by reporting it.
_sync_printer_links broke that rule. It ran on every collector PC push and, when
the payload carried defaultprinter or printers, wrote defaultprinter and
connectedto rows from the report - defaultprinter being one of the two types
_reconcile_edges is meant to own alone, from authenticated routes a human calls.
It was recorded as dormant. It was not. The collector schema ADVERTISED both
keys and the example client in COLLECTOR-INTEGRATION.md set
$payload['defaultprinter'], so the documentation was recruiting callers for it -
while the same document, under "Observed is not assigned", promised that nothing
on this path writes a usesprinter or defaultprinter row. The document
contradicted itself and the code followed the wrong half. Nothing in the fleet
armed it, because the first-party reporter never sent those fields, but any
third-party client written from the docs would have.
Retired the function, its _sync_one helper and the now-unused
PRINTER_LINK_ORIGIN label, and removed both keys from the schema.
IGNORED, NOT REJECTED. Unknown payload keys are not validated, so a client
written against the old docs keeps pushing successfully and only its printer
keys stop doing anything. The response warns, naming the keys and pointing at
POST /api/collector/printers for reporting queues and seed-from-observed for
adopting them. Dropping them in silence would leave an integrator believing
assignments were still being written, which is a worse failure than the one
being fixed.
The docs were the larger half of this. The schema table rows are gone, the
"PC -> printer relationship sync" section is replaced by a note saying what
happens now to anyone still sending the keys, and the example client no longer
enumerates Win32_Printer at all - it points at Report-PrintersToShopDB.ps1,
which posts to the observed endpoint and posts NOTHING on a failed enumeration,
since an empty list would erase the host's rows.
test_collector_printers.py pinned the old behaviour, so it now pins the
opposite: the keys create no relationships, the push still succeeds and says
why, a payload without them stays quiet, a human-made default survives a report
naming a different printer, and the schema no longer offers the keys.
Reported: the warranty page cannot be searched by machine number.
It was there to be found. _related_machine puts the machine's number in the
payload, and it does so for a stated reason - a shopfloor PC is bought,
warranted and replaced as a PC, but it is FOUND by the machine it drives, and
nobody walks the floor looking for an asset number. The table renders it as a
chip. The filter looked at vendor, service level, service tag and asset number,
and never at the machine. So the one identifier the feature was designed around
was the one thing the search box could not match.
This is how a field list goes wrong: a column was added to the table and not to
the filter, and nothing connects the two. Moved the field list into
warrantySearch.js so a test can see it, following controllingPc.js and
labelPages.js. Machine name is searched as well - somebody who knows the bay by
"the Makino" should not have to look up its number to find its warranty.
The specs also pin the case that made the old inline version fragile: a covered
asset whose machine is null, which is every printer and any PC nobody has linked
yet, must not throw. And a missing field must not match: String(undefined) is
'undefined', so a naive coercion makes a search for "und" hit every row with an
empty vendor.
Not changed, worth knowing: the covered asset's NAME is still not searched, only
its number. That looks like the same omission, but it is a wider change than the
report - the placeholder has always said "asset" without saying which half - so
it is left for a deliberate decision rather than folded in here.
Reported: `flask plugin upgrade-all` failed the tools plugin with "table
securesendsecrets already exists".
Every post-cutover baseline emitted a plain op.create_table. That is correct
only while a table's existence and its plugin's Alembic stamp can never
disagree, and they can. Anything that builds tables from the models -
db.create_all(), which `flask db-utils create-all` calls - creates them and
stamps nothing. A run that creates a table and then fails before its stamp
commits leaves the same split, and re-running is the obvious thing to try. In
both cases upgrade-all meets existing tables with an unstamped chain and the
create is fatal.
Nothing caught it because the migration test DROPS the post-cutover tables
db.create_all() built before running upgrade-all. That arranges the one state
where an unguarded create works, and never the state the report came from. The
new test is the same sequence with the drop removed for the plugin under test.
create_table_if_missing and create_index_if_missing are in alembic_template.py
next to create_plugin_tables, which had the idempotency rule right and turns out
to be called by nobody - its docstring claimed every 0001 used it. Fixed the
docstring rather than rewiring the baselines onto it, because the reason they
hand-write their DDL still stands: the helper's per-plugin MetaData cannot
resolve a foreign key to a core table at CreateTable-compile time.
INDEXES ARE MATCHED ON COLUMNS, NOT NAME, and that distinction matters. The two
creators disagree about names - db.create_all() takes SQLAlchemy's default
convention (ix_securesendsecrets_token) and a baseline spells it the way this
repo's naming rules require (ixsecuresendsecretstoken). A name-keyed guard would
have called the index missing and added a second one over the same column, which
MySQL permits with a warning, so every affected site would have quietly grown
duplicates. The test asserts no table ends up with two indexes over the same
columns.
Applied to all nine table-creating plugin migrations, not just the tools
baseline that was reported: the four later migrations that create tables have
the same exposure, and geenforce, measuringtools, printedparts and backups are
the same defect waiting for a different site.
WHAT THIS DOES NOT FIX. It makes a baseline idempotent, not a whole chain. A
longer chain can still meet an ALTER whose column is already present -
printedparts adding gagelabtag to a table create_all built at head is the case
that showed up while testing. Guarding every op one at a time is the wrong
answer to that; a database built from the models is already AT head and its
chains want stamping rather than replaying, and there is no `flask plugin stamp`
today. Left alone deliberately rather than half-built, and the new test is
scoped to single-baseline plugins so it does not pretend otherwise.
Reported: moving machine 3016 on the map and saving answered "this
relationship already exists" and refused the save.
The map had nothing to do with it. Every save of the machine form re-POSTed the
controlling-PC link, and the server was right to refuse it - the row was already
active, and assetrelationships is unique on (source, target, type). Any edit to
a machine that has a controlling PC hit this; a position change is just the one
somebody happened to be making.
The form asked the wrong question. It used "do I know the existing link's id" as
a stand-in for "is there an existing link", and read that id from
rel.assetrelationshipid - a key the API has never emitted. to_dict serialises
column names and the column is relationshipid, so the read was undefined on
every load while controllingPcId was populated from the same relationship. The
two answers disagreed, and the disagreement was invisible: undefined is not an
error, it is just a falsy id.
That single conflation caused both failures. Saving anything re-created the
link, and CLEARING the controlling PC did nothing at all, because "no selection
and no known id" reads as "nothing to do" - so removing a machine's controlling
PC silently kept it. Only the first of those was reported, and the second is the
one somebody would have trusted.
Fixed by making the LOADED PC the authority on whether a link exists, with the
relationship id used only to delete a row. If that id ever goes missing again
the worst case is a link that cannot be removed, never a duplicate create.
The decision moved to controllingPc.js as a pure function so it could be pinned
by tests, following labelPages.js. Unchanged is now explicitly no-op rather than
delete-then-create: re-linking a machine to the same PC opens a window where it
has no controlling PC at all, and resolve_asset_position walks that edge, so a
failure between the two calls would drop the machine's inherited position along
with its link. The specs also cover a select handing back '42' against an API
that gave 42, which would make an untouched link look like a swap and reach the
same duplicate POST by another route.
Not the printer assignment, which was the first guess: PrinterAssignmentPicker
makes no relationship calls, and its save is wrapped in a catch that only logs,
so it cannot surface an error from this form at all.
Secure Send shipped on 21 August and nothing outside the code caught up. The
roadmap still listed it under Deferred as "tabled", and gave as the reason that
the tools plugin is browser-only by design with get_blueprint and get_models
both empty - which stopped being true in the same day's commits that gave it a
blueprint, a table, a migration chain, a permission and a setting. A reader
following the roadmap would have concluded the feature does not exist.
The generated API docs were fine: api-inventory.json and openapi.json both
carry /api/tools/secrets, so the contract gates did their job. What was missing
was every word written for a person, and that is the half where the security
properties live.
docs/SECURE-SEND.md is the new page. The parts worth having in prose rather
than inferring from the code: that a lost secret is lost to everybody including
an administrator, which is the same property as "a compromised database yields
nothing" seen from the other side; that the 4096-character limit is the entire
server-side enforcement, because a server that cannot read what it holds cannot
have an opinion about content, so raising it turns a password tool into an
unaudited file transfer; that redeeming is a POST because preview prefetch in
chat and mail clients would otherwise spend a one-view secret before the
recipient ever sees it; and that somebody should send a link to themselves
through their own mail or chat client and confirm the key survives after the #
before a real credential goes through it, because a product that rewrites links
can strip the fragment and nothing on this side can help.
securesend_allow_guests was in the database, on a settings page, and in no
document. It is now in CONFIG.md with what it actually does - an unauthenticated
write endpoint, rate limited per address, failing closed - rather than only its
one-line description.
Also corrected along the way: site_base_url was documented as scheme + host
while the code deliberately tolerates a subpath in it, and the bundled plugin
table in PLUGINS.md had no row for tools at all.
Still missing after this: PLUGINS.md has no row for backups either.
Reported: no Tools section in Settings. The setting existed in the database and
had no page, which from a person's side is indistinguishable from the feature
not existing.
get_settings_defaults SEEDS a setting. get_settings_cards (ADR-010) is what puts
it in the rail. Declaring only the first is a silent half-feature: nothing fails,
no test notices, and the switch is unreachable.
The page states the cost where the DECISION is made rather than in a document
nobody opens - turn it on and it says a guest-created link records no creator,
so the audit trail can say when and from where but never who.
Admin-only, like every other settings page, because this one decides whether an
unauthenticated caller may write to the site.
Also caught while wiring it: the card asked for a 'shield' icon, which is not in
the settings iconMap, so it would have rendered the fallback puzzle piece - the
glyph that reads as "nobody chose this". Uses wrench, matching Tech Tools.
The row still comes from `flask plugin upgrade-all`; the card appears either
way, and without the row the page loads empty.
/tools, /tools/codes and /tools/secure-send were all behind requiresAuth. None
of them needed to be, and one of them actively contradicted a setting added an
hour earlier: securesend_allow_guests lets somebody with no account create a
link, and the page they would create it on was behind a login wall.
The label generator runs entirely in the browser - no API, no data, nothing to
protect - so gating it only ever cost a guest the use of it.
THE SERVER STILL DECIDES who may create a secret. A signed-in user needs
tools.secrets; a guest needs the site to have opened guest creation. Making the
page reachable is not the same as making the action allowed, and the two were
conflated.
The page now reads the setting on load, so a guest at a site that has NOT opened
guest creation is told to sign in instead of being handed a form that was always
going to be refused - with a line saying that receiving a link never needs an
account, because somebody who lands there confused usually has a link they
should be opening instead.
A signed-in user always sees the form, even without the permission: the server's
"you do not have permission" is a different answer from "sign in", and
collapsing them would send somebody to a login page they are already past.
Two asks, and the first is a real security decision so it is a SETTING that
defaults to the safe answer rather than a code change somebody inherits.
securesend_allow_guests, OFF by default. With it off the model is unchanged: a
login plus tools.secrets. With it on, somebody with no account - at a shared bay
terminal, a contractor - can create a link too.
The cost is named in the setting's own description and pinned by a test: a guest
secret has NO CREATOR, so the audit trail can say when and from where but never
who. That is the whole reason it is a switch a person turns on deliberately.
IT FAILS CLOSED. A missing row, a half-migrated site, an unhappy database - the
answer is no. An error must never be the thing that opens an unauthenticated
write endpoint.
A SIGNED-IN USER STILL NEEDS THE PERMISSION even when guests are allowed, which
is surprising enough that a test pins it: opening the door to guests is a
decision about the door, not about revoking what a role holds.
Guest creation is rate limited per address, same shape as the login limiter. Not
a security boundary - a determined person has more addresses - but an open
endpoint should not be an easy place to park data or fill a disk.
VIEWS ARE NOW A NUMBER, 1 to 100, instead of a four-item dropdown. How many
people need a password is the sender's business: a shift handover is not one
colleague on a call. The floor is what matters, not the ceiling - unlimited
views would just be a password on a web page.
Also folded in, from the retention question: nothing removed a secret that
EXPIRED WITHOUT BEING OPENED. The recipient goes on holiday, the link is never
clicked, and the ciphertext sat there for good - not readable, but it made
"nothing lingers" untrue and grew the table in exactly the case where the
feature failed. Any create now sweeps expired rows and a late click tidies its
own, so it stays bounded without a scheduler an air-gapped site would never add.
Two gates caught mistakes on the way: current_user is a LocalProxy and is never
None, so `current_user is None` silently took the signed-in branch for a guest;
and cache was imported straight from shopdb.extensions when it is already on the
shopdb.api contract surface.
1836 tests green.
Answering "did I set it wrong": no. The setting was right and the description
was misleading, which is what sent the last two commits chasing the wrong fix.
It said "Public base URL of this site (scheme + host), e.g.
https://shopdb.example.net". That is true only of a site mounted at the root.
Both existing consumers append the in-app route to this value and add nothing
themselves:
`${await getSiteBaseUrl()}${config.detailPath(detailId)}` asset labels
`${baseUrl}${defaultPath}` QR codes
so a site served at /shopdb must carry that here or every QR code it prints
404s. The email copy of the same setting has always documented it correctly, as
https://host/ops.
The description now says so, and the comment above the defaults records why -
including that a subpath left out produces links that fail rather than anything
that looks like a configuration error.
Existing rows keep the old text until a site reseeds; the value is unaffected.
a381174 fixed a link missing its subpath and produced one carrying it twice:
https://<host>/shopdb/shopdb/s/<token>.
site_base_url is DOCUMENTED as scheme + host, and sites really do set the full
base including the subpath. Trusting the description rather than tolerating what
is configured is the same mistake as the last one, pointed the other way.
The mount is now appended only when the configured URL does not already end with
it, which covers every combination a site can have: base with the subpath, base
without it, a trailing slash on either, and a site mounted at the root where
there is no subpath at all.
Checked all five by evaluating the expression rather than reasoning about it.
Reported from production: clicking a secret link gave an IIS 404, and the link
read /s/<token> where the site lives at /shopdb/s/<token>.
The link was assembled from window.location.origin, which is scheme and host
only. site_base_url already exists for this and says so in its own description -
"used for QR codes and any absolute link the app emits" - with getSiteBaseUrl()
as the helper the QR pages use. This reached past all of it.
Both halves were wrong, and the second is worse because it fails silently:
MOUNT PATH. A site served under /shopdb or /ops carries that in front of
every route. Leaving it off is the reported 404.
HOST. Whoever creates a secret may have reached the app by an IP, a short
hostname, or a name that resolves in one VLAN, and the recipient opens the
link from somewhere else entirely. The configured site URL is the one address
known to work for both of them.
The shape of this failure is what makes it worth spelling out: the secret was
created SUCCESSFULLY and the sender was handed a link that could never work, so
the failure landed on the recipient, the sender had no reason to doubt it, and
the secret expired unread. A 500 would have been kinder.
Falls back to the browsing origin when a site has not set site_base_url, which
is what every other absolute link here does.
BASE_URL is injected at build time, so the subpath half of this cannot show up
in dev and only appears on a subpath deployment.
The backend has been there since d4b49b9 with nothing to click. This is the
half a person actually touches.
A card under a new "Secrets & Access" heading on Tech Tools, which is where
somebody looks for a utility - the whole reason the plugin moved there.
THE CREATE PAGE encrypts before anything leaves the browser and shows the link
once. The plaintext is dropped from memory as soon as it is sealed, and the page
says plainly that the key is in the link and exists nowhere else, because
somebody who does not know that will paste the link somewhere durable.
THE READ PAGE IS PUBLIC, at /s/<token>, outside the app shell: the recipient
usually has no account here, so it carries no navigation and asks for no login.
Short path because the whole link gets pasted into a chat message.
IT NEVER REVEALS ON LOAD. It asks whether the link is still live, then says
"opening this destroys it" and waits for a click. A page that revealed
automatically would be burned by the first scanner that fetched it - which is
the same reason redeeming is a POST.
A MISSING FRAGMENT GETS ITS OWN MESSAGE. If a link rewriter strips the part
after the #, the recipient is told the link is missing its key and what to ask
for, rather than being shown a generic failure. That is the one failure mode
this design cannot recover from, so it is worth naming to the person looking at
it.
The client-side guards live in securesend.js and are advisory by nature - the
server cannot see plaintext, so its size limit is what enforces. They refuse a
pasted FILE outright, which removes the easiest wrong path without anyone
reading a rule, and they warn on something document-shaped: over the length a
credential needs, more than twenty lines, or starting with a PDF/zip/image
signature.
Web Crypto only, no dependency, AES-GCM so a tampered ciphertext fails to
decrypt rather than producing plausible rubbish. The key is base64url for the
fragment, which survives copy-paste and link rewriting better than the standard
alphabet.
VERIFIED THAT THE PAGES RENDER, headlessly, against the dev server: the card
appears under Secrets & Access and the create form draws with its counter and
options. NOT VERIFIED END TO END - no secret has been created, sent, opened or
decrypted, because that needs the dev API restarted and the table migrated.
9845867 gave it its own plugin because Tech Tools is browser-only and this needs
storage. That was the right reading of the code and the wrong answer for the
person using it: Tech Tools is where you go for a utility, and a tool nobody can
find is a tool nobody uses.
So Tech Tools now has a blueprint, a table and a migration chain where it had
none. Its docstring and manifest say why, because "every tool here runs in the
browser" was load-bearing documentation and is no longer true.
WHAT MOVED
/api/securesend -> /api/tools/secrets
securesend.create -> tools.secrets
securesend0001baseline -> tools0001securesend
PLUGIN_TABLE_OWNERS, EXPECTED_HEAD_REVISION, EXEMPT_ENDPOINTS, the universal
profile and the api-inventory surface all follow it.
THE OLD "NO BACKEND" TEST IS NARROWED, NOT DELETED. It asserted
get_blueprint() is None, which is the sort of guard that gets removed when it
becomes inconvenient. It now asserts the blueprint serves EXACTLY the three
secret routes, with their methods - so a future tool that quietly grows an
endpoint or a table has to edit this test, and therefore make the decision on
purpose rather than by drift.
Asserting methods rather than paths also pins the detail this design turns on:
redeeming is POST-ONLY. Chat and mail clients prefetch GET urls to build
previews, and a preview that could redeem would consume a one-view secret before
its recipient opened the link. Previously that was a comment; now it fails a
test.
UNINSTALLING THE OLD PLUGIN COULD NOT BE DONE WITH THE PLUGIN TOOLING. `flask
plugin uninstall securesend` loads a plugin before uninstalling it, so a plugin
whose directory has been deleted cannot be removed by the command that manages
plugins - the registry entry had to be edited by hand. Worth fixing separately;
it is the same registry-versus-code divergence the device work hit from the
other side.
MIGRATING A CHAIN THAT HAS ALREADY RUN needs the old table and its version row
dropped first, because the relocated chain has no stamp and tries to create a
table that exists. Only relevant to anyone who installed securesend during the
hour it existed - it did reach the publication repo in 4242028 - and the fix is
DROP TABLE securesendsecrets plus DROP TABLE alembic_version_securesend before
running plugin upgrade-all.
1829 tests green. Still backend only: there are no pages yet, so nothing here is
reachable by a person.
The habit this replaces is pasting a credential into Teams, where it stays until
someone deletes the conversation. Tabled on 2026-08-21 pending a decision about
scope; the decision was text only, so this is the half that survives it.
BACKEND ONLY. There is no page yet, so nothing here is reachable by a person.
TEXT, NOT DOCUMENTS, and that shapes everything. Dropping files removed the
export-control question entirely - a password is not technical data - along with
blob storage, size and MIME handling, and the need to authenticate the recipient
so a document fetch could be pinned to a name.
TWO DOORS. Creating needs a login AND securesend.create: anonymous creation
would be an untraceable way to move data out of a system that otherwise records
who did what, and the creator is the only person these rows can ever name.
Reading needs neither, because the recipient usually has no account here and the
link is handed over on a call - the unguessable token IS the authorisation,
which is why it is 128 bits of randomness and why the row is spent on first read.
REDEEMING IS A POST, and that is the detail that would have broken this in
practice. Teams, Outlook and Safe Links prefetch GET urls to build previews and
scan for malware, so a GET that redeemed would let a preview window consume a
secret before its recipient ever saw it. The GET route that remains reports only
whether a link is still live, so the read page can say "opening this will
destroy it" without spending the only view to find out.
THE SIZE LIMIT IS THE WHOLE ENFORCEMENT. The key lives in the URL fragment,
which browsers never transmit, so the server holds ciphertext it cannot read and
cannot tell a password from a page of a drawing. MAX_CIPHERTEXT is therefore not
a tuning knob: raising it quietly turns this into the document transfer it was
scoped not to be. The refusal says what the limit is FOR, because somebody
hitting it is usually trying to send a file and the answer is a different tool.
Line counts, file signatures and refusing a pasted file belong in the browser,
guide an honest person, and are bypassable by anyone willing to POST directly.
NOT ENUMERABLE. Missing, expired and spent all answer identically - telling them
apart would confirm to anyone holding a guessed token whether it ever named
anything. A spent or expired row is deleted on the way past rather than waiting
for a sweeper, because it has no reason to keep its ciphertext.
Its own plugin rather than a Tech Tool: that plugin is browser-only by design,
and this needs storage, expiry and a view counter. Not an asset extension
either, so no assetid and no asset type - the scaffold assumes one and it was
removed.
FOUND BY ITS OWN TESTS: expiryhours=0 returned 201. `int(x or DEFAULT)` treats
an explicit zero as absent, so "expire immediately" silently became "expire in a
day". A caller who sends a number is now told when it is not allowed rather than
quietly given a different one.
BEFORE ANYONE RELIES ON THIS, test that your link rewriter preserves URL
fragments: create a secret, send the link to yourself in Teams, click it from the
message, and check the key is still after the #. If Safe Links strips it the
secret is unrecoverable and the server cannot help, which is the point of the
design. The fallback is passphrase mode, worth building only if that test says so.
Three product gates caught what a new plugin has to declare, and each was right
to: the route-parity gate wanted the endpoints described in api-inventory.json,
the lean-build guard wanted the plugin listed in the universal profile the
released installer builds from, and the authz sweep wanted the public redeem
route named in EXEMPT_ENDPOINTS with a reason. That last one matters most - a
mutating route that answers an unauthenticated caller is a decision somebody has
to write down, not a default, and reading a secret is a mutation because it
spends it.
Reported as a barcode rendering 176 by 0. A bar height of zero is what does it:
the number went straight into a CSS length, height: 0in is perfectly valid, and
an image sized height: 100% of a zero-height box is zero pixels tall. The label
comes out full width with nothing on it, which reads as a rendering failure
rather than as a field doing exactly what it was told.
A cleared field was the same bug wearing a different hat. An emptied number
input holds '', not 0, so the page emitted `NaNin`; the browser dropped the
declaration and the box fell back to auto height. That one merely looked wrong
instead of vanishing, which is arguably worse for finding it.
Measurements now go through labelVars(), which will not emit NaN, an empty
length, or a zero for anything whose zero means the box stops existing. Padding,
the quiet zone and the gap still take zero, because there zero is a real answer,
and offsets stay signed. It is a separate module because this is exactly the
kind of arithmetic that needs tests rather than a preview: every case in
labelVars.spec.js is a value a number input can actually hold.
The stylesheet now carries fallbacks on its var() reads as well, but they are a
net, not the fix: a custom property set to garbage is substituted and then
dropped at computed value time, and the fallback does not apply - it only covers
a property that is absent entirely.
Two separate floors were collapsing a live cartridge into a spent one, and
removing either alone changes nothing.
EMPTY_LEVEL was 5, so everything at or below 5% was assigned daysleft 0
outright. The reasoning was that a printer at 1% is out of toner as far as
anyone standing at it is concerned. But the row exists to say how long is
left, and 2% draining a point a day has two days in it. Flooring put that
cartridge beside ones that genuinely are empty with no way back to the
difference. It is 0 now: empty means empty.
int(level / rate) then truncated the division, so anything under a full day
arrived as 0 whatever the floor did - a cartridge with six hours in it was
indistinguishable from one with nothing, and the band read it as empty.
daysleft is fractional now, rounded to two places, which is about a quarter of
an hour: finer than the estimate deserves, but it costs nothing and keeps the
ordering of two nearly-spent cartridges meaningful.
daysText reads in whatever unit carries meaning: "6 hours", "1 hour", "29 min",
"4 days". Below an hour it goes to minutes with a floor of one, because
rounding hours would land back on "empty" - the same bug one rung down.
BEHAVIOUR CHANGE worth knowing: a cartridge at 1-5% with NO history used to get
daysleft 0 from the floor and land on the order list. A rate needs two
readings; with none there is nothing to divide, and the old answer was right by
accident - it said "empty" about a level nobody had watched move. It now
reports 'no history' and shows under "No estimate yet". This reaches only
printers newly added to Zabbix; anything that got to 3% the ordinary way has
the history to forecast from.
Five existing tests pinned the old rule. They recorded a real decision, so they
are rewritten to the new one rather than deleted. One of them was passing for
the wrong reason: its series (2.0, 1.7, 1.4, 1.1) drops 0.9, under
MIN_DROP_FOR_ESTIMATE, so it never had a rate at all and only passed because
the floor short-circuited ahead of the rate check. It now uses a real 20-day
drop at a tenth of a point a day and asserts the ten days its docstring always
described.
The forecast fixture's black cartridge moves from 1% to 0% so the empty band
keeps its API-level coverage, and a magenta at 3% covers hours-left end to end.
Reported as the CODE128 not appearing in the preview. It was appearing, in 44
of the 45 preset and layout combinations swept - as a 36 pixel stub. The code
box and the caption were both allowed to grow, so a caption of any length took
half the label and then kept taking, and on 1.00 x 0.50in stock a few words of
caption left a barcode too narrow to scan. Present, useless, and indisinguishable
from a failed render. The caption now takes the room it needs and no more, and a
CODE128 holds at least 40% of the label, or all of it when stacked: past that
there are too few narrow bars left to read, and a caption that wraps or clips is
the better trade.
So that a real failure can never again look like this one, an empty code box now
says so in red in the preview. It is screen-only - the marker must never reach a
printed label.
The settings were a narrow column with the preview trailing below them, which
put the labels being adjusted off the bottom of the screen. Settings on the
left, labels on the right in a column that stays in view and scrolls on its own,
folding back to one column on a narrow window.
That scrolling column is a print hazard worth naming: an ancestor with its own
overflow clips a multi-page print to whatever happened to be on screen. Print
resets it, and the check is a six-page two-sided run coming out as six pages
with the label as the only clipping box in the chain.
Reported as "CSF04 now reports all 0%", on the printer's own page as well as
the reports. Every supply on one host reading zero at the same instant is not
what a set of cartridges does; it is what a host that stopped answering looks
like.
getsuppliesbyip turned every unreadable item into 0:
try:
level = int(float(item.get("lastvalue", 0)))
except (ValueError, TypeError):
level = 0
An item that has never collected returns lastvalue as an EMPTY STRING, so
float('') raised and the handler substituted zero. lastclock was fetched in the
same request and read by nothing at all, so a reading from three weeks ago was
presented as current.
Zero is a legitimate level meaning spent, which is what made the substitution
dangerous rather than merely wrong. Downstream it reached `level <=
EMPTY_LEVEL`, which set daysleft to 0, which put the printer on the order list:
the report asked purchasing to buy a full set of cartridges for a printer
nobody had measured. On a waste container it was worse - waste is scored as
100 - level, so an unreadable one read as 100% full, the alarm state.
_readlevel now returns (level, lastseen) with level None for no usable reading:
empty, missing or unparseable lastvalue, or a lastclock older than 24h. A
lastclock that is absent is not treated as stale - a reading that cannot be
aged is not thereby wrong, and discarding it would lose real levels. A genuine
"0" is still 0, which is the half that keeps an actually-empty cartridge on the
order list.
classifysupply gains an 'unknown' status with remaining None. It is not
critical: critical is a claim someone acts on, and it may not be made about a
supply nobody measured.
The low-supplies report skips unknown rather than listing it as needing
replacement, and counts it in summary.unknown - a printer that has gone quiet
should be visible as that, not absent and not masquerading as empty.
_annotate_supply reads .get('level') with no 0 default, which would have put
the whole bug back.
PrinterDetail already rendered `level !== null ? ... : 'N/A'`, so it starts
telling the truth as soon as the backend sends null; the UI had been written
for a case the service never produced. It gains the unknown style - muted, not
a severity colour - and a tooltip separating "no reading at all" from "no
reading since <date>", which are different faults with different fixes.
Alerts needed no change: float(None) raises and that loop already continues,
so unknown supplies stop firing low-toner emails as a consequence.
12 of the 16 new tests fail on the old code. The 4 that pass either way pin
that real zeros still behave, which is the regression this fix could cause.
Not verified against CSF04's actual Zabbix data - dev has no reachable Zabbix.
This fixes the mechanism that turns unknown into 0%; /api/printers/<id>/supplies
now distinguishes them, with null for no reading and 0 for a measured zero.
The forecast identified a cartridge by its part number alone. 'W2020X' only
tells you which toner it is once you already know, and both places that name a
supply - the Order now list and the Part column - showed nothing else, so the
report read as a list of codes.
Both now show marketingname, which is what the box says: '414X Black'. The
part number moves to the title attribute, one hover away, because it is still
the thing an order is placed against. A supply with no name on file falls back
to the number rather than showing an empty cell; marketingname is nullable.
'Copy list' is deliberately untouched. It writes the part number verbatim,
which is what purchasing pastes into an order, and a spec pins that the name
does NOT reach that text.
firstPart() now returns the whole part row rather than its number, since the
cell needs both halves. It has one caller.
Dropped the .partnumber style with its .unmapped variant: nothing renders that
class any more, and the variant was already dead - the unmapped block lists no
part number by definition.
Verified by mounting the component against a fixture. The dev box has no
Zabbix, so the page itself has no rows to look at there.
Correction only helped on the global search page, and people search from the
list they are already on. Extending it turned out to be a shape question rather
than a volume one: thirty routes take a search parameter across a dozen files,
so carrying a suggestion in each of their responses is a large change today and
one more thing every future plugin author has to remember.
So the suggestion moved to ITS OWN ENDPOINT, /api/search/suggest, which any page
can call after rendering no rows. A page that has not adopted it shows nothing,
which is exactly what it showed before - nothing breaks by omission.
The plumbing lives in useListQuery, which already owned the search term, so a
list needs three lines: take `suggestion` and `reportCount` from the composable,
call reportCount(rows.length) after a fetch, and drop SearchSuggestion into the
empty state it already has. A list that never calls reportCount never offers a
suggestion.
Wired: global search, machines, printers, PCs, network devices, measuring tools,
knowledge base, vendors. NOT wired, deliberately: the type and reference lists
(machine types, PC types, VLANs, subnets, operating systems and the rest), which
are small controlled vocabularies nobody typo-searches, and USB, whose empty
state has a different shape and wants doing by hand rather than by pattern.
TRAP FOUND WHILE WIRING IT, and left commented in every page: applying a
suggestion by calling setSearch alone updates the box and the URL and does NOT
reload the list. setSearch only syncs the URL, and the watcher that would reload
is suppressed because search.value already holds the new term - the same trap
the global search page documents in performSearch. Each page calls its own load
function directly.
The composable guards a stale answer arriving after a newer search was typed,
never offers back the word that was typed, and swallows its own errors: a search
that found nothing is already the answer, and failing to improve on it is not
worth an error in front of anyone.
The route-parity gate caught the new endpoint being served without an entry in
docs/api-inventory.json, which is hand-written on purpose; added, and the spec
regenerated from it (283 paths, 418 operations). That regeneration also carries
the openapi version to 0.12.0, left over from the release.
The back side landed last commit carrying text by default, and a back laid out
as text-only renders no code box at all. So a two-sided run came out with a
blank reverse, which reads as the code having failed rather than as a setting
doing exactly what it said. The back now defaults to carrying a code, takes its
own content per row, and chooses its own symbology - a QR on the front that
opens the record, a CODE128 on the back that the bench scanner already reads.
Text-only is refused while the back carries a code, and the contradiction is
reconciled on load, so a saved setting from the broken combination repairs
itself rather than printing another blank box of cards.
A picture can go on a label: above the code, beside it, or behind everything as
a watermark, on either side or both. The file is read in the browser and never
sent anywhere, which is what makes it usable at an air-gapped site, and it is
deliberately not saved between visits - a data URL runs to megabytes and would
take every sizing value with it when the quota gave out. The watermark says
plainly that it costs the scanner contrast.
The generator was a full-screen route with no navigation and a narrow column of
controls against the left edge. That shape exists so a print cannot put the
sidebar on label stock, which is a real hazard and the reason core's label pages
are built that way - but it is a hazard the print stylesheet can answer, and
style.css now hides the shell at print time. A tool someone spends ten minutes
in gets the navigation and the full width; the stock still gets nothing but
labels, confirmed against a print PDF rather than a preview.
Two smaller things found while checking the above. A test print paired each page
with a rendered code by loop position, while the codes are keyed to position in
the whole run, so a two-sided test card could carry the wrong code; the index
now travels with the page. And the back's dotted preview outline outspecified
the print reset, drawing a dashed rule down the edge of every second card.
The corrector landed in fa0f6d7 unwired, pending a week of zero-result logs to
say what it should be aimed at. Attaching it now instead: the guards that make
it safe here are principled rather than tuned to any particular data, so the
logs would confirm the shape of the problem without changing the answer.
ONLY ON ZERO RESULTS. A search that finds something pays for none of this - no
vocabulary build, no scoring, no extra query.
The vocabulary is harvested from columns people actually search - asset numbers
and names, vendors, models, applications, asset types - cached ten minutes and
capped per column. Long enough that someone mistyping twice does not rebuild it,
short enough that a vendor entered this morning is correctable before lunch. The
cap is there because this exists to catch a typo against the names in use, not
to mirror the database into memory to answer one query.
SUGGESTED, NEVER APPLIED. The response carries the better spelling and the
caller decides. Searching for something else on a person's behalf is how
somebody orders the wrong cartridge.
FAILS SOFT. A missing column on a lean install, or anything else, returns no
suggestion and the search answers normally. No vocabulary is worse than the
alternative; it is not fatal.
Three tests through the endpoint rather than only the unit: a misspelling gets a
suggestion, a search that succeeds gets none, and CSF18 against an existing
CSF17 returns no results AND no suggestion - the digit guard holding across the
whole stack, which is the one that would cost somebody a wrong bay.
The zero-result logging stays. It answers "what can people not find", which the
corrector does not: a typo is only one of the reasons a search comes back empty,
and the others are vocabulary nobody has entered and records that do not exist.
NOT DONE: the search page does not render the suggestion yet, so this is
invisible to a person until that Vue change lands.
Two halves of one question - how to handle a misspelled search - deliberately
kept separate, because only one of them should be turned on today.
WHAT NOBODY CAN FIND IS NOT RECORDED ANYWHERE. A search returning zero results
is the only evidence of the gap between what people look for and what is there,
and it vanished. Logged now at INFO with a stable prefix, so a week of it greps
into a list. That list is what should decide whether correction is worth wiring
in: multi-word matching and cross-field matching both changed in the last day,
so a good share of what used to fail may already be found. The failures left
over might be typos, or vocabulary nobody has entered, or records that genuinely
do not exist - and each wants a different answer.
THE CORRECTOR IS BUILT AND NOT WIRED IN. shopdb/core/services/spellfix.py, with
tests, ready to attach in about ten lines once there is evidence about what to
attach it to.
NOT SOUNDEX, which was the obvious candidate. MySQL has it and SQLite does not,
and the suite runs on SQLite while production runs MySQL - so a SOUNDEX() in a
query is either an error in every test or a production path no test executes.
It is also wrong for this data: soundex is English-name phonetics, four
characters wide, and it DISCARDS DIGITS, so CSF16 and CSF17 hash identically.
Half of what people search here is an identifier.
So: character distance in Python, same behaviour on both dialects.
THE VOCABULARY IS THE DATA. No dictionary holds Genspect, Telesis, Keyence or
wax-trace. Terms come from the columns being searched, which also means a vendor
added this morning is correctable this morning.
TWO GUARDS, and they are the point rather than a detail. Digits must match
EXACTLY: CSF16 to CSF17 is one edit and a different bay, so anything carrying
digits is either right or not correctable - while cfs16 to csf16 still works,
because the guard is on the digits and not on identifiers wholesale. And the
first character must match, since typos land mid-word far more often than on the
first key, which costs almost no recall.
Distance is Damerau-Levenshtein so a transposition costs one edit rather than
two - Keyecne for Keyence is the commonest error there is, and plain Levenshtein
pushes it past the threshold on short words. Allowed distance scales with
length. A tie returns NOTHING: two equally good candidates means there is no
answer, and offering either implies a confidence that is not there.
It suggests; it never rewrites. Silently searching for something else is how
somebody orders the wrong cartridge.
13 tests, weighted toward the refusals, because those are the cases where being
wrong costs something.
2bf1531 fixed the reported symptom and introduced the opposite one: markers that
had been landing left came right, and then sat well ABOVE where they belonged.
It was the right diagnosis and the wrong remedy. The drawing is painted with
object-fit: contain inside a fixed 390x300 preview, so it is letterboxed on one
axis and a percentage of the BOX is not a percentage of the DRAWING. That part
was true. But the fix resized the frame to shrink-wrap the image, and the
centring transform above it translates by percentages - which resolve against
that same element. Making the frame image-sized silently broke the transform's
assumption that the frame is the box. Width still matched at 390, height no
longer did, which is exactly x-right and y-wrong.
The frame fills the box again and object-fit: contain is back. Where the bars
fall is now computed instead: paintedBox works out the fraction of the box the
drawing covers and the gap before it, per axis, and the marker maths folds that
in. The transform's assumption stays true because the element it translates
never changes size.
Checked per axis rather than asserted. A tall drawing pillarboxes, so x
compresses and offsets inward - a left edge at 2 percent of the drawing lands at
13.1 percent of the box, which is where the drawing actually starts - and y is
untouched. A wide drawing does the reverse. A level shaped like the box is a
no-op, so nothing that already looked right moves. Centre maps to centre on
every shape.
NEW COUPLING, deliberately noted at both ends: PREVIEW_WIDTH and PREVIEW_HEIGHT
must stay in step with .map-preview in the styles. Resize the preview without
touching them and this drifts back.
Still not confirmed against a live instance, and this is the second attempt at
this bug, so it wants eyes on three cases: one that was landing left, one that
was landing high, and one that was right all along.
Reported from the floor: some assets showed their hover mini-map marker way off
to the left, outside the map, while the SAME asset sat correctly on the main
map. The data was right; the preview was drawing it wrong.
The preview is a fixed 390x300 box and the drawing was painted into it with
object-fit: contain, which letterboxes or pillarboxes to keep the drawing's own
aspect ratio. The marker, though, was positioned at a percentage of the BOX. So
the marker and the drawing were measured in two different coordinate spaces, and
they only agreed when the level's aspect ratio happened to be 390:300.
A level near that ratio looked perfect - 3300x2550 is 1.294 against the box's
1.300 - which is why this read as "some devices" rather than "the feature is
broken". A taller drawing pillarboxes, every marker's x shifts, and one near the
left edge lands in the grey gutter beside the map.
The image now SIZES the frame instead of filling it: the transform box
shrink-wraps the image and the image scales down to fit. A percentage of the
frame is therefore a percentage of the drawing by construction, at any aspect
ratio, and the letterboxing moves out to the preview around it where it does no
harm. dimensionsFor still converts the stored pixels to percentages - that part
was always right.
NOT visually confirmed against a live instance. The reasoning is traced and the
build is clean, but the shape of this bug is that it looks correct on any level
close to the old box ratio, so it wants a look at an asset that was landing left
and one that was already fine.
Related and NOT addressed: the wheel zoom applies translate percentages that
resolve against the element and are then multiplied by scale, so a zoomed
preview may still drift. At rest, which is the reported case, it is exact.
An article's searchable text lives in three places: its title, its keywords,
and the name of its topic. Nobody typing a search knows or cares which word
came from which, so 'CMM Community' means "the article tagged community, under
the CMM topic". Both search paths returned nothing for it.
The plugin's own listing, which is what the KB page calls, built a single
ilike('%CMM Community%'). That needs the whole phrase contiguous in ONE field,
and no article has it: 'CMM' is only in the topic's name, 'Community' only in
the keywords. Global search already split the query into words but only ever
looked at title and keywords, so a word that only the topic could satisfy
failed there too - the same bug wearing a different face.
Every word must now be found somewhere across all three fields, in any order.
_word_match gains an `extra` hook for a word that a RELATED row satisfies
rather than a column of this table, which is how the topic joins in without
colliding with the sort=='topic' join.
The retired-topic rule is unchanged and pinned on both paths: matching more
words is not a way past it.
Verified against the 235-article dev library, where all three of these
returned nothing before: 'Fieldglass Jefferson' (title word plus topic word),
'outage notification' (two keywords, not adjacent), 'compucom Jefferson'.
The same phrase-only pattern is repeated in about ten other list endpoints
(computers, machines, printers, network, printedparts, measuringtools, usb,
notifications). Left alone here - fixing them properly means promoting the
word-match helper onto the shopdb.api contract surface rather than copying it
per plugin, which is a contract bump.
Evaluated a Password-Pusher-style service as a Tech Tool and tabled it. The note
carries the decisions rather than just the idea, because those are what cost the
time: it is NOT a Tools-plugin candidate as it stands (that plugin is browser
only by design, with get_blueprint and get_models both empty), the sender must
always be authenticated, a FILE recipient should be logged in too - which costs
nothing on an intranet-only server where anyone who can resolve the link already
has an account - and zero-knowledge encryption is right for a credential but
questionable for a controlled document, where a server that cannot say what was
shared is a liability rather than a feature.
Intranet-only hosting is what makes the idea viable at all: a link mailed
outside is dead and no ciphertext reaches a third party. Pointing staff at the
public hosted instance instead would reinstate every problem this avoids.
Three things the code generator could not do, each of which had someone
working around it with a control that meant something else.
The gap between a code and its caption was the quiet zone. That is the
clear space a scanner needs around the code, so closing up a caption meant
shaving the thing that makes the code readable, and the only way to get a
tight label to scan was to push the caption further away than it looked
like it should be. Quiet zone now means the code's own margin and nothing
else; gap means the distance to the caption, and adds to the quiet zone
rather than eating into it. Each stock preset carries the gap that suits it.
Position had one pair of controls and they moved the label box on the
stock. That is the fix for a printer whose origin is off, and it is not the
fix for a code sitting in the wrong place on a label - but it was the only
lever there, so it got used for both. The label div is now the media and
holds still; everything printed sits in a content wrapper that Align,
Vertical and Content X/Y move within it. The old pair kept its behaviour
and is labelled Media X/Y.
A badge card has two sides and the generator only ever emitted one page per
card. The back can now carry the same code again, a fixed block of text,
its own per-row content, or nothing. Page order is a choice because the two
kinds of printer want opposite things: interleaved for a duplex card
printer, which takes pages two at a time, and grouped for a single-sided
one whose stack gets flipped and reloaded. A row that leaves its back
column empty still gets a blank back page - a page count that varies per
row shifts every later back onto the wrong front, which prints a box of
perfectly readable cards that are all wrong.
CSV parsing and page building moved to plugins/tools/frontend/labelPages.js
(committed in 875fde9), where the column contract and both page orders are
covered by tests rather than by looking at the preview.
A review of d60ed60 and 8b9b936 found five things. Three were introduced by
those commits.
REFUSING A NAMED DEVICE MADE A MARKER PC CLAIM ITS OPERATION. _sync_partmarker
returned [] both for "not a marker PC" and for "a marker PC that linked
nothing", and the caller reads [] as the first - so a bay whose asset-id.txt
named something unresolvable fell through to the ordinary machine link, and for
a marker PC the machine number IS the operation. It took an active link to a
record that can only have one holder while several markers share it, and the
warning said "not linked". The previous behaviour minted a twin; this traded
that for a contested operation. None now means "not a marker PC" and is the only
answer that lets the machine link run; the response normalises it away so the
API shape is unchanged.
A DORMANT CHALLENGER WAS PROMOTED BY DELETING A FILE. Recording a challenger
dormant leaves a row that the next cycle finds as `reuse` and reactivated with
no incumbent check - so a second PC took a live device by its enrollment file
becoming unreadable. Both reuse branches re-check incumbency now, which is what
_sync_machine_link always did.
AN INCUMBENT UNDER ANOTHER COLLECTOR LABEL WAS INVISIBLE. Incumbency was queried
on our own label, but on a CMM the instrument IS the reported bay, so the
incumbent's link is the machine sync's row. A second PC naming that instrument
found no incumbent and linked actively: two live holders of one instrument, each
invisible to the other. Incumbency now counts any collector-owned label. A row
made BY HAND carries none of them and is still excluded - a person's link is not
the collector's to archive.
THE NETWORK FORM LOCKED OUT THE ROWS IT NEEDED TO FIX. Asset number is disabled
while editing, correctly, but the payload is built in script so the blank was
still sent - and the new server-side guard rejects it. A device with no asset
number could not be saved at all, and the field could not be typed into. It now
unlocks only for a record that loaded without one, with a hint saying why.
"2 in 4.0d" WAS THE LABEL LYING. Replacements are counted across the whole
history window; basisdays is only how long the current cartridge has been in.
Joining them with "in" claimed two changes inside four days - the exact shape
reported as unbelievable, except here the data was right. Now "2, this one 4.0d".
Two toner dips the same review found:
A MULTI-POLL OUTAGE STILL MINTED A PHANTOM SWAP. Only single readings were
dropped, so 90, 0, 0, 90 survived and 0 -> 90 scored as a change. Dips of any
length are handled now. One bad sample stays a candidate whatever the polling
cadence, because a reading is an instant; several consecutive low ones only
count as one outage when they are close together, since days at zero is a real
empty period. That time bound also separates an outage from a swap, ordinary
consumption, and a second swap, which have the same shape in levels alone.
THE DIP FILTER ATE REAL SWAPS OF NEARLY-FULL CARTRIDGES. Recovery was tested
with an absolute difference, so 95 then 5 then 100 read as a recovery because
100 and 95 are close, and the swap evidence was deleted. Toner only falls: a
recovery comes back at or BELOW where it left, a new cartridge comes back
higher.
The review also proved by reverting each feature that the previous tests did not
pin the median burn rate or the near-full rule - both passed with the bug
restored. Verified by the same method that all four toner behaviours now fail
when reverted, and the burst assertion is tight enough to tell 0.2 from 0.88.
Reported from the floor: two printers showing 81 and 83 percent, both forecast
to run out in a fortnight. That is a rate near 5.8 percent a day on a cartridge
barely touched.
burn_rate required four readings and a 2 point drop, and checked only that some
time had passed - not how much. Supply items are commonly polled every few
minutes, so four readings can span a quarter of an hour, and a 2 point drop
across fifteen minutes extrapolates to nearly 200 percent a day. The report then
sorted that confident wrong number into "soon", next to cartridges that really
are about to run out. It did not look broken; it looked urgent.
A rate now needs two days behind it. Two days is the smallest span that survives
a printer's daily rhythm, so one heavy morning does not become the whole picture.
Below that the answer is "not enough history yet", which the report already has
a home for: band() returns None and the row lands in the "No estimate yet"
section with its reason shown, rather than competing for attention with real
urgency.
This is the third face of the same fault. A phantom replacement truncated the
run - "2 replacements in 2.3 days" was the same printer saying so - and the rate
was then fitted to whatever short stub remained. The near-full rule and the dip
filter stop the truncation; this stops a stub from producing a number at all.
Two things reported from the floor, one cause each.
"5 CHANGES IN 90 DAYS, THAT'S HARD TO BELIEVE." It was. A replacement was any
+10 rise between readings, with no check on where it landed, so two shapes that
are not swaps scored as swaps: a supply reading 0 or near-0 while it was out of
the machine and then reading normally again, and a coarse gauge ticking back up
after a reseat or a power cycle.
A swap must now also LAND near full, because that is what a new cartridge reads,
and a single dip that RECOVERS to roughly where it came from is dropped before
anything looks at it. The dip filter keys on shape rather than cause, which is
why it holds for all of them - a supply pulled out to be shaken, a door open
mid-poll, or a site whose preprocessing maps the Printer MIB's unknown
sentinels onto 0. It is NOT a Zabbix timeout: an item that does not answer
records nothing rather than writing a zero. A genuine near-empty reading before
a real swap does not recover, it jumps to full, so it survives and its swap
still counts.
find_replacements and current_run now read one predicate. When they disagreed, a
phantom rise reset the run and threw away the history the estimate needed - so
the bad count was quietly damaging the rate as well, which is why both were
wrong at once. Expect replacement counts to FALL and per-cartridge history to
lengthen.
A BURST BIASED THE RATE FOR THE LIFE OF THE CARTRIDGE. The rate was the slope
between the first and last reading of the run, and two endpoints cannot tell
"steady" from "burst then stopped". A cartridge that lost 20 percent in two days
and then barely moved for a month read as 0.83 percent/day forever after, so the
report kept promising it would run out long after printing slowed. It is now the
median of the per-interval rates: the burst is one interval among many rather
than one of two points. Rising intervals are dropped as noise; flat ones stay in
at zero, because a cartridge that did not move is real information. If every
interval is flat or rising yet the run dropped overall, it falls back to the
whole-run slope rather than reporting nothing.
Where the intervals disagree by 5x or more the rate carries a marker and an
explanation on hover. The number is still the best estimate available; the flag
stops it reading as a measurement.
The "Changed" column is "Replacements", and its cell says "2 in 90d" rather than
"2 / 90d", which was read as a date, a ratio and a version number.
A review of last week's device-identity work found these; two were writing bad
data and one was reproduced against a live server before being fixed.
AN UPDATE COULD BLANK AN ASSET NUMBER, on all six asset update paths. Create
validates it and the column is NOT NULL, but the conflict check only runs when
the value DIFFERS, and '' collides with nothing - so an empty assetnumber went
straight through to a required column. This is the likely source of the assets
found with no number: a form that loaded blank and was then saved.
MACHINEFORM COULD LOAD BLANK AND LET YOU SAVE IT. One try/catch wrapped eight
reference loads AND the machine fetch, so a single transient failure among them
- one page of listAll() timing out during a collector cycle is enough - rejected
the whole block and rendered a fully editable EDIT form with every field empty,
the error banner far below next to Save. Typing an asset number and saving then
wrote the blanks over a real machine. The record now loads in its own try, and a
failure shows the reason INSTEAD of the form: an empty edit form is
indistinguishable from a record whose fields are genuinely empty.
NAMING A DEVICE THAT DID NOT RESOLVE STILL MINTED A TWIN. Both device paths
warned "not linked" and then fell through to mint <HOST>-PARTMARKER or
<HOST>-CMM - the hostname-derived twin the resolution order exists to prevent.
The warning was true about the typo'd number and false about the twin. Naming a
device is a commitment: if the name does not resolve, or resolves to the wrong
kind of thing, link nothing and say so. Silence still means "work it out", so a
bay with no file keeps the reuse-then-mint behaviour it always had.
TWO PCS COULD BOTH HOLD ONE DEVICE, ACTIVELY, WITH NO WARNING. Verified against
a live server: report as one host, then as another naming the same marker, and
both controls rows stayed active. Neither device path had ever looked at who
else held the target - only at links whose source was THIS PC - so a replaced PC
kept its link forever and an asset-id.txt copied to a second bay claimed the
device silently. It now reuses the machine link's rule rather than inventing a
second one: an incumbent that has gone quiet past the claim window or been moved
off In Use has yielded and is archived, never deleted; a live incumbent keeps
the device and the challenger is recorded dormant.
The swap test asserted the old behaviour and now asserts the new one, split in
two: a live incumbent keeps it, and handover completes once the incumbent
yields. Two other tests were passing while their names lied - the unknown-device
one checked only that the typo'd asset was not created, not that nothing was
linked, and it passed while a twin was minted beside it.
Three fixes, all of them a page not doing what its siblings already do.
MODELS WERE NOT FILTERED BY VENDOR. PCForm, MachineForm and PrinterForm each
narrow the model list once a vendor is chosen; NetworkDeviceForm bound the whole
catalogue, so picking Palo Alto still offered every Dell and Zebra model. Same
computed as the others, including the same rule that no vendor selected shows
everything - an empty dropdown reads as "no models exist" when it means "pick a
vendor first". Audited the rest: this was the only gap. The other views holding
a modelnumberid have no vendor picker to filter against, and settings ModelsList
is where a model's vendor is ASSIGNED, where filtering would be circular.
THE HERO RAN THE LABEL INTO THE VALUE - "Asset #FW-OAV..." as one string. The
page used detail-item / label / value, which match nothing in the stylesheet, so
the two spans got no layout at all. Every other detail page uses hero-detail /
hero-detail-label / hero-detail-value, which stacks a small uppercase label above
the value. Renamed to those; no CSS added, because the styles already existed and
this page simply was not using them. It was the last page using the unstyled
names.
MACHINES LIST LINKED BY THE WRONG ID on its fallback path. `/machines/:id` keys
on machineid, the plugin extension id, and the row click and View button fell
back to `item.assetid` - which lands on whichever machine happens to carry that
number: a wrong page that looks right, which is worse than a 404. That defect has
been fixed twice before in other views (AssetRelationships, then the GE-Enforce
reports table) and BackupHistory carries a comment warning about it; this was the
fourth copy. The list endpoint always sets item.machine, so the fallback could
not actually fire here - it is removed as a latent trap rather than a live bug,
and with no machineid the cell now shows plain text rather than a link that
misleads.
Phase 3 was written as "the collector reads a registry value". It is all three
components moving together: imaging writes DeviceId, GE-Enforce resolves device
identity registry-first, and the collector reads the registry ahead of the file.
The argument for it is already in the fleet's own code. Install-FromManifest
resolves TargetMachineNumbers gating from the eDNC registry BEFORE
machine-number.txt, and says why: the file is written ONCE by startnet.cmd at
the PXE menu and is not updated on reassignment, so it goes stale.
asset-id.txt inherits that defect exactly. Nothing rewrites it when a device is
swapped, moved between bays or replaced, and phase 1 put it at the TOP of the
resolution order - so a stale value is adopted with no warning, because it still
resolves. A file only imaging writes cannot be the top of a resolution order for
something that changes during a PC's life.
Also records the consequence: phase 3 needs a reassignment path the way
Set-MachineNumber exists for the bay number, or it moves the staleness from a
file into a registry value instead of curing it.
The phase 3 section named the key and left the scope to be inferred from the
fact that deviceid is one field. Spelling it out: DeviceId is a single value,
not a key per device type, and the pc-type decides which sync consumes it - so
CMM, Keyence, Genspect, wax-trace and part markers all read the same value, and
a family declared later through subordinatedevice_<pctype> is covered the day
it is declared.
Also records what the value REPLACES per family, which differs: for a CMM it
replaces cmmid.txt as identity only, for a part marker it replaces minting from
the hostname, and for the other three it replaces nothing because they never
had an identity at all.
Phase 1 shipped without a plan around it, and the next two steps both touch a
different repository and the live boot image, so the reasoning needs to outlive
this week rather than being rediscovered from the code.
The finding that shapes the whole thing: cmmid.txt CANNOT simply be retired. It
is a PATH KEY, not just an identity - Verify-And-Heal-Staging stages
installers-post\cmm\backups\<cmmid>, 09-Setup-CMM locates the staged set by it,
and Restore-CMM restores from <BackupRoot>\<cmmid> as a same-bay operation. Only
one of its five consumers is an identity use. That one is also the one doing the
job badly, reporting a CMM's INSTRUMENT id in the machinenumber channel so the
server can resolve it back to a measuring tool at step 4 of its own fallback
chain.
So phase 2 has imaging write asset-id.txt ALONGSIDE cmmid.txt - two files, two
questions - and demotes the identity use while every path use stays exactly
where it is. cmmid.txt survives phase 3 too; it is not on any deletion list.
Phase 3 is the registry, HKLM:\SOFTWARE\GE\ShopDB\DeviceId. No new key and no
new read path: the collector already opens that key for BaseUrl and ApiToken and
Install-GEEnforce already writes it. The proposal argues for doing phase 3
BEFORE phase 2 rather than after, because then imaging writes the registry for
new bays and skips the file era entirely - the file survives only for bays
staged between phases, and the cleanup is deleting a branch instead of sweeping
a fleet.
Also recorded: the rail that must not be crossed (machinenumber is what
TargetMachineNumbers gates on, so a device id in that channel stops every
bay-gated entry from matching, silently, forever), what each phase does NOT
change, per-phase rollback, and a note that phase 1 stops new duplicates without
merging the ones a PC swap has already made - so the reconciliation backlog
should be read BEFORE imaging starts writing identities over the top of it.
A bay with an NTLARS/eDNC MachineNo registry is identified by it, and a CMM by
cmmid.txt. Everything else - Keyence, Genspect, wax-trace, part markers, and
whatever a site declares next - had no stable identity at all, and the two
mechanisms that stood in for one both key off the PC: reuse looks for a prior
link from THIS PC asset, and minting builds `<PC number>-<SUFFIX>` where a PC's
number is its hostname.
Both survive a re-image. NEITHER survives a PC SWAP. A new hostname is a new PC
asset with no prior link and a predicted number that has never existed, so the
same physical device gets a second record while the first keeps its config and
its backup history under a dead PC's name. That is how 43 legacy MT-#### tools
ended up shadowed by minted twins, three records deep in places. The metrology
path learned this and gained an explicit id file; the part-marker path was
modelled on the metrology path as it stood BEFORE that fix, so it inherited the
defect - and its own docstring said so, describing minting "the same way it
already mints a CMM".
C:\Enrollment\asset-id.txt is now that identity for both, and for anything
declared later through subordinatedevice_<pctype>. It holds one line, the
device's assetnumber, and deliberately does NOT record what kind of device it
is: the pc-type already does, so a new device type needs no new file and no
client change. Resolution puts it first, ahead of everything PC-derived.
NOT machine-number.txt. machinenumber answers "which bay is this" and is what
GE-Enforce TargetMachineNumbers gates on, so naming a device there would
silently stop every bay-gated manifest entry from matching. A part marker still
files partof the operation from machinenumber; asset-id.txt changes which marker
the PC controls, not which operation that marker belongs to.
The wire field is `deviceid` rather than `assetid`, because assetid is already
this contract's RESPONSE field for a PC's integer primary key - the two would
have sat side by side meaning different things. measuringtoolid, shipped in
0.12.0, is accepted as an alias and the client still reads the older file, so a
bay staged in the last day keeps reporting; deviceid wins when both arrive.
Two guards, shared by both device families: a value that resolves to nothing
warns instead of minting a phantom, and a value that resolves to the WRONG KIND
of asset is refused with the asset named, so a machine number pasted into the
file cannot be filed under a device label the collector also owns.
Row creation is now one helper instead of a copy per path, because the named and
minted branches both need get-or-create on the extension row and the control
link.
1744 tests green, including a new file that pins the swap case both ways - with
the file one device, without it two. Five of its tests failed first time because
the fixtures built bare assets with no extension row and the type guard refused
them, which is the guard working.
VERIFIED ON WINDOWS 11 (build 26200), five cases: the new file alone; both files
present, new winning; the legacy file alone; neither, sending no field; and a
padded value with a trailing line.
The server has accepted `measuringtoolid` since the adoption work landed, and it
is the FIRST entry in the resolution order precisely because it is the identity
that survives a PC swap. Nothing ever sent it. The reporter read the eDNC
registry, cmmid.txt, machine-number.txt and pc-type.txt, and its own comment
said metrology bays have no per-bay id and therefore send nothing - so a Keyence
or Genspect bay fell through all four steps to minting, which the server's own
docstring calls the last resort.
Minting derives the asset number from the HOSTNAME, so a permanent instrument
inherits the identity of whichever PC drove it that week: replace the PC and
either the number lies or a second tool appears for the same physical unit. That
is how 43 legacy MT-#### tools ended up shadowed by minted twins. The half that
prevents it was built, tested and undeliverable.
The reporter now reads C:\Enrollment\measuringtool-id.txt and sends it when
present. Its own file, NOT machine-number.txt: machinenumber answers "which bay
is this" and is what GE-Enforce TargetMachineNumbers gates on, so naming a tool
there would silently stop every bay-gated manifest entry from matching.
The paste-ready reporter in COLLECTOR-INTEGRATION.md is a second implementation
of the same payload, so it gets the same resolver rather than being left to
drift. The field was also missing from the payload table and from the classic
api.asp mapping, and there was no prose anywhere describing how a tool is
resolved - added, including why minting is last and what the two guards refuse.
VERIFIED ON WINDOWS 11 (build 26200), four cases: a named instrument is read and
sent; no file sends no field and exits 0; a whitespace-only file behaves as
absent rather than sending an empty string; and a padded value with a second
line yields the first line trimmed.
Printers become a property of the bay, and the container stops throwing away the
site's own files every time it is updated.
Printer assignment is now end to end. The assignment belongs to the MACHINE and
reaches whichever PC controls it, so a reimaged or swapped bay comes back with
its printers and nothing had to be saved off the old box. One picker serves both
forms rather than two implementations of the same override. A relationship type
can declare itself singular, so setting a second default REPLACES the first
instead of silently losing to the older row. Alongside the assigned half there is
now an observed half: a bay reports what it actually has, kept strictly apart
from what it is told to have, because a drifted bay's own state becoming its
desired state would make every configuration error permanent. The client script
corrects a drifted queue in place rather than only installing a missing one, and
a driver rollout can be spread across waves so 300 bays do not pull 30 GB through
one five-minute window.
DOCKER SITES SHOULD READ THIS PARAGRAPH. The compose stack never persisted the
instance directory, so `docker compose build api && up -d api` - the update path
the docs themselves gave - discarded plugins.json and every upload with it. It is
a volume now, and DEPLOY.md carries the one-time rescue for a stack that predates
it. The image also could not be built at all: the frontend stage never copied the
plugin staging script its own prebuild hook runs, so every build since that
script landed failed. Air-gapped sites need a fresh offline bundle, because the
stack moves to MySQL 8.4 LTS and an existing tarball carries only the 8.0 image.
The Windows installer already bundled MySQL 8.4, so this closes a gap between the
two halves rather than moving anyone.
Adds a topology migration guide: IIS to Docker, Docker to a new host, and back.
The data moves cleanly; what costs time is that the server address is baked into
GE-Enforce manifests, the generated collector script, the printer client scripts
and printed QR codes.
1736 tests green. Contract stays at 0.20.0 - nothing under shopdb/api changed.
INSTALL-WINDOWS-IIS.md has said MySQL 8.4 LTS is standard for new installs since
8.0 reached end of life in April 2026, while both compose files and the offline
bundler still pinned 8.0. A site reading the Windows runbook and a site reading
the Docker one were being told to run different servers, and the migration page
written this week sent people onto the dead one.
Verified against a real server rather than by editing a tag: 8.4.11, core chain
plus five plugin chains applied clean, 66 tables at a single utf8mb4_unicode_ci
collation, six alembic version tables. The image's PyMySQL authenticates against
8.4's caching_sha2_password, which is what requirements.in already pins
cryptography for.
Existing servers need one thing done FIRST: 8.4 removes mysql_native_password,
so an account created on 5.6 or 5.7 must be moved to caching_sha2_password
before the upgrade or it cannot authenticate afterwards. In-place also has no
downgrade path, and 5.7 cannot reach 8.4 in one hop. For databases this size a
dump into a fresh 8.4 server is the better trade: same outage, and the old
server stays as the rollback.
Air-gapped sites need a fresh offline bundle, because the tarball carries the
MySQL image alongside the app image.
Also here, found by having it bite during that verification: the db healthcheck
pinged over the unix socket, and the entrypoint's init pass answers on the
socket while running the server with --skip-networking. The probe therefore
reported healthy DURING init, which is what `depends_on: service_healthy` gates
api and migrate on. A ping passed at 8 seconds and the next query failed because
the server was mid-restart. Probing 127.0.0.1 keeps it red until the real server
is listening.
The pieces existed across three pages and nothing connected them, so "can we go
from IIS to Docker" had no answer to point at. It is a fair question with a
short answer: the application keeps state in exactly two places, the database
and the instance directory, and nothing is encrypted at rest with SECRET_KEY or
JWT_SECRET_KEY, so a move is a dump plus a directory copy. The schema is
identical across topologies.
What the page spends its length on is the part that is NOT the data, because
that is where the time goes. The server address is baked into things that are
not the server: GE-Enforce manifests, the generated collector script, the
printer client scripts, and printed QR codes, which cannot be swept at all.
Keeping the hostname and repointing DNS makes the migration invisible to the
fleet; changing it does not.
Three other traps, each of which has a symptom that shows up later rather than
at cutover: an aliased IIS site needs MOUNT_PATH and a dist built for that
subpath, while the image builds for the root; an older dump can carry latin1 or
3-byte utf8 table definitions that load quietly into a utf8mb4 server and only
misbehave on the first accented name; and a restored instance directory needs
chown, because docker cp writes under the copying user's uid and the container
runs as shopdb.
Covers both directions plus Docker to a new Docker host, and ends with a cutover
checklist that leaves the old stack stopped rather than removed until a bay has
checked in on a working day.
Also fixes two links in START-HERE that pointed at files which are not there:
the ADR index needed its adr/ prefix, and LLM-GUIDE.md is llms.txt.
`db_data` was a volume and the instance directory was not, so the documented
update path - `docker compose build api && up -d api` - recreated the container
and discarded everything the site had written. `plugins.json` is only the loud
part: maps, branding, model and application images, employee photos, warranty
proofs, slides, printed-part files and the Dell OAuth token all live under
instance_path too. MySQL rows survive and point at files that are gone, so the
second symptom is images 404ing rather than an error anybody sees.
Reported by an adopting site, which read it as having updated too fast. It had
not; nothing it could have done differently would have kept those files.
DEPLOY.md had been telling sites to back up `instance/` since it was written.
The template never gave them anything to back up.
The air-gap `migrate` service mounts the volume too, because
`flask plugin upgrade-all` rewrites plugins.json and that service exits
immediately after.
The image now creates instance/ ITSELF, owned by the app user. Docker seeds an
empty named volume from image content at the mountpoint, ownership included;
with no such directory in the image the mountpoint is created root-owned 0755
and the container, which runs as shopdb, cannot write into its own instance
directory. Caught by running the built image rather than by reading it: the
volume mounted clean and `touch` came back Permission denied. Verified fixed the
same way.
A stack that predates the volume needs its files moved across ONCE, while the
old container still exists - the volume is seeded from image content, and the
image ships instance/ empty, so it comes up empty rather than inheriting the old
container's writable layer. DEPLOY.md carries the procedure, including the chown
after `docker compose cp`, which writes files under the copying user's numeric
uid rather than the app user's.
Also here, found while checking what an upgrade actually runs: the connected
update steps ran `flask db upgrade` and stopped. Per-plugin Alembic chains
(ADR-008) are not part of that, so a connected site taking an image with a
bumped plugin migration ran the core chain and silently skipped every plugin
chain. The air-gap stack had it right all along. Both commands are in Step 9
now, plus a `db current` check against `db heads`.
`npm run build` fires a `prebuild` hook that runs
`node ../scripts/stage-frontend.mjs`, the script that copies each plugin's
frontend into the Vite tree and codegens routes.gen.js. The Dockerfile's
frontend stage copied `frontend/` alone and flattened it to the stage root, so
`../scripts` resolved to `/scripts`, which does not exist. Every image build
since the staging script landed has died there on `Cannot find module`.
The stage now keeps the repo-relative layout - frontend/, scripts/ and plugins/
as siblings under /build - because the script resolves its repo root as its own
directory's parent. The final stage copies from /build/frontend/dist to match.
Verified with a real build, not by inspection: 16 plugin frontends staged, vite
produced dist, and the finished image carries frontend/dist with 241 assets.
This also unblocks SITE_PLUGINS lean image builds, which until now only
scripts/build-site.sh could do.
GE-Enforce gives each PC a start offset of SHA256(hostname) % 5 MINUTES and then
repeats every five minutes. That was sized for reading a few KB of manifest
JSON. A driver set is 100 MB for the two universals and 226 MB for a full site,
so the day a driver entry lands, every bay pulls it inside one five-minute
window: roughly 30 GB across 300 bays, at something like 800 Mbps, on the same
share the whole floor needs for everything else. The failure mode is not slow
printers, it is a floor that stops converging.
-WaveStart with -Waves spreads that out. Each PC derives its own wave from its
hostname, so there is no central coordination, no per-bay configuration, and no
list of who has had it yet. The hash is the same idiom Register-GEEnforce.ps1
already uses for its start offset, SHA-256 rather than MD5 because FIPS-enforced
bays disable MD5 outright and would throw.
Measured over 300 hostnames at 10 waves: 23 to 44 bays per wave against a mean of
30, so the peak wave moves about 4.3 GB rather than the 3 GB an average implies.
Hash bucketing is uneven and the peak is what sizes a link, so do not quote the
mean.
THE GATE RUNS BEFORE THE MANIFEST IS READ, because the manifest is on the share
too. A bay that is not due must not touch the share at all - one read is cheap,
300 bays deciding to read in the same five minutes is the entire problem.
It FAILS CLOSED on an unparseable date. Failing open would restore exactly the
stampede this exists to prevent, and 30 GB cannot be un-sent, whereas a typo that
installs nothing says so in the log every cycle and is fixed in a minute.
A bay powered off during its wave installs on its next cycle instead. The wave is
an earliest-time, not a deadline, so nothing needs chasing afterwards.
-TestOnly reports a bay whose wave has not opened as COMPLIANT, because not
installed is genuinely its desired state today; DSC would otherwise call
SetScript every pass to be told to wait. -IgnoreWave is for proving a pilot bay
before opening anything.
Verified on Windows 11 build 26200, six paths: not-due installs nothing and exits
0; TestOnly while not due exits 0; a garbage date exits 1 having installed
nothing; -IgnoreWave installs against a future start; an opened wave installs;
and no wave arguments at all installs, which is what imaging needs.
ShopDB knew what a bay SHOULD have and nothing about what it DOES. Adding the
observed half makes a rollout a review instead of a typing exercise: the floor
reports itself in, you look, and you adopt.
The collection uses the mechanism that already exists rather than a new one.
POST /api/collector/printers dispatches to the printers plugin's
apply_collector_payload, the same ADR-006 hook the computers and backups plugins
implement. New client script, new plugin-owned table, no new transport and no new
credential.
OBSERVED AND ASSIGNED STAY APART, and that is the point rather than a detail. A
collector report can never write an assignment row: _reconcile_edges is the only
function that writes usesprinter/defaultprinter, it has two call sites, and both
are authenticated routes a human calls. If a drifted bay's own state were allowed
to become what it is told to install, every configuration error would become
permanent the next time that PC checked in.
Seeding an assignment from observed state is explicit -
POST /assignments/seed-from-observed - because a rollout adopts many machines at
once. It routes through the same _reconcile_edges as the editor, so there is one
write path with two doors, and a queue matching no known printer is REFUSED
rather than guessed into an assignment. That last rule is the lesson from the
measuring tools: adopting on a weak key produced 43 duplicate instruments.
Two fixes on top of what the agents built. The replace deleted a host's previous
rows by exact case-folded name while the read path treats a short name and its
FQDN as one machine, so a PC that changed spelling appeared to hold every queue
twice - which reads as drift that is not there. And the client sent 'reportedat'
where the declared schema said 'observedat'.
Also here: the legacy loader now imports machines.printerid, the classic system's
record of each machine's default printer, which it silently dropped - the
production import would have lost every one. And Set-ShopdbPrinters.ps1 finally
registers the per-user logon task, staging Apply-ShopdbDefaultPrinter.ps1 to
C:\ProgramData first because the share it lives on is mounted only during the
enforcement cycle and the task runs at logon when it is gone.
VALIDATED ON WINDOWS 11 (build 26200), not just on Linux pwsh, which parses these
scripts happily and executes none of the spooler branches.
The reporter: posts a correct payload with the X-API-Key header; resolves BaseUrl
and CollectorKey from HKLM when given no arguments; suppresses the virtual queues
by port; resolves port addresses; and reads the CONSOLE USER's default out of
HKU rather than SYSTEM's own, which is a different and usually wrong answer.
Two results matter more than the rest. With the spooler stopped, both the cmdlet
and the CIM path fail and the script posts NOTHING - verified against a capture
server that recorded zero requests, where an empty list would instead have
erased that host's observed rows and read as a bay that lost its printers. A
genuinely empty host still posts [], because that is a real and different fact.
The logon task registers as the Users group at Limited, and falls back to the
well-known SID S-1-5-32-545 when the group name will not resolve, as it will not
on localised Windows. It was then run with the source directory RENAMED AWAY, to
stand in for the share being unmounted, and it still moved the user's default -
which is the whole reason the script is staged to C:\ProgramData rather than run
from where it lives.
The guarantees against damage were re-checked rather than assumed: an empty
assignment changes nothing, an unreachable server changes nothing, -WhatIfOnly
leaves no queue, no task, no staged file and no registry value behind, and a
drifted queue is repointed IN PLACE with Set-Printer so whoever has it as their
default keeps it.
Not covered by any of this: the driver-staging path, which needs a real vendor
package rather than the class drivers a VM ships with.
A queue was matched by NAME alone, so a bay whose printer had moved, or whose
queue was built on a driver the site has since replaced, looked converged and
printed to the wrong device. Absence was fixed; drift was not.
Set-ShopdbPrinters.ps1 now repoints a queue whose port does not match the
address ShopDB holds, and swaps a queue built on the wrong driver. Both are
things ShopDB is authoritative about: where the printer IS, and what drives it.
CORRECTED IN PLACE with Set-Printer, never removed and recreated. The queue keeps
its name, its sharing, its permissions, and whoever has it as their default keeps
it - which is what makes this safe to run every cycle on a live floor. There is
still no removal code path in this script at all.
Two guards, because a repair that breaks a working printer is worse than drift:
the driver is only swapped when the wanted one is actually staged, and
-WhatIfOnly reports both kinds of correction without making either.
Verified on Windows against a queue that had the right name, the wrong port, the
wrong driver AND was the logged-on user's default: both fields were corrected and
the queue was still the default afterwards. The earlier no-op guarantees were
re-run and still hold - nothing assigned changes nothing, and an assignment with
no default leaves the user's own default alone.
The assignment belongs to the MACHINE, and until now there was no way to set it
except the generic relationships card or the API - the form for the thing the
feature is about did not exist. MachineForm now carries the picker, and PCForm
uses the SAME component rather than its own copy: the PC's set overrides the
machine's, and two implementations of that would drift, with the two ends of an
override disagreeing being exactly the bug nobody would spot.
The shared picker also fixes what PCForm did on save. It wrote row at a time
through the generic relationship endpoints, which is a non-atomic reconcile: an
HTTP failure part way left a PC half-assigned with nothing recording what was
meant. It now calls the reconcile endpoint, which validates the default before
writing anything.
A relationship type can now say it allows one active row per asset
(relationshiptypes.issingular, migration 7d34), and defaultprinter says it.
Cardinality belongs to the type rather than the printers plugin: core's create
path is where every hand-made link passes, and the next type meaning "exactly
one" gets the rule for free. Setting a second default REPLACES the first instead
of refusing, because "make this the default" means that - and a card answering
409 would leave the user hunting for the old row.
Without it the schema was happy to hold two defaults: the unique constraint is
(source, target, type), so two different targets are two valid rows, and the
resolver takes the OLDEST - the new default silently lost. Proven by disabling
the new rule and watching the tests fail.
FOUND WHILE TESTING IN A BROWSER, and it was not mine: MachineForm read
.data.data off computersApi.listAll(), which resolves to the ARRAY - fetchAllPages
has already unwrapped every page. The whole parallel load threw into the catch,
so every dropdown on the machine edit form came up empty and the machine's own
values never loaded. A build cannot see this; only opening the page can.
GET /api/printers/assignments/for-asset/<id> returns an asset's OWN assignment,
without inheritance, because the editor must show what this asset's rows say -
otherwise a machine's printers appear ticked on the PC that inherits them and
unticking one silently creates an override.
The last of the buildings-and-levels bugs, and the shop-floor clients brought
into the product.
Anyone on 0.11.0 through 0.11.2 should take this. Every hover mini-map in the
product reported "this asset has a position but no level" - the levelid prop
added in 0.11.0 was passed by none of its seven call sites - and the map PDF
printed markers from every floor onto one sheet, which nobody can correct once it
is carried onto the floor. The legacy import loader, still to run against
production, created markers with no level at all.
The gate that should have caught all three asked whether a FILE mentions levelid
rather than whether each position does. It now checks per occurrence.
Also: printers can be assigned to a MACHINE and reach whichever PC controls it,
so a reimaged bay reinstalls its own printers with nothing saved off the old PC;
printerdrivers can name a vendor, so two rows cover 41 of 44 printers instead of
twenty-one near-duplicates; and the collector reporter and EventSaver now live in
the repository with no site baked into either, the reporter generated per site on
request.
The client scripts were validated on Windows 11 against a live ShopDB, not only
by the suite: a bay with no rows of its own created both queues from its
machine's assignment, bound them to the right universal drivers, and set the
per-user default.
The version and the changelog are the release; the detail is in the entry.
Two rows now cover 41 of 44 printers. printerdrivers could only bind a driver to
ONE modelnumberid, so the HP and Xerox universal drivers - which between them
cover almost the whole floor - would have needed 21 near-duplicate rows pointing
at the same package. That is a table nobody keeps true, and it is why 42 of 44
printers resolved no driver at all.
printerdrivers gains vendorid, and resolution runs most-specific-first: the
printer's model, then its vendor, then the pre-vendorid convention of matching
the vendor word in the driver's name so a site that populated the table before
the column existed does not silently lose every driver on upgrade. A row that
names a vendor is never matched by its text, because a mis-set vendor resolving
to the wrong package is worse than resolving to none.
Six rows now resolve 44 of 44 printers at the reference site, and the DesignJet
correctly takes its own driver over the HP universal one.
Set-ShopdbPrinters.ps1 is the client half: ask for-host, create the queues that
are missing, record the desired default. It NEVER removes a queue - a bad minute
from the API must not take printers away from a working bay - and it never
fetches a driver, because downloading 48 MB while somebody waits to print is the
wrong moment. The common scope stages those.
Apply-ShopdbDefaultPrinter.ps1 applies the default in the USER's context, which
is the only context that can: SYSTEM cannot set a per-user default for somebody
else. It also turns off "Let Windows manage my default printer", without which
Windows silently overwrites the choice the next time anyone prints elsewhere -
a fix that undoes itself within a day.
VALIDATED ON WINDOWS 11 AGAINST A LIVE SHOPDB, not only by tests. Printers were
assigned to a MACHINE; a PC controlling it, holding no rows of its own, created
both queues bound to the right universal drivers, recorded the default and set
it, and a second run changed nothing. The first attempt failed with
"Relationship types are not seeded - run: flask seed reference-data", which is
the deployment trap the plan predicted, caught by an explicit error rather than
silently resolving nothing.
Printers belong to the bay, not to the box currently driving it. The assignment
goes on the MACHINE asset and reaches whichever PC controls it, so a reimaged or
swapped PC comes back with the right printers and nothing had to be saved off the
old one. The asset register is the backup.
New relationship type usesprinter ("this printer is installed here"), beside the
existing defaultprinter ("which of them is the default"), both seeded and both
given a propagation rail through controls. The rails are consumed at READ time
only: the create-time fan-out skips directional through-types, and controls is
directional, so assigning a printer to a machine does not copy rows onto its PC.
That is what keeps own-beats-inherited possible.
Resolution for a PC is its OWN rows if it has any, otherwise one hop out along
controls to the machines it drives. Whole set at a time, not merged: a PC with
its own assignment is overriding the bay deliberately, and the UI has to say so
or a tech "fixing" a bay by editing the PC will shadow the machine's record and
wonder why they keep disagreeing.
GET /api/printers/for-host/<hostname> is what the convergence client asks every
cycle. Resolved by hostname because the collector upserts PCs by hostname and an
office PC has no machine number. An unknown host, a site without the computers
plugin, and nothing assigned all return an empty set - that is the client's
designed no-op and it must stay indistinguishable from "assigned nothing".
PUT /api/printers/assignments/for-asset/<id> reconciles the whole set in one
call. The endpoint was specified, documented and asserted by three tests, and
never written - the verification pass caught that, with four failures. It
validates the default BEFORE any write, so a rejected request changes nothing;
soft-deletes rows that went away; and REACTIVATES soft-deleted rows rather than
inserting, because the unique constraint spans inactive rows and a blind insert
after an unassign raises IntegrityError on MySQL while passing on SQLite.
One default per asset, enforced here because the schema cannot: the constraint is
(source, target, type), which accepts two different defaults quite happily. Two
active defaults are still reachable through the generic relationships endpoint,
where the oldest silently wins - recorded in the proposal as the next thing to
close.
printerdrivers gains drivername: the exact string the INF declares, which
Add-PrinterDriver matches on and nothing else. Deriving it by parsing INFs on
hundreds of bays is fragile; a human confirming it once is not.
Assigning a printer to a bay is useless if the bay cannot install it, and the
fleet data says why that mattered: 42 of 44 printers could not resolve a driver.
This is the delivery half - the drivers themselves, staged once per bay, so that
creating a queue never waits on a download.
Install-ShopdbPrinterDriver.ps1 does one driver: trust the package's signer, then
pnputil /add-driver, then Add-PrinterDriver. Install-ShopdbPrinterDrivers.ps1
does a site's whole set from drivers.json, and answers a compliance question with
-TestOnly, which is what makes it a clean DSC Script resource rather than a
fire-and-forget install.
Deliberately SEPARATE from assignment. Drivers are large, near-identical across a
fleet and change rarely; assignments are small, per-bay and change often. Staging
the set in the GE-Enforce common scope means the assignment client only ever
creates a queue - it never fetches a 48 MB package while somebody is waiting to
print, or discovers the share is unmounted at the worst moment.
THE SIGNER TRUST STEP IS THE WHOLE TRICK, and it took a real driver to find it.
certutil -addstore on the .cat file satisfied the Xerox package and failed every
HP INF with "The publisher of an Authenticode(tm) signed catalog has not yet been
established as trusted" - a coin toss, not a mechanism. The certificate is now
extracted with Get-AuthenticodeSignature and added to Trusted Publishers, for
every catalog under the package rather than the first INF's neighbours. On a
locked bay there is no prompt to answer, so the old failure was silent.
Verified on Windows against real packages, not by reading: all six drivers this
site needs install through the script, a second run is a no-op, a wrong driver
name fails with the names the package actually offers, and the DSC cycle behaves
- TestOnly exits 1 on a clean box, install exits 0, TestOnly then exits 0.
The packages themselves stay out of git: they are licensed vendor binaries, and
they belong on the share beside the other imaging payloads.
DEPLOYING-DRIVERS.md carries the GE-Enforce entry, the DSC configuration and the
Intune shape, plus the constraint that has cost a session before: the SFLD share
is mounted only during the enforcement cycle, so this runs as a manifest entry
and never as its own scheduled task.
A site adopting ShopDB had to be handed two files and told what to edit in them.
Both are now the product's, and one of them the server writes for you.
GET /api/computers/client-script (admin) returns Report-AssetToShopDB.ps1 with
this site's values already in it: site_base_url becomes the -ApiUrl default and
the new computers_routableranges setting becomes -AllowedRanges. Only the
PARAMETER DEFAULTS are substituted - the copy in plugins/computers/client/ stays
runnable, so there is no second version to drift from the first - and everything
stamped stays overridable by argument or registry, because a bay may need to
differ from its site. Settings > Computers > Asset reporter edits the ranges,
downloads the script and shows its SHA-256.
The collector key is deliberately not stamped in, and a test fails if it ever
is. That file lands on every shop-floor PC, and a token spread across hundreds
of bays cannot be rotated quietly; it stays in the registry, provisioned per
ADOPTING-AT-ANOTHER-SITE.md.
The routable ranges are the last thing that was hardcoded in that script. They
are now a setting, so West Jefferson's two CIDRs move out of source code and
into that site's own configuration - which is what ADR-015 asks for - and a site
that sets nothing still works, because the script falls back to the NIC carrying
the default route.
EventSaver joins it in plugins/slides/client/, source only: EventSaver.cs and
EventSaver.ini, no compiled .scr - a binary is a release asset, like the
installer exe. The share path that was compiled into Config.Folder is gone. It
used to be the fallback when the ini was missing, which silently pointed a new
site at the reference site's file server; it is now empty, and failing visibly
beats displaying another site's slides. Verified by compiling the edited source
in the Windows VM with the in-box csc.exe: 15,872 bytes, exit 0.
Also: the DSC example in the adoption guide gains a CollectorRanges resource and
stops passing -ApiUrl to a script that already reads BaseUrl from the registry
the same example writes, and the guide points at the generated download instead
of hand-editing a URL.
The contract test caught the endpoint importing shopdb directly for the version
string, which ADR-002 forbids a plugin from doing. The product and contract
versions are in app.config now, which a plugin reads through current_app.
Adds docs/proposals/printer-assignment.md: assign printers to a PC in ShopDB and
let the bay install them, with what the fleet data says about drivers - HP and
Xerox cover 41 of 44 printers with universal drivers, there are no Brother
printers at all despite 208 files of Brother inkjet drivers in the installer,
and printerdrivers holds one row pointing at a per-model folder instead of a
universal driver.
Report-AssetToShopDB.ps1 lived on one site's imaging share and was, per the
adoption guide, "provided on request" - which is not a distribution mechanism for
a product meant to be adopted. It now lives in plugins/computers/client/, beside
the collector contract it implements (ADR-006), so the two version together.
Three things named West Jefferson and no longer do (ADR-015):
- The server. It resolves from HKLM:\SOFTWARE\GE\ShopDB BaseUrl - the value
Install-GEEnforce.ps1 already writes and the enforcement client cannot run
without - or from -ApiUrl. With neither it logs and exits 0 rather than
posting somewhere wrong. Any bay running this script runs the enforcement
client, so the value is present wherever it is deployed.
- The corporate VLANs. Two hardcoded CIDRs decided which NIC's address was
reported, with a comment reading "update if site re-VLANs". A site may now
name its ranges (-AllowedRanges, or a CollectorRanges registry value); with
none configured the NIC carrying the DEFAULT ROUTE is used, which expresses
"the routable NIC, not the controller NIC" without knowing any site's
addressing.
VERIFIED IN THE WINDOWS VM, not by reading it - and the VM earned its keep. The
local array was called $allowedRanges, which is the SAME VARIABLE as the [string]
parameter $AllowedRanges because PowerShell names are case-insensitive; the array
was silently coerced to an empty string, and .Count on a scalar string is 1. The
script therefore believed a range was configured, skipped the default-route
fallback, and reported no IP at all. Linux pwsh parsed it happily. Renamed to
$rangeList, and the four paths were then confirmed on Windows: no config skips
cleanly, BaseUrl resolves from the registry, an unconfigured site picks the
default-route NIC, a configured range selects or excludes as asked.
The hover mini-map said "This asset has a position (2835, 1410) but no level"
for every asset in the product. When 0.11.0 gave LocationMapTooltip a levelid
prop, NONE of its seven call sites were taught to pass one - printer, machine and
PC detail pages, the toner report, enforcement reports, the warranty chip and the
dashboard cards - so the component correctly reported a missing level and the
preview never drew. Two payloads behind those views also emitted mapx/mapy with
no level: the toner report and the enforcement report.
The map PDF export had the ORIGINAL bug still in it: it plotted every filtered
asset onto the sheet, so exporting the ground floor printed second-floor markers
on it. Worse than on screen, because nobody can correct a sheet once it has been
printed and carried onto the floor. It now exports only the level being viewed.
The legacy import loader sent mapleft/maptop with no level at three call sites.
That loader is the one still to run against production, and every marker it
created would have been undrawable. It now resolves the site's default level -
the legacy schema predates levels and has one floor plan, so that is what its
coordinates mean.
THE GATE MISSED ALL OF THIS because it asked whether a FILE mentions 'levelid',
not whether each position does: one module emitted 'mapx' six times and 'levelid'
once and passed. It now checks per occurrence, covers scripts/ as well as shopdb/
and plugins/, and fails any Vue file that binds tooltip coordinates without
:levelid. Both new rules were confirmed to fail the build against planted
violations before being relied on.
Printer QR labels: the asset number is no longer printed. A label now reads name
(8201-HPLaserJetPro), QR, FQDN, then IP. The name falls back to the assetnumber
because that is where sites actually keep it - every printer here has an empty
name field, so preferring the Windows queue name alone would have printed a blank
line on every label.
Bug fixes for the buildings-and-levels work in 0.11.0, every one of them found by
using the feature rather than by the suite.
Anyone running 0.11.0 or 0.11.1 with more than one level should take this
release: the map drew markers from every level on whichever floor plan happened
to be showing, which is the failure ADR-017 exists to prevent, and it was in the
one component that draws the map. The viewer also had no way to choose a level,
and the editor never accepted a click at all - its handler was bound only if the
map was already a picker at mount, and the editor opens with nothing selected.
Also: a level's native size could not be set (the settings page had no height
field), the same search could return different rows because fourteen searchers
truncated without an ORDER BY, and both map pages carried a scrollbar from
subtracting the wrong page chrome from the viewport height.
No schema change, and the plugin contract stays at 0.20.0.
The version and the changelog are the release; the detail is in the entry.
The previous commit stopped the panel header from colliding with the "Assets"
heading, but left the search box sharing a row with the type dropdown: 139px of
input for a placeholder needing 216px, so it still read "Search name o". A
control whose own label does not fit is not a narrower control, it is an
unlabelled one.
One control per row in a 318px panel. Measured in the browser: the input is now
286px against 216px of text, so the placeholder reads in full.
Clicking the map in the editor did nothing, ever. The click handler was bound
inside `if (props.pickerMode)` at mount, and the editor mounts with no asset
selected, so the handler was never attached; selecting an asset flipped the prop
but nothing rebound it. The asset forms were unaffected because their picker
mounts inside a modal that is already in picker mode, which is why this looked
like an editor-only fault. The handler is now bound unconditionally and
handleMapClick keeps its own picker-mode guard.
Verified in a browser against the dev instance, both ways: with the old binding
a click on a selected asset produced no position at all; with the fix the same
click reports 1652, 1138.
Markers are now drawn in picker mode too. Placing one relative to the machines
already on the floor is the entire task, and the old code skipped rendering them
whenever the map was a picker.
Two things that looked like stray widgets:
- The editor's panel header put a heading and three controls on one row inside a
320px panel, squeezing the search box until its placeholder read "Search na".
The heading takes its own row and the controls share the next.
- The legend drew its bar and border even with nothing to put in it, which read
as an empty input box under the toolbar. It renders only when it has entries.
Five defects, four of them mine from 0.11.0, found by using the feature.
THE SERIOUS ONE: ShopFloorMap never checked a marker's level. It skipped null
coordinates and drew everything else on whatever blueprint was displayed, so
level 1 markers appeared on level 2 - the exact failure ADR-017 exists to
prevent, in the one component that draws the map. The build gate did not catch it
because that rule checks files EMITTING mapx, not the component consuming it.
Markers are now filtered to the drawn level, and a position with no level is
omitted rather than approximated.
The map page had no way to choose a level at all. It read currentlevelid only to
title the PDF, so a second floor was unreachable from the viewer that most people
use. Adds a level selector (hidden when a site has one level), passes it to the
map, and switches the drawing, the bounds, the coordinate space and the markers
together - swapping the image without the bounds would place every marker against
the wrong scale.
Searching the map now follows results across levels: a search whose matches are
all on another floor showed an empty map while the filter counted them.
Floor map settings had NO height input - only width - so a level's native size
could not be set even while empty, which is the one time it is editable. Both
fields are there now, and size is editable on a level that has markers, because
refusing it blocked the case the feature was built for: a new blueprint of new
dimensions on a floor already full of markers. It confirms first and points at
landmark recalibration.
Search results differed between the sidebar box and the results-page box:
- 14 of 16 searchers truncated with .limit() and no ORDER BY, so the database
could return a DIFFERENT subset of matching rows for the same query. Every
searcher now ends in a total order (display key plus primary key).
- Searching a term already in the URL was a duplicate navigation the router
aborts, so the route watcher never fired and the button did nothing. The
sidebar never hit this, because it always navigates from another page - which
is why the two boxes appeared to disagree.
Also removes a scrollbar from both map pages. They subtracted 2rem and 40px from
100vh for the page chrome, which is really 90px of padding on .main-content, so
each overflowed by the difference. The padding is now a CSS variable both the
layout and the pages read. Measured in the browser before and after: 1058 vs a
1000px viewport, now 1000.
A patch release. The pagination-cap fixes, and the documentation that missed the
0.11.0 installer by a few minutes.
docs/UPGRADE.md is one of the documents the Windows installer puts on the server,
and its buildings-and-levels section was committed after 0.11.0 was tagged. So
0.11.0 shipped the page an operator reads WHILE upgrading without the guidance
about that upgrade - which matters most on an air-gapped box, where there is no
other way to reach it. The installer now also ships FLOOR-MAP.md for the same
reason the Windows runbooks ship.
No application code changed on the server side and the plugin contract stays at
0.20.0.
The version and the changelog are the release; the detail is in the entry.
Follow-up to the application-picker fix. Three of these were already wrong on
data that exists today, not merely latent.
Batch label printing is the worst of them: AssetLabelBatch asked for 500
machines or PCs, got 100, and printed a sheet that looked complete. With 262
machines and 290 PCs in the catalogue that is a physically short run with no
error anywhere - the operator finds out at the label printer, or later at the
bay with no label on it. PrinterQRBatch, USBLabelBatch and PrintedPartsLabels
had the same shape and are fixed alongside it, before their tables cross 100
too.
MachineForm's "controls" PC dropdown offered the first 100 of 290, so a
machine could not be linked to a PC sorting late in the list. NetworkDeviceForm
had it for models, which are already past 100 - and the same file already
called modelsApi.listAll() correctly two lines away.
Adds listAll() to the machines, computers, printers, network, measuring-tools,
USB and printed-parts APIs, all delegating to fetchAllPages().
Still outstanding: callers of vendors, locations, business units and the type
catalogues that ask for more than 100. Those tables are all well under the cap
today, so they are correct for now and wrong the day they are not.
get_pagination_params clamps perpage to MAX_PAGE_SIZE (100) and reports
nothing about having done so, so a caller asking for perpage: 1000 gets the
first 100 rows and a success response. Every picker built that way looked
complete and was not.
Found on a live site with 126 active applications: the 26 sorting last were
absent from the knowledge-base topic dropdown, so an article could not be
filed against them. Nothing was wrong with those application records, and
editing them could never have helped.
Adds fetchAllPages() to the api module, generalizing the one call site that
already handled this correctly (modelsApi.listAll), and points the four
application pickers at a new applicationsApi.listAll(): the KB article form,
the KB list's topic filter, the notification form, and the report filter
builder.
Lists that render a page at a time are untouched - they page for a reason.
Other callers still asking for more than 100 rows of vendors, locations,
models, subnets and the rest are latent: correct only while those tables stay
under 100, and silent on the day they do not.
The 0.11.0 release changed what a map position means and six documents still
described the model it replaced. Each of these could have caused a real mistake
rather than being merely out of date:
- IMPORT-API mapped legacy mapleft/maptop to mapx/mapy with no mention of the
level, so a scripted import - including the classic-ASP one still to run
against production - would have produced markers the map shows as "level
unknown". It now maps levelid too and says how to resolve the default level.
- API-REFERENCE enumerates the unauthenticated surface in full, because that is
what a deploy reviewer reads, and the three public /api/maplevels reads were
missing from it. Also records why the write split is asymmetric: repositioning
needs assets.edit, creating a level needs admin, since a level's dimensions are
the coordinate space every marker on it is expressed in.
- CONFIG still presented the four map_* settings as live, telling the reader to
re-upload a blueprint in a settings page that no longer drives the map. They
are marked superseded and kept for downgrade.
- UPGRADE gained a 0.11.0 section: nothing moves on screen, and replacing a
blueprint with one of different dimensions moves every marker on that level, so
recalibrate from landmarks rather than editing width and height.
- PLUGIN-HOOKS now states that a map overlay keys on assetid and must not return
coordinates or a level - a second copy of a position is one that can disagree.
Adds FLOOR-MAP.md, the operator's page: loading a plan, placing markers, and
what to do when the plan changes, with the reasoning left in ADR-017. START-HERE
routes to it from the new-site path, and specifically as the page to read BEFORE
a floor plan changes.
The floor map became a set of drawings instead of one picture, and search now
reaches what the forms have been collecting all along.
A site can hold more than one building and a building more than one level, and
every map position records which drawing its coordinates belong to. An existing
single-map site upgrades into one building and one default level carrying the
blueprint it already had, and nothing moves on screen. Landmark recalibration
moves a level's markers onto a redrawn plan, solved per axis from the landmarks
and never from the image dimensions, dry run by default, snapshotted before it
applies.
Two identifiers the UI collects were findable in almost no way and two plugins'
records in no way at all: gaugelabreference matched only measuring tools,
maintenancereference matched nothing anywhere, and USB devices and printed items
were absent from search entirely because neither is an asset. All four are
searchable now, from the global bar and from every list's own search box.
Contract 0.20.0 is additive: a plugin that writes a map position must write its
level with it.
The version and the changelog are the release; the detail is in the entry.
gaugelabreference and maintenancereference are collected on the form and shown
on the detail page for all five asset types, and no list's ?search= matched
either. Someone holding a physical tag could read it off the machine, type it
into the list they were already looking at, and get nothing back - while global
search, fixed in the previous commit, found it.
The clause is copy-pasted once per plugin, which is how all five came to omit
fields their own forms collect, so the test is one parametrized pass over all
five lists rather than five per-plugin tests that would drift the same way. It
also pins that widening the clause did not turn the box into a pass-through.
The api-inventory entries enumerate the fields each search matches, so all five
were stale the moment the queries changed; updated with the OpenAPI spec.
Not touched: the collector does not send either identifier, correctly - a
lab-assigned tag is not something an agent on a PC can discover. The legacy
import loader does not map them either, and the classic schema has no
gauge-named column to map from.
The map was one picture of one floor. A second floor was added, the blueprint
changed size, and machines moved, so a position now records WHICH DRAWING its
coordinates belong to.
Buildings and levels (ADR-017). Each level owns its blueprint per theme and its
own native pixel size; assets.mapx/mapy are pixels of assets.levelid, not of the
site. A position whose level is unknown renders "level unknown" and is never
drawn on the default level, because a marker on the wrong floor plan looks
entirely correct while pointing at the wrong place.
Repositioning in bulk: filter by unplaced, needs-review or level, search, place,
confirm. Landmark recalibration solves the transform PER AXIS from landmark
pairs and never from image dimensions - the canvas grew taller without
rescaling, so a dimension-derived scale would stretch Y by 1.57 and be wrong
everywhere. It defaults to a dry run, reports what would land off the drawing,
snapshots before applying, and clears mapverifiedat because a transform is a
guess awaiting review. Snapshots restore, including the level and the review
state, and a restore snapshots first so an undo is undoable.
Search: gaugelabreference was matched only for measuring tools and
maintenancereference was matched nowhere at all, for any asset type, while
Settings happily offers both identifiers on machines and PCs. A tag an operator
is told to record has to be findable or it is a write-only field. USB devices
and printed items were unreachable from search entirely - neither is an asset,
so the generic asset search could not see them and no searcher existed; they
now match on serial, asset tag, label, bin code and gage-lab tag, honouring
isactive, with Settings toggles and result labels to match.
The retired-application rule was half a rule: GET /api/knowledgebase hid
articles whose topic application is retired while global search still returned
them and printed the retired application as the subject. A filter is only real
if every path that reaches the row applies it.
Contract to 0.20.0 (additive): Asset gained levelid and mapverifiedat, Location
gained levelid, and resolve_asset_position returns the levelid belonging to
whichever source supplied the coordinates. The five plugins that write a map
position are re-pinned. The install-list text format gained levelid as a NINTH
field, appended, because the shipped Pascal installer reads fields 0-7 by index.
That installer still compiles in one drawing's dimensions and bundles one
blueprint, so its map is accurate for the default level only; /api/maplevels is
deliberately unauthenticated so it can read both at runtime once rebuilt.
Recorded in PRINTER-INSTALLER.md section 6 along with the other known gaps.
Migration 7d33 converts an existing single-map site into one building and one
default level carrying the old map_* settings, then assigns every placed asset
and location to it. Nothing moves on screen. Old settings rows are kept so a
rollback still finds them. Verified end to end on MySQL 5.6 from a
production-shaped database.
A privilege-escalation fix in the Windows installer, the multi-site portability
work, and a documentation and API surface that a second site can rely on.
The installer created C:\shopdb-flask, C:\Python314 and C:\MySQL84 without
breaking inheritance from the drive root, whose default DACL grants
Authenticated Users an inherit-only Modify. Two files inside those roots are
executed by someone else - shopdb-admin.ps1 self-elevates, and instance
config.py is loaded unconditionally by the app - so any logged-on user could
replace either and wait. Verified fixed as SYSTEM in the Windows 11 VM against
the shipped functions.
Contract 0.19.0 is breaking: get_dashboard_widgets returns data and shape, not a
component name. The change itself shipped earlier without a bump, with the
contract docs still teaching the dead shape; both are corrected.
The version and the changelog are the release; the detail is in the entry.
It claimed thirteen bundled plugins and listed thirteen; there are fifteen, and
the two missing ones - backups and tools - were undiscoverable from the first
page anyone reads. The count is gone rather than corrected, because a number
typed here goes stale exactly the way it just did; the generated map carries it.
The version and count guard now covers README.md and CONTRIBUTING.md, not just
docs/. A rule that leaves the most-read file in the repository outside it is
most of a rule.
Also names the reference site as the reference site rather than by plant, adds a
pointer to START-HERE, and corrects the distribution paragraph: it told a reader
of the public mirror that the code is only available on an internal server,
which is both wrong and unhelpful now that the installer ships as a release
asset.
The export gate now carries the site patterns, and three kinds of file tripped
it - two legitimately, one by construction.
Two test files held real internal subnets as fixtures. They are documentation
ranges now, which test the same logic and disclose nothing.
The project-map generator and a changelog entry named a file that is excluded
from publication, so a public reader was pointed at something they cannot see.
Both now describe what happened without naming it.
And the naming script has to CONTAIN the site patterns in order to grep for
them, so written literally the rule's own definition fails the gate that
enforces it. The patterns are assembled from fragments, the same trick the docs
publishability test already uses for the same reason. Verified the hard way: a
planted literal is still caught, so the fragmentation did not quietly turn the
rule into one that matches nothing - which is the obvious way for this fix to
have gone wrong.
DEPLOY-WINDOWS-IIS was a second copy of the manual IIS procedure that had
diverged from the first: a different MySQL version (8.0, which reached end of
life in April), a different port, a different plugin list, and a profile file
that does not exist. Two runbooks for one procedure means a reader follows
whichever they found, and one of them was wrong. INSTALL-WINDOWS-IIS covers
everything it did plus a preflight step and the subpath method, so the one
section it uniquely had - redeploying a hand-built server - is folded in there,
with the plugin-chain step it was missing and a note to back up first, and the
duplicate is gone. Everything that pointed at it now points at the survivor.
Three ADR statuses said something untrue.
ADR-013 said PROPOSED while half of it had shipped and ADR-014 had been accepted
on top of it. A decision that has been implemented and depended upon is not
proposed, and leaving one that way devalues every other status in the index. The
catalog half is still unbuilt, which is the ordinary state of an accepted
decision: accepted means settled, not delivered.
ADR-016 said ACCEPTED for a design where nothing is built - the endpoint and
permissions it describes do not exist, so a reader goes looking for them. The
status stands, because the decision does; the header now says so plainly and
points at where today's credentials actually live.
ADR-003 and ADR-004 were ACCEPTED with their own Decision lines still opening
"**PROPOSED:**", which reads as though the decision was never taken.
And the dashboard proposal carried Status: ACCEPTED, which belongs to a decision
record. A proposal is a proposal; the contract it produced is the ADR.
Four gaps a second site hits and cannot resolve by reading.
**Restoring on Windows** was one sentence - "the standard mysql < dump.sql" -
with no ordering. Restoring a database under running code that expects a
different schema turns a restore into a second incident, so the steps are now
ordered and each says why. It also says what `.env` costs if it is lost, which
is the part nobody discovers until they are already rebuilding: the dump does
not contain it, and without the JWT secrets every issued token dies, so every
collector and every GE-Enforce client on the fleet needs a new key.
**Rolling back** had a paragraph saying downgrades are refused and a backup is
the way back, but not the procedure. Rollback is restoring a matched pair, code
and the schema it expects, in that order - and the doc now separates it from the
case it gets confused with: a migration that failed mid-update has already been
rolled back by the installer, and fixing forward is the only move.
**Sizing, acquisition and support** were absent from the install guide entirely.
A reader could not learn how big a server to ask for, where the .exe comes from,
or where to raise a problem. The sizing is small and the reasons are stated, so
a site does not over-provision a VM for a load that is a few dozen users.
**Credentials** were described in three documents from three ends, so three
answers existed for where a key lives. One table, both ends - server and PC -
plus the two rules behind it: what a shop-floor PC holds is scoped to exactly
what it does, and a credential is delivered rather than typed, because a value
entered per machine is a value that is wrong on some machine.
The wiki had no entry point worth the name. Its Home page was hardcoded inside
the generator, had drifted into recommending INSTALL-WINDOWS-IIS - the manual
procedure these same docs tell you not to follow for a new site, because it
produces a server the installer then refuses to upgrade - and the page carrying
that warning, llms.txt, was never published at all, because the generator copied
only .md files. The adoption guide had zero inbound links.
START-HERE routes by what a reader came to do: stand up a site, deploy the
shop-floor tools, write a plugin, integrate with the API, understand a decision,
or fix something. It is the shortest correct path per role, not an index - the
sidebar is already the index.
FLEET-ARCHITECTURE is the page nothing else could assume. The server, GE-Enforce,
the asset reporter, the backup collectors and EventSaver were each documented
alone, each assuming a reader who already knew the other four. It draws the
shape, states the rule that explains most of the behaviour - nothing on the
server reaches out to a PC, the fleet asks - and ends with the table that says
which of the five programs to open for a given symptom, since knowing that is
most of the diagnosis.
wikigen renders START-HERE as the landing page rather than a list maintained in
a second place, and publishes llms.txt as LLM-GUIDE.
The inventory is hand-maintained, and should stay that way - its value is the
prose, and what an endpoint is FOR cannot be derived from the code. An audit of
all 372 documented operations found zero phantom routes and zero wrong parameter
names, so the maintenance is in good order.
What hand-maintenance cannot do is notice a route somebody added. Twenty-two
were undocumented: the entire backups plugin surface, every one of the dashboard
card endpoints added with contract 0.19.0, the GE-Enforce publish preflight, the
employee SSO resolver, the protocol update verbs, and the four /api/docs routes -
so the spec did not describe how to fetch the spec.
Coverage is now a test. It walks the live url_map and fails when a served route
has no entry, which means adding an endpoint includes describing it, in the same
commit, while the author still knows what it is for. The reverse direction is
checked too: a documented route that no longer exists sends a reader to a 404.
Writing that test found one more thing. The inventory writes multi-verb routes
as "PUT|PATCH", and neither the parity check nor the generator split on the pipe
- so those operations were absent from the published spec entirely, with nothing
reporting it. The spec now carries all 394 operations the code serves, which is
the first time the two numbers have matched.
The generator's own docstring claimed the inventory could be regenerated. It
cannot; nothing generates it. That sentence is why nobody noticed it was falling
behind.
The generated spec carried no `parameters` and no `requestBody` on any of its
372 operations. That is invalid OpenAPI 3.1, and the practical cost was worse
than the formal one: the MCP server builds its tools from this file, so every
tool had an empty input schema and silently dropped whatever the caller passed.
A request for one asset returned the list, and nothing anywhere reported an
error. All 118 templated paths now declare their path parameters, typed from the
Flask converter that named them, and write verbs declare a JSON body.
The body is an open object carrying the prose description rather than an
invented schema. The inventory describes bodies in sentences, and a field list
this generator guessed at would be worse than none - but "an object, described
here" is the difference between a client that can send a body and one that
cannot send anything.
Security was wrong on 123 operations. `jwt-optional` means "works logged out,
returns more logged in", which OpenAPI expresses as the empty requirement
alongside the scheme; publishing them as bearer-required told every reader that
a public endpoint needs a token.
Responses were one hardcoded 200, so a generated client had no idea a call could
fail. Every operation now documents the error envelope - and the envelope itself
is a defined schema, because its error nests under `data.error` rather than at
the top level, which is the single thing people get wrong when writing against
this API.
95 summaries were cut at 120 characters mid-word, which is what a tool picker
shows a user as the whole description of a call. They now end on a word.
Tests pin the shape rather than the prose. One of them contradicted an older
test that REQUIRED the contract version as a literal in PLUGIN-HOOKS.md - the
same copying that left nine documents stale - so that test now asserts the doc
points at the generated map instead.
Six procedures that could not be followed as written.
Eighty-nine curl examples single-quoted `Authorization: Bearer $TOKEN`, so the
shell never expanded it and the server answered 422 "Not enough segments". Nine
more did the same with X-API-Key. The other 129 examples in the same file
already used double quotes, so this was drift rather than a convention, and the
spec regenerated from it carried the fault onward.
The GE-Enforce report example put a `//` comment inside a JSON body. The server
parses with silent=True, so it saw `{}` and answered "hostname is required"
about a body that plainly has one - the worst kind of error message, one that
sends the reader to the wrong field entirely.
The IIS install ran `flask db upgrade` and a per-plugin install loop but never
`flask plugin upgrade-all`, leaving every plugin's own chain unapplied. That is
precisely the 1054 "Unknown column" a deploy then hits somewhere else, days
later, on the page that uses the new column.
The pilot runbook looped `flask plugin enable` over plugins that were not yet
installed; enable refuses those, so on a fresh database it exited 1 on every
iteration and enabled nothing. ADR-013 had already recorded that defect.
`apply-profile` installs and enables in dependency order, which is what the step
was reaching for.
DEPLOY-WINDOWS-IIS named a profile file that does not exist; the shipped ones do.
And PLUGIN-EXTERNAL-REPO never mentioned PLUGIN_TABLE_OWNERS, while the
migration engine raises for any plugin missing from it - so the guide's own
step 5 fails for any external plugin that owns a table. That the registry lives
in the framework repo is deliberate, so the guide now says so, and says what it
costs: a table-owning external plugin is a two-repository change, and a plugin
that owns no tables avoids it entirely.
Nine documents carried a hand-typed contract version and every one was stale.
One was load-bearing: PLUGIN-EXTERNAL-REPO.md told an external author to pin
">=0.13.0,<0.14.0" while the contract is at 0.19.0, so a plugin built by
following that guide is refused by the loader at startup. The plugin count was
wrong in six more.
They now point at docs/PROJECT-MAP.md, which is generated. A test enforces it:
no document may declare a version literal, a stated current version must match
the code, and a stated plugin count must match the tree. ADRs are exempt from
the current-version rule, because an ADR states the version a decision was taken
AT - that is a record of the past, and rewriting it would falsify the record
ADRs exist to keep.
CONTRACT-STABILITY.md was missing 0.17.0, 0.18.0 and 0.19.0 - including the only
BREAKING change in the series - in the one document a site reads to choose its
pin. All three are recorded, with 0.19.0 called out: it took something away, and
it shipped before it was written down, which is the argument for pinning tight
rather than trusting that a minor bump is safe.
The publishability gate caught internal tooling names and developer paths but
nothing site-specific, so roughly sixty leaks reached the wiki: the site name in
ten documents, real fleet hostnames in the collector and GE-Enforce examples, an
internal database name through the whole import guide, imaging-share paths, and
a maintainer's username as the Deciders line of every ADR and inside a generated
curl example.
None of it is a security matter on an air-gapped fleet. It matters because these
pages are read by engineers at other plants, and a document that names one site
throughout reads as that site's notes rather than a product's documentation -
which is exactly what it then gets treated as.
Examples now use neutral hostnames, the site is "the reference site" where the
distinction carries meaning, and ADRs are decided by "ShopDB maintainers". The
gate carries all of these patterns, so the next one fails a build.
Two documents leave docs/ because they were never written for an outside reader.
PROJECT-REVIEW.md is an internal health memo pinned to a commit from July, whose
headline finding (an untracked playbook) has since been fixed - it is history,
and git holds it. PILOT-DEPLOY.md is one site's own cutover runbook, complete
with a "re-measure before publishing" placeholder; it moves next to the loader
it belongs to, in scripts/site_imports/wjf/.
ADR-015 is AMENDED rather than rewritten. Its enforcement section still said
report-only and its backlog still listed hardcodes that are now cleared, which
left the record contradicting itself. The amendment says what changed and why
the report-only period ended; the original text stays, because what the decision
looked like when it was taken is the part worth keeping.
Also corrects llms.txt's response envelope, which had errors at the top level
and pagination at meta.total. Both are nested one deeper, so anything written
against that description read undefined on every error it tried to handle.
An ACCEPTED ADR ended with a stray tool-call fragment - it said nothing to a
reader and everything about how the document was produced, which is not what a
decision record is for. And the cutover playbook carried a real internal IP in a
file otherwise correctly scrubbed to documentation ranges. Both were published.
The publishability gate now catches all three shapes, so the next one fails a
build instead of reaching the wiki.
The adoption guide also told a site to copy two files that ship from nowhere:
the asset reporter and EventSaver are both on the reference site's imaging
share, not in this repository, and the guide never said so - it could not be
followed as written. It now states, per artifact, where each comes from, which
of them are pending a move into this repository, and that EventSaver can be
rebuilt from its one source file rather than taken on trust as a binary.
EventSaver was undocumented anywhere a reader could find it. What it is, the two
source modes and why HTTP is the one to use, how order.txt and the ShopDB feed
each decide the running order, the three command-line switches Windows calls it
with, where it must NOT run (a screensaver over a live dashboard or a metrology
run is a defect), how to build it with the in-box compiler, and a table of
symptoms with the first thing to check.
The adoption guide also stopped short of the question it invites: the fleet
tools assume GE-Enforce is already on the PC, so how does GE-Enforce get there.
Both real paths are now written down - imaging time, which is how the shop-floor
pipeline does it, and Intune, which is how the display cohort is deployed today
because those PCs are Entra-joined with no share. Machine Configuration is
possible and unshipped, with the caveat that matters: DSC and GE-Enforce are
both convergence loops, so give each a layer to own rather than pointing them at
the same configuration.
Three core pages and four plugin pages each imported qrcode and jsbarcode
directly, and each carried its own answer to the same questions: what margin,
what width, which error correction, how big a module must be before a scanner
can read it. The answers had already drifted - margin 0 in one place and 2 in
another, width 150 against 160 - and on a label that is the difference between
a sticker that scans and one that does not.
frontend/src/utils/codes.js owns it now: the label-stock presets, the quiet-zone
and margin defaults, CODE128 with no printed value, and the printer-resolution
arithmetic that only the Tech Tools generator had. A view passes what is
specific to its own label and nothing else - MachineBadge still asks for CODE39,
because the badge readers predate the shop-floor scanners and decode nothing
else, and that is exactly the kind of thing a call site should say out loud.
views/print/qrLogo.js is folded in rather than left as a second half-shared
helper that only some of the pages reached into.
The check script now fails a build that imports either library outside that
module. Without it this re-forks within a month: the next label page starts by
copying the nearest existing one, which is how it happened the first time.
Tests cover the part no amount of looking at a screen verifies - a QR that
looks fine at 96 dpi on a monitor can be unreadable at 203 dpi on half-inch
stock.
The asset reporter and EventSaver are both already built to be repointed - the
server URL, the API key and the targeting are parameters, an ini file and
manifest targeting, not code. Nothing said so, so the question "can another shop
use this" had no answer that did not involve reading PowerShell.
Worked examples for all three deployment paths, because sites have different
management planes and the choice is not ours to make: Intune (a remediation for
the reporter, a Win32 app for the screensaver, plus a Machine Configuration/DSC
form for estates already governed that way), a GE-Enforce manifest entry, and
manual installation for a pilot or a single bay.
The two traps are written down rather than left to be discovered. EventSaver
falls back to a path compiled into the binary when its ini is missing, and that
path belongs to the reference site - a missing ini is not a neutral default. And
a config enforced by hash reverts a hand edit on the next cycle, which is the
feature working correctly and reads exactly like a bug.
Also notes the reporter's -ApiUrl default still points at the reference site, so
every example passes it explicitly until that is fixed.
CLAUDE.md is read at the start of every session and claimed contract 0.16.0 and
1159 tests while the code was at 0.18.0 and 1567, along with a plugin count and
a chain head that were both wrong. Hand-written facts about a moving codebase
are wrong within a fortnight, and a reader who cannot tell which lines are stale
has to re-derive all of them.
So they are derived. scripts/gen_project_map.py reads the versions, the plugin
inventory, every Alembic chain head, the ADR index with status and the size of
the codebase out of the code, and --check fails a build where the committed map
no longer matches. CLAUDE.md now points at it, leads with the multi-site rule,
and lists the gates to run instead of restating the conventions they enforce.
The changelog's Unreleased section covered 8 of the 46 commits since 0.9.0 and
had no Changed, Fixed or Security sections at all. It now carries the whole
fortnight, including both contract bumps - which had never been recorded even
though ADR-002 makes contract versions their own series.
The scanner has been reporting the same count for weeks, which is what a rule
that only prints becomes. It now FAILS the build, and it looks where the leaks
actually were: PowerShell, the installer, the seeds, generated JSON, the
frontend - case-insensitively, across plugins, shopdb, scripts, deploy, tools.
A line that is deliberate declares itself with an ADR-015-OK marker and a
reason, so the claim is visible in review instead of tolerated in silence.
What it found, fixed here:
- The shadow client wrote one site's ShopDB URL into HKLM whenever the registry
disagreed. At the site it was written for that reads as healing drift;
anywhere else it overwrites the site's own address on every enforce cycle,
and the site cannot win because the cycle repeats. The bay's value now wins,
an explicit -BaseUrl seeds it, and with neither there is nothing honest to
write, so it says so and skips.
- The kiosk dispatcher fell back to one plant's host when HKLM was unset, so a
kiosk elsewhere quietly opened a server it has no business reaching. The
fallback is now this site's site_base_url, baked in at seed time, and the
dispatcher refuses rather than guessing when neither is set. Its legacy
shortcut matcher derives the host from that URL instead of naming one.
- The OpenAPI generator hardcoded a production hostname into every spec it
generated, which then published to a public wiki. The relative mount is the
only server it can honestly name; a site passes its own by environment.
- Placeholders and examples in the UI and the client help offered real internal
subnets and a real production URL. They now use documentation ranges.
Both publication gates - the export scrub and the docs publishability test -
carry the site patterns, which neither did. One plant's hostname, FQDN and
internal networks are out of the documentation and the generated specs.
Comments naming the reference site are reworded rather than deleted: the
reasoning is worth keeping, the plant name is not what makes it true.
get_setting on BasePlugin namespaces what it reads to plugin.backups.<key>,
while get_settings_defaults declares - and the settings page writes - the bare
key. So the retention read never found the operator's value and always fell back
to 0, and 0 means keep everything. Retention was configurable in the UI and did
nothing. It is the only place in the codebase using the namespaced helper.
The share root also stops shipping one site's file server as its default. That
put a site's internal topology in a bundled plugin and in the public mirror, and
pointed a second site at a server it cannot reach. Blank now, per ADR-015, and a
share kind with no configured root returns nothing rather than composing a path
from somebody else's hostname - a path built on an empty root is not a lesser
answer, it is a wrong one.
The publish gate exists because a minor version bump that NARROWS behaviour is
not backward compatible: _CmmVersion arrived in lib 2.6, and an older lib does
not know the field, so every gated entry looks unfiltered and it installs every
PC-DMIS version it cannot detect, on every CMM, in one cycle.
It was comparing the fleet's reported library versions against manifestversion.
That is the manifest's own 'Version' field. For a share-imported manifest the
two numbering schemes happen to coincide; for a scope authored in code they do
not, and seed_display_scope writes '2.0' - which every kiosk exceeds. So the
gate passed on the scope that most needed it.
A scope now declares minlibversion. Unset, the requirement is DERIVED from what
the manifest actually uses, so a scope written before this column existed is
still judged on its contents rather than on a number that says nothing about the
library. Only features that narrow behaviour belong in that table; an addition
an old lib ignores harmlessly needs no floor. manifestversion remains the last
fallback, which preserves what share-imported manifests already relied on.
Deleting a relationship is soft, so the row survives with isactive False, and
three things read them without knowing that.
Re-adding a deleted link answered 409 "this relationship already exists" about a
link the page no longer shows, and there was no way forward from the UI at all -
the row cannot simply be inserted again, since the triple is unique.
Reactivating IS the create for an inactive row.
The inverse guard blocked on a deleted inverse, which made "remove the existing
one first" - the instruction in its own message - fail to unblock anything.
fix-controls-direction retired the reversed row whenever a correctly-directed
one existed, without checking whether that one was itself deleted. So it removed
the only live link and reported a successful clean-up. It now reactivates the
row pointing the right way before retiring the one pointing the wrong way.
These are the commands the docs tell an operator to run against production.
An SVG is an XML document that may carry a script, and it is an accepted image
type because floor-plan maps and branding genuinely want vector. Loaded through
an img tag that script never runs, so the tiles and maps were never the risk.
Opening the file's own URL is - and the application image route is public, so
that URL needs no session.
Every route that serves an upload now goes through one helper that sends
Content-Security-Policy: default-src 'none'; sandbox, and nosniff. Seven routes
across core and five plugins, so a new one added later starts from the same
place rather than repeating the reasoning. Banning the format instead would
have cost the maps their only sensible one.
The app also sent no security headers at all. It now sets nosniff,
frame-ancestors self (as X-Frame-Options too, for the display bays' browsers)
and a referrer policy. Deliberately NOT a page-wide CSP: this serves an SPA with
inline styles, so a real script-src policy is a change worth making with the
frontend in front of you, and a permissive header claiming one would be worse
than having none.
Contract 0.19.0. send_upload is on the shopdb.api surface, because a plugin
serving user-supplied bytes should not have to remember these headers. The same
bump records that get_dashboard_widgets has taken data and shape rather than a
component name since the dashboard was rebuilt - that shipped without a bump,
while BasePlugin and PLUGIN-HOOKS.md both still documented the shape nothing
renders, which is how five plugins came to declare widgets pointing at
components nobody had written.
Minting derived a measuring tool's asset number from the HOSTNAME, so a
permanent instrument inherited the identity of whichever PC drove it that week:
replace the PC and either the number lies or a second tool appears for the same
physical unit. And because idempotency was keyed on the collector's own label,
it could not see a tool it had not created - on prod that left 43 legacy MT-####
tools shadowed by minted <HOST>-CMM twins, three records deep in places.
Resolution order is now most-stable-identity-first: the instrument named by
measuringtool-id.txt, then a prior collector link, then a tool this PC already
controls that somebody else created, then the reported machine number, and only
then mint. What minting produces should be read as a placeholder until a real
identifier is recorded.
Three separate 500 loops came out of the same mistake, looking a relationship up
by LABEL when assetrelationships is unique on (source, target, type):
- On a CMM the instrument IS the reported bay, so the machine sync has already
made a row for that exact triple - and it finds its own rows by that label.
Relabelling hid the link, so the next cycle built a second row for the same
triple and MySQL rejected it: 200 once, then 500 forever. The machine link is
now recognised and left alone; adoption only needs the identity.
- A part-marker PC hit it twice over, once on its partof row and once because
the marker's asset number is derived from the PC and could already be taken.
Both are get-or-create on the triple now, and an existing asset of that number
is adopted rather than duplicated.
A named instrument also supersedes a minted twin properly: the stale link is
archived by TARGET, not by object identity, which is what left a PC reading as
the controller of two instruments.
Reported identifiers are matched exactly rather than with ilike. They arrive
from a text file on a shopfloor PC, and ilike reads _ and % as wildcards, so
MT-600_ adopted MT-6001 and a bare % adopted whatever active asset came first.
A named id that is not a measuring tool is refused with a warning rather than
linked as one.
Core's Alembic env got connection.commit() when the stamp bug was found; the
per-plugin template did not. MySQL commits DDL implicitly, which flushes
everything queued before it including the previous migration's version stamp,
and the LAST migration of a run has no DDL after it - so its stamp rolled back
at close while its schema change survived. flask plugin upgrade-all then exited
0 having silently re-run that migration, and re-ran it again on every deploy
after. Invisible for exactly as long as every plugin head happened to be
idempotent.
Two were not.
backups 0003 cleared lastseenat for EVERY row, which is correct once and
destroys evidence on each repeat. It is now scoped to the backfill's actual
signature, COALESCE(collectedat, createdat) - the expression 0002 wrote - plus a
date bound. Both conditions are needed. Matching on collectedat alone misses
every row whose collectedat is NULL, so precisely the rows carrying the most
invented value would have kept it forever; and value equality is not a signature
on MySQL, where db.DateTime is second-precision and the collector writes both
stamps in one statement, so a genuinely fresh revision would read as a backfill
and be wiped. SQLite keeps microseconds, which is why no test could show it.
geenforce 0003 added a column unconditionally, so it failed on a fresh database
built from the models and on any re-run. Guarded like network0003prefix.
A directory created directly under C:\ inherits the drive root's DACL, and the
default carries an inherit-only Modify grant for Authenticated Users. So
C:\shopdb-flask, C:\Python314 and C:\MySQL84 were writable by every logged-on
user on the server.
That is a local privilege escalation here specifically, because two things this
installer puts inside those roots are executed by someone else: shopdb-admin.ps1
self-elevates with -Verb RunAs, and instance\config.py is loaded unconditionally
by the app through from_pyfile. Replace either, wait for an administrator or the
app pool to run it, and the code runs as them.
Hardening is applied at creation and RE-APPLIED on every run, so an upgrade over
an installation that predates this repairs the ACL rather than leaving the hole
in place. The MySQL data directory is locked down too: it holds the users table
and its password hashes, and it inherits ProgramData's read-for-Users.
The app pool is granted RX on the Python root alongside the app root. A venv's
python.exe is a copy but the DLL and the standard library are still read from
the base install, so without that grant the site 500s on every request once the
roots stop inheriting.
A Python installation this installer did not create is reported, not seized:
something else may depend on the ACL it has.
Verified as SYSTEM in the Windows 11 VM against the functions as shipped
(extracted by AST, not retyped): the inherited ACL really does grant non-admins
write; after hardening only Administrators and SYSTEM remain; a service identity
granted afterwards keeps access and later-created files inherit it; re-running
is a no-op; a missing path is silently skipped. 14 checks, all passing.
Every network device on this fleet already follows one convention, applied by
hand: AP-<name>, SW-<name>, SVR-<name>, IDF-<name>. 45 records, no exceptions.
The create form demanded the asset number anyway, so the same value was typed
twice and the convention held only as long as everyone remembered it.
The prefix now lives on the device type, and a blank asset number is generated
as <PREFIX>-<name>. Left explicit, an asset number always wins: a device
carrying a real identifier of its own - a vendor tag, a controller name, a
serial - keeps it. That is the platform rule, adopt where an identifier exists
and derive only where none does.
The prefix is NOT derived from the type name. "Access Point" and "Access Panel"
both initialise to AP, and assetnumber is unique, so the second type would
collide with the first on every device it created. It is nullable, so a type
that wants no prefix generates the bare name rather than needing one invented.
Names are sanitised before they reach a business key - the existing data
already shows why, with IDF-Telco-Demarc-#1 carrying a '#' into an identifier.
An existing prefix is never stacked: IDF-03 under type IDF stays IDF-03.
Test bays reported 0 installed / 0 skipped / 0 failed and nothing was being
processed. The share is a TRANSIENT mount: GE-Enforce.ps1 maps it for the length
of its cycle and unmounts at the end - mounted 12:10:02, unmounted 12:10:27 on
FB9TP7V3. The shadow task ran on its own 15-minute schedule, so it woke with the
drive gone, handed the engine a path that no longer resolved, and the summary
zero-filled. A silent nothing, indistinguishable from a healthy no-op, which is
why it read as "not processing" rather than as a failure.
Register-ShopdbShadow.ps1 is replaced by Invoke-ShopdbShadow.ps1, which runs AS
a manifest entry and invokes the runner directly. The share is mounted because
the enforce cycle is what called us; there is one cadence instead of two; and
there is no task to register, drift, or heal. It also unregisters the superseded
task, so bays that already carry it clean themselves up rather than keeping a
run that cannot see the share and overwrites the real result with 0/0/0.
The runner gains the Test-Path guard that should have been there: an unreachable
-ShareManifestPath now logs, writes an event, and reports a real failure naming
the reason, instead of running the engine against nothing. That silent zero is
what made this expensive to find.
Verified on the win11 VM with a substituted drive removed between runs - the
same call gives real counts while mapped and the guard when not. The earlier
task self-heal is moot now; it was a fix at the wrong layer, and testing against
a local path is what hid a mount I had already noted was dynamic.
A 3201dw is monitored and alerts correctly - Zabbix reports its cartridge names
and levels straight off the printer - but the Toner Report had no reorder chip
for it, because part numbers come only from modelsupplies and no family matched.
lookupsupplies returns [] for a model with no rows, so the printer showed levels
with nothing to order.
Adds the 218 family: 218A standard and 218X high yield in all four colours,
matched on 3201 and 3301 so the MFP siblings are covered by the same entry.
Part numbers and yields cross-checked against three independent sources rather
than inferred from the series - a wrong toner number means someone orders a
cartridge that does not fit, which is worse than the blank chip it replaces.
No drum or waste unit: none is listed as a separately orderable part for this
series, so the family is toners only.
"Checked 13 Aug 1:20 PM. Verified the backup taken 12 Aug 5:20 PM is still
current." Two facts, and one date could not carry both.
lastseenat moves on every successful confirmation and proves the check is still
running. collectedat moves only when the configuration actually CHANGED and says
what is stored. Showing lastseenat alone hid what is in the backup; showing
collectedat alone read as "last backup was a month ago" on a machine that is
perfectly healthy, because a stable config writes no new revision. The hover now
states the check and the capture separately and says outright that the second
being older is the normal case.
The stale wording gets the same treatment: it names the check that stopped AND
the newest copy that exists, which is the thing someone recovering from it
actually needs.
collectedat is exposed as backupcollectedat. Both stay tooltip-only - the badge
is still just the kind and a colour, so nothing here changes what a new backup
kind has to do to inherit it.
Two fixes to the same table, in the same regions of the same files.
ASSET LINK POINTED AT THE WRONG RECORD. The Asset chip linked
/machines/<assetid>, but /machines/:id keys on machineid - the plugin extension
id - as MachineDetail itself does everywhere. So the link landed on whichever
machine happened to carry that number: a wrong page that looks right, which is
worse than a 404. Same for /measuringtools/. The API now returns
machinepluginid / toolpluginid beside the asset ids and the view links on those.
Both lookups are import-guarded, and with no plugin id the number renders as
plain text rather than a link that misleads. AssetRelationships already resolved
this correctly; this brings the reports table in line.
BACKUP COLUMN READ AS NEGLECT. It showed a raw date, and a revision is only
written when the config CHANGES - dedup means a machine stable for months has a
months-old newest revision and is perfectly healthy. The column already used
lastseenat, the last time the collector CONFIRMED the config, but a bare
timestamp says "nothing has happened since", which at the default 24h collection
interval IS the healthy steady state. It made a working system look stalled and
made the reader do arithmetic against a setting they would have to go and find.
It now returns backupok and shows a badge naming the kind, green when confirmed
recently, red when not, with the date and an explanation in the hover. backupok
is tri-state on purpose: null means no revision at all, and renders as NO badge
rather than a green one, because "never seen" must not read as healthy. The
threshold is the backups plugin's own backups_staledays, read through its
service so there is one definition of stale rather than a second drifting here.
Nothing in the badge is kind-specific, so a backup kind added later inherits it
by existing. docs/BACKUP-KINDS.md records that, the BackupKind contract, and why
the rule is time-based rather than per-kind.
Registered fine and never fired once. Three faults, all found on the win11 VM.
A `-Once -At (Get-Date)` trigger does NOT run immediately: its first run is the
start boundary PLUS the repetition interval, 15 minutes out. This ships as a
DetectionMethod=Always entry, so it ran every enforce cycle, 5 minutes apart,
and each Register-ScheduledTask -Force reset the start boundary to "now" -
pushing the first run back to +15 before the previous +15 could elapse. 5 < 15,
so the task sat Ready at LastTaskResult 267011 (SCHED_S_TASK_HAS_NOT_RUN)
forever. It now registers only when the task is missing or its arguments
changed, and starts it once on first registration rather than waiting out the
first interval.
A bay provisioned by the broken version carries a task with correct arguments
that has never run, so "leave it alone if it matches" would have stranded
exactly the machines that hit the bug. If the task has never run it is kicked
once; after that LastRunTime is set and the check is a no-op.
None of this was visible. The engine records only "ps1: <path>" and an exit
code for a PS1 entry, so Write-Host reached nothing, and with the fail-safe
`exit 0` on every path a silent early-out was indistinguishable from success.
It now also writes C:\Logs\Shopfloor\shadow-setup-<date>.log.
Scope is no longer hardcoded either: this script is shipped by more than one
scope now, and a wrong value would shadow the wrong manifest silently. It is
derived from the script's own directory, the same way the share manifest path
already was, so the two cannot disagree.
The report answers a purchasing question, and it was answering it in seven
columns, two tables and a rowspan. What someone actually needs from it is a
short list of what to buy.
So it opens with that list, grouped by part number with a quantity. Two
cartridges of the same part in different printers is a quantity of two, which
is the number an order needs and the one a per-printer table made the reader
count by hand. It covers what is empty plus what goes within a fortnight -
ordering only what is already empty means running empty. There is a copy
button, because it ends up pasted into a mail.
Below it the cartridges sit in urgency bands rather than in one long list
sorted by a number. The question is which pile a thing is in, and a pile that
is empty is worth seeing as empty. Everything past "empty" starts collapsed;
the order list above already covers the same ground in a tenth of the height.
The row is a cartridge now, not a printer, so it can carry its own part number,
its own level bar and its own countdown. Nesting supplies under a printer meant
opening a printer to find out whether anything on it needed doing.
Cartridges with no part mapped are counted on a single line rather than given
one each. They cannot be dropped, since that would quietly shorten the order,
and they cannot be ordered from here either - the job they represent is
mapping them, which is one job however many there are.
Bands and the order horizon are decided server-side, next to the arithmetic
that produces them, so a heading cannot disagree with what got added to the
list.
Checked against a fleet of 43 dev printers with real part mappings, driven by
a stub Zabbix - live Zabbix is not reachable from the dev box.
Prod grew rows saying both "PC controls 2005" and "2005 controls PC", and a CMM
PC showing "<- controls from CMM4" beside its own outgoing link. Only one
direction can be true: a PC drives a machine, never the reverse.
Nothing stopped it. The duplicate check was keyed on (source, target, type), so
the inverse inserted cleanly, and the Add Relationship dialog offers an incoming
direction that writes exactly that. The legacy import stores controls the wrong
way round as well. Directional creates now refuse the reverse with a 409 naming
the row that already holds it, and refuse self-links, which render as a
duplicate on the asset's own page and mean nothing. Symmetric types are exempt:
Dualpath stores both directions on purpose and the card collapses them. The
propagation fan-out got the same guard so a rail meant to spread one direction
across sibling bays cannot manufacture a pair.
fix-controls-direction only matched source assettype 'machine', so every
measuring_tool, printer and network_device row it was written to clean survived
it - which is why running it would never have fixed the CMM. It now matches any
non-computer controlled BY a computer.
New `flask relationships audit` reports what is already stored: reciprocal
pairs, self-links, and PCs controlling several assets of one type. Read-only,
and it prints each row's label because that usually names the writer outright -
collector:* means this code made it, anything else means a person or the import
did. That distinction decides the fix for duplicate device assets, which is not
in this commit: the collector keys idempotency on its own label, so a device
somebody created by hand is invisible to it and it mints another, and the
adoption rule needs the audit run against prod before it can be written.
Two false positives were found writing it, against the dev database, and both
would have made the report useless. A self-link is its own inverse, so it was
counted as a reciprocal pair AND printed twice. And Dualpath siblings looked
like duplicate devices - a dual-bay machine is one physical machine with one
controller and controls is propagated to both bays deliberately. That was 30 of
32 findings, consecutive bay numbers pair by pair.
The cheap cutover the doc described - manifest over API, payloads still on the
share - does not work. Measured on the win11 VM against lib 2.6 with the same
manifest and engine, varying only InstallerRoot: the payload cache gives
0 installed / 2 failed ("CMD/BAT not found" under the cache), the scope
directory gives 2 installed / 0 failed.
Two causes. Invoke-ShopdbEnforce.ps1 takes no -InstallerRoot and hardcodes the
payload cache outside shadow mode, which is right for displays where every
entry is inline and wrong for every smb entry. And -IncludeCommon cannot work
for smb payloads at all: they live under their own scope directory, so a merged
manifest needs two roots where the engine takes one, and Join-InstallerPath
rejects rooted values by design so absolute paths are not a way out. The real
dispatcher never merges - it calls the engine once per scope with that scope's
root - and a fix should follow that shape.
Both claims are corrected in the doc rather than deleted, because reading it
was what made this look like a twenty-minute job; it would have failed 29 of 31
entries on the pilot bay.
Shadow mode is unaffected - it points InstallerRoot at the share scope dir,
which is the working path - so the pilot proceeds. Register-ShopdbShadow.ps1
is the reference copy of the script that puts a share-attached bay into shadow:
it writes BaseUrl, then registers the task as SYSTEM. It derives the share
manifest from $PSScriptRoot rather than hardcoding a drive, because GE-Enforce
mounts the share to a letter it picks at run time and SYSTEM has no standing
UNC access. Delivered as a manifest entry gated to one hostname, so the pilot
needs no hands on the bay and heals if it drifts.
Four defects stacked into one nonsense report: cartridges at 20% claiming four
days, cartridges at 1% claiming weeks.
The root cause is a Zabbix API detail. `limit` caps the whole result set rather
than each item, and the query sorted ascending, so the cap kept the OLDEST rows
in the window. A four-cartridge printer polled every five minutes writes over
100k readings in 90 days; the forecast was fitted to the first few days of that
and nothing since. Every rate was real and every rate described a cartridge
thrown away three months ago. Nothing in the output looks wrong, which is why
it needed pinning in a test rather than a comment.
A 90-day burn rate does not need every individual poll, so a long window now
reads hourly trends - the table meant for this, a tenth of the rows, and kept
longer. Raw history serves short windows and any item a site keeps no trends
for. Both are fetched newest-first with the budget scaled per item.
Second, the countdown was computed from the last stored reading while the level
displayed was the live one, so the two could disagree by a whole cartridge. The
live level is now what the countdown divides. A live level far above the stored
run means it was swapped since the last reading, and that is reported as a
replacement rather than as a collapse in the burn rate.
Third, at or below 5% a cartridge reads as empty rather than as a slow drain.
At 1% losing a tenth of a point a day the arithmetic says ten days. The printer
is out of toner, and it is the first thing to order.
Fourth, the days-left column spanned the printer's rows, so the printer's
soonest figure was printed beside every supply it had. That alone accounts for
the shape of both complaints: a healthy cartridge wearing its neighbour's
deadline, and an empty one wearing a number that belonged to nothing on its row.
Also fixes float-typed supplies vanishing from any printer that also had an
integer-typed one - they live in different history tables and the fetch stopped
at whichever answered first.
Not verified against live data: Zabbix is not reachable from the dev box.
The fleet table has two time columns and staleness only looked at one. A PC can
go quiet in either direction: silent to the server, or still posting while its
own cycle has stopped advancing. Only the first was caught, so a machine whose
Last check-in had not moved since the previous morning still showed 'ok'.
The two are tested independently rather than by taking the newer of them.
receivedat is the server's own record and cannot be argued with; lastcheckin is
asserted by the client, so a wrong clock there must not be able to vouch for a
PC the server has not actually heard from. Checking them separately means a
future-dated lastcheckin cannot mask real silence.
The tooltip now names both times, since which one stopped says what kind of
failure it is.
A report records how ONE cycle went. Nothing ages it, so a PC that stops
reporting keeps the status of its last good cycle: switch a machine off after a
clean run and it shows 'ok' indefinitely. One had been offline more than a day
and still read 'ok'.
Silence is a different fact from the last cycle's outcome, so it is computed
separately rather than by rewriting the stored status. receivedat - the server's
own clock, not anything a client asserts - is compared against
geenforce_reportstaleminutes, default 30, which is roughly two missed cycles at
the usual cadence. Set it to 0 to turn the check off.
In the table 'stale' takes the badge, because a status from a machine that has
not spoken since is not evidence of anything. What it last reported stays in the
tooltip with the time it was heard. The stored status is untouched: it is still
a true record of that cycle, just not proof the PC is alive.
A site whose scope enforces less often than the threshold will read stale while
healthy, which is what the setting is for.
Core called the roles dashboard / lobby / partskiosk. The kiosks call them
Dashboard / Lobby / 3DPrintRoom, which are the literal contents of
C:\Enrollment\display-type.txt, read by the GE-Enforce dispatcher to pick a
target. Two vocabularies for three kiosks, each with its own copy of the same
route map.
That is not cosmetic. A display reporting its own type sends what its file
says, so it could report a role core would not accept, and core could store
'partskiosk', a value no dispatcher would ever match. The enforcement report
column would have shown one vocabulary from the device and the other from the
DashboardDefault fallback, in the same column.
The machine's file wins, because that is what a person edits. DISPLAY_ROLE_PATHS
takes the kiosk spelling and the display scope now uses that dict rather than
holding a second one, so the two cannot drift again. normalize_display_role
resolves any casing and the retired 'partskiosk' forward; the dispatcher already
matched its map case-insensitively and the server now agrees with it.
Nothing is turned away over a capital: the API accepts any spelling and stores
the canonical one, displaypath resolves through the normalizer so rows written
before this keep working, and the settings dropdown canonicalises on open so an
old value does not render as a blank select.
A reported subtype is normalised on the way in, but an UNRECOGNISED one is kept
verbatim. That is a kiosk with a typo in its file or a role nobody declared, and
both are worth seeing in the fleet table rather than blanked or guessed at.
Contract bumped for the added names. DashboardDefault is finally listed in
__all__ too - 0.17.0 put it on the surface and never exported it.
SQLAlchemy 2.0 connections do not autocommit, and on MySQL alembic reports
"non-transactional DDL", so context.begin_transaction() is a no-op. Nothing in
a migration run committed.
It looked like it worked because MySQL implicitly commits on DDL: every
ALTER/CREATE flushed whatever was queued before it, including the PREVIOUS
migration's version stamp. The LAST migration of a run has no DDL after it, so
its stamp was rolled back when the connection closed. `flask db upgrade` then
exited 0 with the schema change really applied and alembic_version one revision
behind, and re-ran that same migration on the next deploy. A migration that is
not idempotent applies twice.
Found while adding a core migration: the column default really changed and
alembic_version still named its parent, with UPDATE alembic_version followed
immediately by ROLLBACK in the log.
Existing databases upgraded before this are one revision behind their real
schema. Compare `flask db current` against `flask db heads`; re-running the
upgrade re-applies the final migration once and lands the stamp.
Install-ShopdbKiosk.ps1 lays the enforce client down once at bootstrap and
never refreshes it. So a client change rode the code deploy to the server
and then sat one directory away from where kiosks actually fetch, waiting
for someone to re-stage the installer bundle by hand - which is how the new
display-type reporting reached prod and changed nothing on any kiosk.
The module now ships as a manifest entry like everything else in this
scope: inline over HTTPS, Hash detection against the exact bytes shipped,
written to the same path the installer uses so bootstrap and self-update
cannot disagree. Ordered first, so a stale client refreshes before anything
leans on it. The installer keeps its real job - a fresh kiosk still needs
something that can talk to shopdb - it just stops being the update path.
Self-modifying by design: this module is what stages payloads, but
PowerShell loads it into memory at start, so rewriting the file mid-run is
harmless and lands on the next cycle. Pilot a client change on ONE kiosk
before the fleet: a broken module cannot fetch its own replacement, and on
a share-less display that means a site visit.
A display knows whether it is a Dashboard, a Lobby screen or the 3D print
room - the dispatcher reads C:\Enrollment\display-type.txt to choose which
page to open. It never told shopdb, so the fleet table inferred it from the
DashboardDefault fqdn mapping, which is empty unless somebody added a row
per kiosk. The column was blank for every display.
The client now reads that file and reports it, the report stores it, and
the API prefers the reported value with the old mapping left as a fallback
for hosts still on an older client. Reported by the device beats inferred
from a lookup table, the same way enforcerversion already works. A PC with
no display-type.txt reports nothing rather than something invented, and an
empty string lands as NULL.
Two guards had to learn about it. The DDL parity check read only the 0001
baseline, so a column added by a later revision looked like drift even
though its migration existed; it now runs the whole chain, which is what
'do the models match what the migrations build' means. 0002 added a whole
table rather than a column, which is why this is the first time it bit.
Two caps, one inside the other: the section shell at 1400px and the reports
view at 1100px. Fine for nine columns; the reports table now carries twelve,
including the asset a PC drives, its location and its backup state, so it
was being squeezed while empty page sat to the right. Both are gone, and
the manifests editor gets the width too.
The Asset column no longer falls back to the PC's own asset number. The
collector stores a PC's hostname AS its assetnumber, so for any PC that
drives nothing that column just repeated the Host column next to it. It now
shows a dash, and carries content only when it says something Host does
not: the machine a bay PC controls, the tool a measuring PC controls, or
the role of a display.
The reports table 500'd on every load: the backup lookup ordered with
ORDER BY lastseenat DESC NULLS LAST, which SQLite accepts and MySQL
rejects outright. Every test passed and the real database refused the
query - the tests run on SQLite, so the dialect difference was invisible.
Sorting in Python removes the dependency for nothing: the rows are one per
host per kind. The regression test pins which revision wins, including
that one never confirmed does not, and says why the sort lives here so it
does not get helpfully moved back into SQL.
The enforcement reports needed to name what a display IS, and reached
straight into shopdb.core.models.dashboarddefault to do it. Plugins may
only touch core through shopdb.api, and the contract test said so.
The role belongs on the surface rather than behind it: it lives in core,
no plugin owns it, and a plugin reporting on displays has no other way to
resolve it. Added there and the version bumped, which the docs test pins.
A report row carried a hostname and some counts. Everything an operator
wants next hangs off the asset behind that host, and none of it was there.
Host now links to its PC page. Beside it, what the PC IS or DRIVES: the
machine number for a bay or part-marker PC, the measuring tool for a tool
PC, the role for a display - each linking to that asset's own page. A map
pin appears only when the asset has coordinates and hovers to the same
floor-plan preview the asset pages use; an icon that opened an empty map
would be worse than no icon.
The backup column is LAST CONFIRMED, not last changed. Dedup means an
unchanged config writes no revision, so a machine stable for six months has
a six-month-old newest revision and is perfectly healthy - the one to worry
about is the machine whose backup stopped running. It reads lastseenat and
names the kind rather than assuming ntlars, since udc/file kinds on the
share are coming.
Resolution is bulk, never per row: this table shows the whole fleet, so a
lookup inside the loop would be one query per PC. It reads the collector's
existing 'controls' relationship rather than re-deriving which machine a PC
drives - that same resolution living in two places is what put a wrong
subtype filter on the map. Every plugin it touches is optional, so each
lookup is ImportError-guarded and a lean build renders the table without
those columns. A host ShopDB has no asset for still shows: the enforcement
result is real even when the inventory is behind.
Every column in the entries table was sized to its button text with nothing
spare, so with flex-wrap on the action cell the Edit and Delete buttons
wrapped out of their 128px column and sat on top of the entry description.
The order column was tighter still, and its Up/Down buttons carried almost
no padding, which made them hard to hit as well as hard to read.
Widened the order, type and action columns to fit their contents on one
line, stopped the action cell wrapping, gave the buttons real padding, and
added vertical cell padding with top alignment so a two-line entry no
longer runs into the row beneath it.
The engine treats a minor-newer manifest as backward compatible and carries
on. That holds for additions which WIDEN behaviour - an old lib skips a Type
it does not know - and inverts for one that NARROWS it. _CmmVersion arrived
in lib 2.6 as a minor bump, so a PC on 2.5 does not recognise the field,
reads every gated entry as unfiltered, and installs every PC-DMIS version it
cannot detect, on every CMM, within one cycle.
The share runbook already says push the lib first. A runbook is not a
control, and the failure is silent, fleet-wide and about five minutes fast.
ShopDB already had the evidence and was not using it: every enforcement
report carries the enforcer version, and publish_scope had no gate at all.
It now compares the scope's manifest version against the versions PCs
actually report for that scope and refuses when any is behind, naming the
hosts. force=True for someone who knows why. A report with no or an
unreadable version counts as behind - that field arrived with the
summary-emitting engine, so its absence IS an old lib, and treating unknown
as safe is precisely how this fails open.
A scope nobody has reported for still publishes, or a fresh site could
never publish anything. Versions compare numerically, since as text '2.10'
sorts below '2.9'.
Also exposed as a preflight endpoint so the UI can warn before someone
clicks publish, and as a 409 with the offending hosts rather than a 500.
Every kiosk was sitting on Edge's restart-to-update prompt. The scope
already sets RelaunchNotification=2 so Edge restarts unattended, but
RelaunchWindow defers that restart to 02:00-04:00, so during the day the
update waits and there is nobody on site to dismiss the prompt.
A one-shot entry runs the Edge updater and stops the browser. It does not
relaunch it - the enforce task is SYSTEM in session 0, where a launched
browser is invisible - so it leans on the watchdog that already relaunches
the kiosk from the Startup shortcut. That relaunch is what applies the
staged update.
One-shot is DetectionMethod=MarkerFile. The engine writes the marker only
after a 0 exit, so a failed run retries next cycle instead of being
recorded as done, and the script exits 0 when no Edge was running - that
is a success, and failing it would withhold the marker and re-kill Edge on
every cycle from then on. The marker path carries a date, which is the
re-arm mechanism for a future update.
Ordered after the watchdog entry: a display seeing both for the first time
must have its relauncher registered before anything stops the browser.
The publication rsync excluded 'tools' to keep the repo-root tools/ dir
off GitHub. rsync patterns without a leading slash match at any depth, so
naming a plugin 'tools' meant plugins/tools/ was dropped too. It was
committed, exported, bundled and deployed, and the first sign of trouble
was `plugin install tools` on the server failing on a manifest.json that
had never been shipped.
The root-intended excludes are anchored, and the sync now verifies that
every plugins/*/manifest.json in the working tree came out the other side.
A silent omission of a whole plugin should not be something a deploy
discovers for us.
Ten routes shipped over the last few commits without reaching the inventory,
which is the file everything else is generated from - openapi.json, the /api/docs
browser, and the per-site build. An endpoint absent from it is invisible to
anyone integrating against the API even though it answers requests.
The auth notes matter more than the paths. API-REFERENCE tells a deployment
reviewer that essentially every GET is optional-auth; two of these are not, and
an exception buried in a table of 372 operations is one nobody reads. The
installer download and the warranty proof are called out with why: one is
licensed vendor software, the other carries pricing and a service tag. The
application image is listed with the fully public endpoints, since tiles render
before login.
The classic ASP site prints a week number under the site title, and people
quote it in conversation and on paperwork. Anyone with both sites open needs
the two to agree, so this is a port of the old arithmetic rather than a fresh
interpretation of what a fiscal week is: ISO 8601, week 1 contains 4 January,
and the week's Thursday decides which year it belongs to. That rule is what
makes late December and early January land in the right year, which is exactly
where a naive day-of-year count goes wrong, so it is what the tests cover.
Worth recording: a true GE fiscal calendar need not follow ISO weeks. Nobody
has asked for a different rule, and inventing one here would silently disagree
with the site people compare against.
Computed in local time on purpose. The number people quote is the one on the
wall where they stand, and a UTC week rolls over hours early in the evening at
a US site. The sidebar re-checks every half hour; the shop-floor board picks it
up with the clock it already ticks.
The toner report says what is empty now. It could not say what to order, and
nothing recorded how fast anything drains - every level read was cached for
five minutes and then discarded.
Zabbix has been keeping the history all along; we simply never asked. One
history.get gives both answers, because a cartridge only goes DOWN while it is
in use: a rise is a replacement. Count the rises and you have how many
cartridges a printer has been through; fit a slope to the readings SINCE the
last rise and you have days-to-empty. Fitting across a replacement averages a
spent cartridge with a fresh one and describes neither.
Sorted by days left, which is the point. A cartridge at 60% dropping 5% a day
needs ordering before one sitting at 8% that has not moved in months, and a
level-sorted list ranks those backwards.
It refuses to guess. Too few readings, a level that has not moved enough - many
printers report in 10% steps and sit on a plateau for a fortnight - or a recent
replacement each produce no estimate and say which. Those printers are listed
separately rather than sorted in as 0 or as 999, since a printer without an
estimate is neither urgent nor safe. Estimates show what they rest on, because
"9 days from 21 days of readings" and "9 days from 2 readings" are not the same
claim.
A separate report card, not an extension of the toner report: that one is an
exceptions list a tech acts on today, this is an ordering view read monthly,
and the history query is heavier than the live read it would have slowed down.
The analysis is pure arithmetic over a list of readings, so the 14 tests cover
the noise wobble, the plateau, the swap, junk rows and division by zero without
needing Zabbix. Zabbix being unreachable is reported as such rather than
rendering an empty table that reads as "nothing is due".
A provider lookup answers whether a unit is covered. It does not produce the
invoice or the extended-warranty certificate, and a manually entered warranty
had nowhere to keep one - so the proof stayed in somebody's mailbox until they
left.
Two columns rather than one: the served URL of the stored document, and the
name the vendor sent it under, because "Dell invoice 4471.pdf" is what a person
recognises a year later and "warranty-12.pdf" is not. The download route sends
the original name back.
Authenticated in both directions, unlike an asset photo: an invoice carries
pricing and a service tag. One document per warranty, replacing any prior
extension so a re-upload as .pdf does not leave the old .png behind claiming to
be current. Capped at 25MB - a certificate is a document, not a disk image.
Office formats are allowed because purchase records genuinely arrive as .msg
and .xlsx, not only as PDFs.
Adding an application meant typing an image FILENAME and trusting someone had
dropped the file into the frontend's own directory by hand, and typing an
install path from memory. Both are uploads now, following the model-image trio
that models and part photos already use.
The two differ deliberately. The image is public, because application tiles
render before anything is authenticated. The installer is not: it is licensed
vendor software, an open URL would publish it to anything that can reach the
site, and it is always sent as an attachment rather than rendered.
Installers are capped at 500MB and the size is measured by seeking the stream
rather than trusting Content-Length, which a chunked upload does not send and a
client can understate. Anything larger belongs on the share, and the error says
so rather than just refusing.
Files are chosen before a new application exists, so they are held and uploaded
once there is an id to attach them to. A failed upload leaves the saved record
alone and reports, rather than losing what saved fine.
Removing an installer only clears installpath when it pointed at the upload - a
share path was typed by a person and is not ours to wipe. The detail page reads
both shapes, since entries from the classic site hold a bare filename that is
still served from /images/applications/.
The card track was 28rem, which fits four across a full-width page but only
three once the sidebar takes its 250px - and 1920 with the sidebar is the
common case here, so the board showed three. The track is 22rem now, with an
explicit four-column cap above 96rem: left to auto-fit alone a wide screen
reaches five, and a fifth column only makes the cards narrower until the rows
they hold start truncating again.
Measured at 1366, 1600, 1920 and 2560: three, four, four, four.
The floor-plan preview drops from 500x385 to 390x300. At the old size it
covered the row it was launched from, which is the row you are trying to read.
routes.gen.js spreads `.toplevel` for every plugin uniformly, but only a
few own full-screen routes, so Rollup warned on eleven of them every
build. The `|| []` guard was always doing its job - the warning was noise,
and noise in a build log is where a real warning goes to hide.
A place for the small utilities a technician reaches for at a bench. The
plugin owns no API and no tables: every tool runs entirely in the browser,
so an air-gapped site gets them for free and a bad network cannot break
them. Adding the next tool is a view, a route, and one entry in tools.js.
First tool is a barcode/QR generator. Content is typed text, a URL, or a
CSV (content,label,copies - quoted fields and an optional header both
handled), so a batch of a few hundred is one paste. Label stock is
adjustable in inches with five presets, and the code renders to an SVG
data URI rather than a PNG: a bitmap gets downscaled to label size and
smears the module edges a scanner reads, where SVG rasterizes at the
printer's resolution with hard edges.
It also carries the dot-grid rule that is easy to get wrong by eye. A
thermal head cannot render a fraction of a dot, so a code sized off the
grid gets uneven modules; pick a DPI and the page says what the current
size lands on and what to use instead. The quiet zone is blank label
rather than white baked into the code, so it can be tuned - and it applies
to CODE128 too, which needs clear space at each end and was letting bars
run into the caption.
Tech Tools is the first bundled plugin that owns no schema, which two
guards did not model: it belongs in the universal installer profile, and
upgrade-all reports it 'no-migrations' where every plugin was assumed to
report 'ok'. The migration test now asserts that status explicitly for
schema-less plugins, so a table-owning plugin whose chain went missing
still fails.
The report exists to answer "what needs replacing and where do I get it",
and it was answering neither. The part numbers were already in the
lowsupplies payload and simply never rendered; a chip per part now shows
them, with capacity tier and page yield on hover, since a model can list
several tiers for one colour.
Asset # and Location columns are gone. Location is replaced by the
floor-plan preview the asset pages already use, hung off the printer name
via its mapx/mapy.
The IP is now the site's FQDN (printer_hostname_template, built from the
IP exactly as PrinterForm does) and links to the printer's own web page in
a new tab - the report is a worklist, and losing your place in it to visit
one printer means finding your row again. The raw IP stays on hover.
Cartridge names were ellipsised inside a fixed 120px column, hiding the
one thing being reordered. The supplies cell is a grid with a max-content
name column, so names show in full and still line up across a printer's
rows.
CSV and emailed exports follow the screen, with one row per part number so
the result is a copy-pasteable order list.
"All assets 704" sat beside "all assets in use 737", and both were correct
about different populations. The totals summed five specific asset types and
subtracted dual-bay secondaries; the status counts took every asset row of any
type with no collapse, so USB devices and hidden secondary bays inflated one
side of a comparison the layout invites. Status is now counted over exactly the
same assets the totals describe.
Warranty rows fell back to asset.name when the covered asset had no hostname,
and an asset's name is usually the MACHINE's descriptive name - which is how a
column meant to identify a PC ended up showing a machine. Hostname, else the
asset number, never the name. The machine number loses its label too: the row
is hostname, machine, state, and "machine 3015" spends a word on what position
already conveys.
Printer names now carry the floor-plan preview on hover, the same
LocationMapTooltip the printer's own page uses - a location name tells you the
room, the map tells you where to walk. Declared as map.maphover on the card, so
any card with coordinates gets it; a row without them shows a plain link rather
than being dropped.
Cards are four across rather than five. At five columns a row holding a
hostname, a machine number and a state truncates on exactly the rows that
matter. auto-fit, so two cards fill the width instead of leaving empty tracks.
Not covered by a test: the count fix. I started one and it was interrupted, and
I have not gone back for it - the assertion worth having is that in-use can
never exceed the total.
The shopfloor board lost every photo and started showing lowercase SSOs where
names belong, and nothing in the system could say why. Both resolvers returned
None on any problem and three separate bare excepts threw the cause away, so an
unreachable HR host, a rotated credential, a renamed column and a genuinely
unknown SSO all produced identical output and no log line. That is not a bug in
the resolution so much as a hole where the diagnosis should be.
GET /api/employees/resolve/<sso> returns the same answer the board gets, plus
which source produced it (directory, hrdirectory, useaccount), which mode the
directory is in, and the exception text when a source failed. It is the
difference between "the board is broken" and "the HR host refused the
connection".
The two bare excepts in the shared resolvers now log rather than pass. The
external-directory branch is the one that fails on a live site; it was the one
saying nothing.
No behaviour change to the board itself - it still falls back exactly as
before. What changed is that the fallback is now visible.
PRINTER CARD is now supplies at or below 5 percent, a new
printers_dashboardpercent setting. The report and the card want different
scopes: the report lists anything the thresholds call low, which is right for
planning an order, while the dashboard is asking what to walk out and change
today - and a cartridge at 18 percent is not that. Lowest first.
PRINTER LOCATION WAS ALWAYS EMPTY. The lookup went through db.session.get on
locationid and produced nothing even where a location is set; the printers list
has always read it through the asset relationship, so the card does too now.
WARRANTY ROWS are identified the way the floor identifies them: the PC's
hostname and the MACHINE it drives, reusing the same lookup behind the warranty
page's machine column so the board and the report cannot disagree about which
bay a PC belongs to. No dates - expired or expiring is the whole decision when
scanning a board, and the exact day belongs on the report you order from.
NOTIFICATIONS are stacked: the type in full on one line, the message beneath,
trimmed to 100 characters with the rest on hover. Inline, the type was
truncated to make room for prose that was then truncated anyway, and neither
read. A tooltip is omitted when the text was not trimmed, because one repeating
what is already on screen is noise.
Two general additions: layout: 'stacked' on a card, and map.detailtooltip.
The printer card was a name followed by a comma-joined string of cartridges and
a location - the widest row on the board, and the one running past the card
edge.
Now: the printer name links to its page and reveals its location on hover, and
each depleted cartridge is its own chip showing "Black 4%", revealing the part
number to order on hover. The percentage says something is wrong; the part
number says what to do about it, which today means opening the printer's page
to find out. Every capacity tier is listed, as the report has always done.
Two additions to the card contract, both general: 'chips' maps a row key to a
list of {text, title, level}, and 'titletooltip' puts context on the row title.
Nothing load-bearing goes in a tooltip - hover is not discoverable and does not
exist on touch - so a chip always states the fact and only explains it on hover.
Chips are bordered rather than filled: a row of solid red pills reads as an
emergency even when a cartridge is merely low.
Also repaired a self-inflicted mess. A string-slice edit used a marker that
appears EARLIER in the file, so the slice was empty and two helpers were
injected at line 1, above the module docstring. Removed; the file parses and
the helpers live beside the route they serve.
Four fixes, all from looking at the real board.
"and N more" now links to a page showing them all. Telling someone 35 more PCs
are silent and leaving them to find the list is worse than not saying it. Each
card names its own destination and a test checks it against the routes that
actually exist - a viewall pointing at a route nobody wrote is the same rot the
endpoint check already guards, just failing in the browser instead of the API.
PRINTER ROWS ESCAPED THE CARD. A flex child will not shrink below its content
width unless told to, so text-overflow never engaged and a row carrying three
cartridge readings plus a location simply ran past the border. min-width:0 on
the row parts is what enables the ellipsis; meta shrinks first because it
matters least, and the card clips as a backstop.
THE STAT TILES WERE INCOHERENT. Two counted asset TYPES, two counted asset
STATUSES, and nothing said which - with the status one labelled "Active", which
reads as "not deleted" but meant status = In Use across every type. Each tile
now counts one thing and its label says so.
PCs GONE SILENT IS NARROWER, and better for it. A PC that never reported at all
is usually a hand-made or imported record rather than a bay that broke, and a
PC that is not In Use is silent ON PURPOSE - that is the status doing its job.
Both were burying the real signal: a machine that was working, is not now, and
nobody has marked as anything else.
lastseenat already recorded it and the API already returned it; nothing
displayed it, so from the UI a healthy machine still looked abandoned - one
revision from last spring and no sign anything had looked at it since.
The history page gains a Last verified column beside Captured, and the asset
panel a Verified field. Only the CURRENT revision carries one: an older
revision was superseded, so saying it was verified today would be false - what
was verified is the configuration the PC holds now.
A current revision with no check yet says "not yet checked" rather than showing
a blank or borrowing the captured date. That state is real and temporary: the
column is new, so every chain reports it until its PC next posts.
Three faults, visible only once the board ran against production data.
BACKUPS SAID THE WHOLE FLEET HAD STOPPED. The lastseenat backfill was wrong. It
seeded from collectedat, reasoning that the last change was the last provable
moment - but an unchanged config writes no revision, so a machine whose settings
last changed nine months ago got a nine-month-old lastseenat and was instantly
reported as a dead backup. Every chain lit up at once, which is worse than no
card: it says the site is broken when it is fine.
The honest value is NULL. Before the column existed nothing recorded when a
config was last confirmed, and inventing a date does not change that. Migration
0003 clears the backfill, and staleness now IGNORES a NULL chain rather than
substituting timestamps that mean something else. A chain becomes measurable the
first time its PC posts, which for NTLARS is within a day.
TONER READ "None%". The supply dict has no 'percent' key - it is 'remaining'.
Supply names are also shortened, because "Black Toner Level 4%" spends three
words saying what the card already says.
THE CARDS READ AS WALLS OF TEXT. Rows wrapped into paragraphs and a card with
forty PCs pushed everything below it off the screen. Now: at most five rows with
"and N more", one line per row that truncates rather than wraps, meta pushed
right and dropped first since it matters least, and severity reduced to a small
dot beside an uppercase label instead of a coloured card - six severity-painted
cards read as a crisis, which is how a board stops being read.
Worth recording that none of this could fail in a test. Every one needed real
data on a real fleet.
Three plugins still declared widgets naming Vue components nobody wrote.
Converting them honestly meant three different answers, not one.
notifications gets a real card: the active notifications themselves, not a
count. "4 active" tells an admin nothing; knowing WHICH message the shop is
looking at is the point, and it is how a stale one gets noticed and taken down.
machines gets machines out of service - anything not In Use, excluding
Inventory, because a spare on a shelf is stock rather than a problem. Someone is
supposed to be chasing each of those and today they are visible only to whoever
thinks to filter the list by status.
network gets NOTHING, and its declaration is deleted rather than converted.
Network devices carry no live status - no polling, no reachability check,
nothing that can be wrong - so the only possible card is a count of how many
exist, which is precisely the always-true number this dashboard exists to get
away from. A comment records that, so the next person does not re-add it. If
reachability is ever collected, that is the card.
Also adds a contract test over every declared card: no component names, a valid
renderer and severity, and - the one that matters - the endpoint must be a REAL
route. A declaration pointing at a route nobody wrote is exactly how the old
widgets rotted unnoticed for months, and now it fails the build instead.
Wave one complete. Three cards, no new data and no migrations.
Printer supplies reuses the existing low-supplies query and its five-minute
cache; a Zabbix round-trip per printer on every dashboard load would make this
the slowest page in the app. One row per printer listing every depleted
cartridge, criticals first - a row per cartridge would report one printer three
times and read as three problems, and showing only the worst class would hide a
low cartridge behind a critical one on the same machine when whoever walks out
there wants to carry both.
While there: the low-supplies REPORT itself was including healthy cartridges. A
printer with one empty black and three full colour ones listed all four, so the
reader had to find the problem inside the row. It now lists only what needs
replacing, and the test that asserted the old behaviour now asserts the new.
Expiring warranties keeps already-expired entries on the list rather than
dropping them the day they lapse, which is how they get missed. Horizon is
warranty_expiringdays, default 90, because that suits a site budgeting
quarterly and nobody else.
Mis-numbered bays promotes check-shared-machines out of a CLI command nobody
will remember to run - it found seven bays that had been wrong for weeks. It
reports only numbers with NO child assets, so part markers legitimately sharing
an operation stay silent: that distinction is the whole card, and without it it
would list correct data beside faults and be ignored.
Printers also loses its dead component-named widget; notifications, network and
machines still have theirs.
The stale-backup card could not be built as designed, and the reason is more
important than the card. Dedup means an unchanged configuration writes no
revision, so collectedat moves only on a CHANGE. A machine stable for six
months has a six-month-old newest revision and is perfectly healthy. Keying a
staleness card on revision age would have flagged most of the fleet - exactly
the noise that makes a board worth ignoring.
Underneath that: ShopDB could not distinguish those cases at all. On a no-op
the server returned "unchanged" and wrote nothing, so "we checked yesterday and
it matched" was discarded. That fact is the one thing a backup system must be
able to prove, and the only record of it was a line in a log file on the PC.
lastseenat records the check rather than the change. Touched on every matching
post including the no-op; set on creation, since a new revision has by
definition just been seen; backfilled from collectedat or createdat so existing
rows start from the last moment the config can be PROVEN current, rather than
from now - claiming a check that never happened would be worse than silence.
The card keys on it, one row per CHAIN rather than per asset: a machine with
two part markers can have one still reporting while the other stopped, and a
per-asset view would report the machine as fine. It stays deliberately silent
about assets never backed up, because whether one SHOULD be is a question only
the manifest can answer, and guessing would list a hundred healthy machines.
The rule lives in services/staleness.py rather than the route, so it is
testable without an auth layer in the way - the same split retention.py uses.
Threshold is backups_staledays, default 3, and 0 disables the card.
Second wave-one card. GET /api/computers/dashboard/quiet lists two populations
and deliberately does not merge them into one count. A PC that reported and
went quiet is probably off, moved or broken. A PC that has NEVER reported is
worse: not enrolled, or enrolled against the wrong pc-type, so nothing enforces
anything on it and no backup of it exists. That one hides indefinitely because
nothing about it fails loudly - the same shape as the bay that carried a wrong
machine number for weeks.
Never-reported sorts above the merely quiet, then longest silence first: the
order someone should work down the list, not the order rows left the table. A
soft-deleted PC is excluded - a decommissioned machine is silent on purpose,
and listing it would train people to ignore the card, which is the failure this
whole board exists to avoid.
The window is computers_quietreporthours, default 24, because every site will
disagree with any number picked here (ADR-015). A malformed value falls back
rather than failing the card.
This also replaces the computers plugin's old widget declaration, which named a
component nobody ever wrote. Four such declarations remain and will convert as
their cards arrive.
Two fixes to the renderer found while wiring this up. Meta specs now support a
trailing unit, so a row reads 'quiet for 3 days' rather than 'quiet for 3'. And
the card styles hardcoded hex colours against the frontend standard, including
a var(--card-bg) that DOES NOT EXIST - the variable is --bg-card - so the
fallback would have painted every card white and broken dark mode entirely.
Now --bg-card, --border, --danger, --warning, --primary and --link throughout.
The frontend now calls /api/dashboard/widgets. It never had, which is why five
plugins have been declaring widgets into a void for months, pointing at
components nobody ever wrote.
Core owns three generic renderers - exceptions, metric, list - and a plugin
declares data, a shape and a link template. The mapping logic lives in a plain
module beside the component, the same split as pluginAssetPanels.js, so it is
unit tested without mounting anything: 16 tests covering row mapping, empty
handling, ordering and gating.
The behaviours worth naming, because each is a decision rather than an
implementation detail:
Cards fetch INDEPENDENTLY and a failure becomes null. One hung endpoint - a
Zabbix call, a plugin mid-upgrade - cannot blank the board. A card whose fetch
failed HIDES rather than drawing empty, because "nothing wrong" and "I could
not tell" must not look the same.
Empty cards disappear by default. A card reporting nothing every day teaches
people to stop reading the page, which is precisely how a fleet log reached
3,234 lines with 17 that mattered. A card opts into a one-line presence only
when its absence is itself news.
Severity outranks position, so an info card can never sit above a failure.
Permission filtering happens BEFORE fetching: no point firing a request that
would only 403, and the dashboard must not become a way around RBAC.
An unknown render mode is skipped, so a plugin built against a newer core
degrades instead of leaving a hole.
A row whose link substitution is missing keeps the row and drops the link -
a PC shopdb does not know still reports its failure, and that is the bay most
likely to be misconfigured.
Cards sit ABOVE the totals: what needs a person first, context second. The
existing stat cards are untouched for now.
Wave one of the dashboard proposal, built as a vertical slice so the contract
is proven by something real before the other five cards follow.
GET /api/geenforce/dashboard/failures lists entries that FAILED on their PC's
most recent enforcement cycle. Per ENTRY, not per report: "three PCs failed" is
a number, while "Install OpenText failed with exit 1603 on WJSF1234" is
something a person can act on. Only current reports count, so a failure that
has since been fixed clears itself instead of needing dismissing. Hostnames
resolve to computerids in one query so each row links to the PC, and a PC
shopdb does not know still appears - the failure is real even when the
inventory is behind, and that is the bay most likely to be misconfigured.
The data has been there all along. The only way to see any of it was to open
one PC's report modal, one PC at a time.
The widget declaration is the contract change. The old shape named a Vue
component per widget, which cannot survive a lean build where a plugin's
component may never be staged into the bundle - which is exactly why five
plugins declare widgets pointing at components nobody ever wrote. This declares
data, a generic renderer, a permission and a link template, the way ADR-010
already does for asset panels. A test asserts no 'component' key, so the old
shape cannot creep back.
empty: hide is part of the contract, not decoration. A card reporting "nothing
wrong" daily teaches people to stop reading the page, which is how a fleet log
reached 3,234 lines with 17 that mattered.
Frontend rendering comes next; the endpoint and declaration stand alone and
change nothing that exists.
The dashboard reports totals that are true every day and actionable on none of
them. Meanwhile a bay returned 500 to every collector report for a day and a
half, eleven markers' backups overwrote each other for weeks, and GE-Enforce
records failures, stale manifests and repeated self-heals that surface nowhere.
Every one of those was found by someone going to look.
Two things found while writing this shaped it. The plugin mechanism already
exists - get_dashboard_widgets, an endpoint that aggregates across enabled
plugins and isolates failures, and five plugins declaring widgets. And nothing
consumes it: the frontend never calls the endpoint and not one of the declared
components was ever written. So this finishes a half-built feature rather than
starting one.
It also proposes fixing the contract while it is still unused. Naming a
component per widget does not survive a lean build, where a plugin's component
may not be staged into the bundle at all. ADR-010 already solved this for asset
panels - declare data and a render mode, let a generic core component draw it -
and the same three renderers cover every card listed.
The rest is principles that came out of this week: empty cards shrink, because a
card that says "nothing wrong" daily trains people to stop reading, which is how
a log reached 3,234 lines with 17 that mattered; every row links to the thing;
cards declare a permission so the dashboard cannot become a way around RBAC;
thresholds are settings with neutral defaults.
Wave one is six cards whose data is already reliable. Wave two is the valuable
one: diffing what a manifest says a PC should have against what it reports
having, which nothing answers today and which is what GE-Enforce exists to
guarantee.
Product version to 0.9.0, frontend in lock-step, Unreleased notes moved into a
dated section per ADR-007. The plugin contract stays at 0.16.0: it moved after
v0.8.1 and is already recorded in this release's notes, and nothing since
touched the contract surface.
A minor rather than a patch: collector behaviour changed in ways an integrator
must know about. A reported machine number no longer becomes the PC's asset
number, it builds a controls link instead; a second PC claiming a machine is
treated as a claim rather than a handover; and a backup revision chain is now
per source PC rather than per asset.
The check-shared-machines example carried two actual prod hostnames. docs/ is
excluded from the code bundle and goes to the GitHub wiki, so an example is the
wrong place for live fleet identifiers. Replaced with placeholders.
The MECHANISM was already a documented platform contract - ADR-001 defines
partof as composition, makes controls propagate through it, and walks it first
for map-position inheritance. The part-marker work used that rail rather than
inventing one.
What was undocumented is the PATTERN built on it: several devices answering to
one identifier, each becoming its own asset filed under a parent. It existed
only as a collector behaviour for part markers plus a docstring in the device
map, so nothing told anyone how to apply it to another device type, or when not
to.
ASSET-COMPOSITION.md covers when to reach for it and when the shared identifier
is a numbering fault instead, what propagation buys, how to declare a device
type through the map or a per-site setting, what a backup kind must do to
follow the device rather than the parent, how to find the next case with
check-shared-machines, and why the parent is not disposable once devices hang
off it - deactivating it breaks filing, and a hard delete cascades through
backuprevisions.
A PC that drives a device which is its own asset had been implemented twice.
METROLOGY_TOOL_MAP covered CMM, Keyence, Genspect and wax-trace, minting a
measuring_tool. A separate path keyed on one hardcoded pc-type minted a Part
Marker machine and filed it under its operation. Both create a device, link the
PC with controls, and archive that link when the PC is re-imaged: one mechanism
with different nouns, written out twice because the second case arrived later.
That is the same trap as the site literals in ADR-015 - a pattern implemented
per instance rather than declared - and it has a known next occurrence. Part
markers already share operation numbers, and any site with two marking lasers
or two wax-trace units on one number needs identical treatment.
One SUBORDINATE_DEVICE_MAP now declares asset type, type name, naming suffix,
whether the device files partof the operation, and the relationship label. The
labels are unchanged per case on purpose: those values are in the production
database and only rows carrying them are archived by a collector push. A site
overrides or adds an entry through subordinatedevice_<pctype> settings, per
ADR-015, so the next case needs no code. A malformed override falls back to the
default rather than failing the push, because a bad setting must not stop a bay
reporting its inventory.
metrology_tool_for stays as a shim over the same map: filters.py and the older
tests read it, and unifying must not change what it returns. A test pins that.
Also adds flask relationships check-shared-machines, which finds the next 0615
rather than waiting for someone to notice duplicate backups. Several devices
legitimately sharing a number and two PCs mis-numbered at imaging look the same
from outside; the difference is whether child assets exist, so that is what it
reports. Read-only.
Controller and share credentials are currently edited as cleartext into a
script on the SFLD share, one value for the whole fleet, re-applied by
GE-Enforce every cycle. Rotating means editing that file and hoping - nothing
reports which bays picked the new value up - and sites need per-bay variation,
which a manifest scope cannot express because scopes are per PC TYPE. A scope
per bay is a spreadsheet, not a model.
ShopDB therefore owns the definitions and the targeting, and a PC receives a
decided answer rather than rules to evaluate. Targeting uses the axes
GE-Enforce already filters on plus those the asset model knows: hostname
(including -like globs), machine, model, controller OS, PC type with alias
expansion, GE-Enforce profile, and a selector over the stored DNC projection -
"every machine whose DataHost is that share" is answerable from data already
held, and it also answers which machines need a credential at all.
Overlapping matches are guaranteed rather than exceptional, so precedence is
explicit: an integer priority, then scope specificity, then rule id for
stability. The resolver must be able to explain which rule won and why, or
nobody will trust it. It reuses plugins/geenforce/filters.py; there are already
two implementations of this matching logic kept honest by a parity harness, and
a third would drift silently.
Targeting stays separate from detection, mirroring PCTypes versus
DetectionMethod: who gets a credential is a server question, whether it is
already applied is a local one, and conflating them rewrites the secret every
cycle instead of healing drift.
Stored Fernet-encrypted with the key OUTSIDE the database, write-only through
the API and masked on read, fetched with a dedicated credentials.fetch scope so
a leaked collector key does not yield controller passwords, and every fetch
audited.
PROPOSED, not accepted: it makes ShopDB a credential store, and the key becomes
a single point of loss - restore the database without it and every credential
is unrecoverable, which docs/BACKUP-RESTORE.md must state in the same change
that implements this. Registry-only provisioning is recommended as a first step
regardless, since it removes the cleartext from the share immediately and the
client helper is identical either way.
Backup-NtlarsSettings reads backups_intervalhours from /api/settings/public,
because it runs before it holds any credential, and its Get-IntervalHours falls
back to 24 on any failure. The plugin never declared the key public, so the
endpoint did not return it, the fallback fired on every PC, and the setting
looked configurable in the UI while changing nothing. The fleet log shows the
symptom plainly: "Throttled: last attempt under 24h ago", every cycle,
regardless of what the setting said.
Declared public. A collection cadence is not a secret. backups_shareroot stays
private - it is internal topology - and the test asserts both directions so a
later edit cannot quietly widen it.
Same defect as the 3D parts kiosk label prefix already in this changelog: a
logged-out reader against an allowlist its key was not on. Worth noticing that
the pattern has now bitten twice.
ShopDB is a product several sites adopt, developed at one site against that
site's live fleet, so every feature arrives carrying West Jefferson's
hostnames, networks and share paths. A scan of plugins/ and shopdb/ returns 19
site literals, of which about 8 are a shipped default or a hardcode rather than
a comment: a kiosk URL baked into a PowerShell payload a sister site cannot
override, a WJ UNC path as the backup share default, and device.geaerospace.net
shipped as a SETTING DEFAULT - inherited silently by anyone who adopts.
The mechanisms already existed; the rule did not, so each value landed wherever
was convenient. The rule: a setting with a NEUTRAL default first, a
site-namespaced directory second, seed data third. Blank must be a working
state. Fleet-wide vocabulary such as the gea-shopfloor-* imaging types is not
site data and is out of scope - it already has a per-site override.
Enforced by an addition to check-naming-and-style.sh, which CI already runs, in
REPORT-ONLY mode. Making a 19-item backlog a hard gate the day it lands blocks
every unrelated commit until someone clears it; SITE_LITERALS_ENFORCE=1 turns
it into a gate once the listed hardcodes are done. The ADR carries that backlog,
including that the display FQDN domain is defined three times in three files.
COLLECTOR-INTEGRATION.md told integrators that machinenumber maps to
Asset.assetnumber and that a real value updates it on an existing PC. That is
precisely the behaviour that returned 500 to every bay reporting a machine
ShopDB already knew, and it is the document another site reads to build its
reporter.
Rewritten to say what the field is for: it identifies the machine, the PC keeps
its hostname, and the number builds a controls link. Adds a section covering
the claim-versus-handover rule and its quiet window, the alert gate defaulting
off, and how a part-marker PC gets its own marker asset under the operation.
Backups collected before markers became assets were filed against the operation
number the PC reported. The collector now files a marker PC's backup against
its marker, but only for backups collected since; everything already stored
stayed where it was, so the new marker assets read as empty while the operation
holds a mixed history.
The attribution is exact rather than a guess. A revision records the PC it was
read from, and the collector records which marker each PC drives, so the move
is that mapping applied. Revisions with no source hostname stay put, since
nothing says which marker they came from.
Moving a revision can duplicate one the collector has already written against
the marker, so the script collapses those as part of the move, keeping the
earliest so the date a config was first seen survives. Dry run by default.
A PC first seen before the machine-number fix was created with the machine
number as its OWN asset number, and the fix deliberately does not overwrite an
existing PC's asset number. So resolving the reported number can return the
reporting PC itself, and the marker was then filed partof its own PC - which
reads, on the machine page, as the PC being the operation.
The machine-link path already guarded this case; the marker path did not. It
now refuses and says why, naming the repair: rename the PC asset to its
hostname, or create the operation asset.
Several Telesis markers serve one operation number - 0613, 0615 and WJPRT each
have more than one - so treating the operation as the marker collapsed separate
devices into a single record. Their configurations differ by COM port, so in
the backup history they overwrote each other, and no question about an
individual marker could be asked at all: how many there are, which port one is
on, which one failed.
There is one marker per PC, which makes the PC the marker's identity, so the
collector can mint the marker the same way it already mints a CMM or a Keyence
unit for a metrology PC. A marker PC now gets a Part Marker machine asset, the
PC controls it, and the marker is partof the operation whose number the PC
reports. An operation holds any number of markers.
A marker PC therefore does not claim the operation directly. controls
propagates through partof, which reference-data already seeds, so control of
the operation still follows from controlling its marker - without two markers
contesting a link only one of them can hold.
Backups from a marker PC resolve to the marker rather than the operation, and
fall back to the machine number whenever the marker cannot be resolved: no
hostname on the payload, a lean build without the computers or machines plugin,
or a marker PC that has not reported to the computers collector yet. Filing
under the operation is the old behaviour and beats rejecting a backup.
Moving a marker to another operation archives the old membership rather than
deleting it, so where a marker used to live stays answerable.
Dedup compared a posted config against the latest revision for the ASSET,
which is only correct when a machine number means one PC. Several PCs share one
here: the part markers on 0613, 0615 and WJPRT are separate devices, differing
by COM port, filed under one machine number. Each marker's post therefore
differed from whichever marker had posted last, nothing ever deduped, and the
table grew by one row per PC per collection cycle.
A chain is now (asset, kind, source hostname). An unchanged config is a no-op
again, and each PC keeps its own history against the machine. NULL sources -
rows written before the column was populated, and hand-loaded ones - form their
own chain via IS NULL; `column == None` never matches in SQL, so without that
those rows would have re-posted forever.
Two consumers assumed the old key and are fixed with it. Retention pruned per
asset, so a busy marker's revisions could evict a quiet marker's only backup;
it now prunes each chain separately, protecting the newest and oldest of each.
The revision diff compared against the previous revision on the machine, which
across two markers reported one device's COM port as a change on the other; it
now compares within the source's own chain.
scripts/collapse_duplicate_backup_revisions.py cleans up what the old rule
wrote. It removes only a revision whose hash repeats the one before it in the
same chain - rows the fixed code would never have written - and keeps every
genuine change, every chain's newest and oldest, and every source. Dry run by
default. Its --report mode explains what grew each chain, which separates a
legitimately shared machine number from two PCs wrongly carrying the same one,
and from a value inside the config that changes on its own.
Treating "another PC is linked to this machine" as proof of replacement was
wrong. A PC imaged for machine 3010 carries that number from the bench, before
it has replaced anything, and several PCs sharing one machine number is a
normal state at this site: the part markers do it. Both PCs then reported on
their own schedules, each report moved the link and raised an alert, and the
pair traded the machine back and forth for as long as both were alive.
The PC holding a machine now keeps it while it is still alive. Alive means it
has reported within MACHINE_CLAIM_QUIET_HOURS and its asset is still In Use. A
challenger is recorded as a dormant link instead, which doubles as the marker
saying the claim has already been announced, so a PC sitting on a bench does
not alert on every collector cycle.
The handover still happens on its own once the old PC has been quiet for a day,
which is what a PC pulled off a machine does. Moving the old PC off In Use -
Retired, Inventory, In Repair - hands the machine over on the next report,
which gives IT a one-step way to force a swap the moment it happens rather than
waiting out the window. A day is long enough that a PC switched off overnight,
or one behind a network outage, never loses its bay to a spare.
Alerts for both cases are gated on a new computers_machinelink_alerts setting
and ship OFF. Several part markers legitimately share a machine number here, so
the alerts would fire on correct data. Links, warnings in the collector
response, and archived history are unaffected; only the sending is gated.
Also: the alert goes through send_alert rather than resolving recipients by
hand, which had missed the SMTP_ALERT_RECIPIENTS environment fallback, so a
site configuring SMTP by environment would have got the webhook and no email.
A bay reporting machinenumber 3015 got a 500 from the collector every five
minutes since it was imaged, and would have forever: the reported number was
written to the PC's own assets.assetnumber, which is uniquely indexed and
already held by machine 3015, so the insert failed with "Duplicate entry '3015'
for key 'ix_assets_assetnumber'" and the entire report was discarded. Operating
system, boot time, applications, printers and access protocols never landed.
Every retry did the same thing, so there was no path out of it.
A new PC now takes its hostname as its asset number, which is what the data
already shows: of 289 computers none has a numeric asset number and 214 use
their hostname. An existing PC's asset number is left alone; overwriting it
renamed the PC onto the machine's identifier, changing how that PC is
identified everywhere else.
The machine number instead does what it was collected for. It resolves the
machine and links the PC to it with a 'controls' relationship carrying a
collector:machine origin label, the same discipline the printer and
measuring-tool links use, so a link made by hand is never archived by a
collector push. Reporting a different machine archives this PC's previous link;
a machine ShopDB does not know is reported as a warning rather than invented.
When another PC was already linked to that machine it has been replaced. The
old link is archived rather than deleted, so which PC ran a machine in a given
month remains answerable, and an alert goes out by email and webhook. The
retired PC's status is deliberately not changed: the collector cannot tell
whether it was shelved, sent for repair or re-imaged for another bay, and
guessing would overwrite what a person set.
A line of the form "Name=" with nothing after the sign is not strictly legal,
but it occurs in real exports - the part marker's WJPRT.reg has KRelay1 like
this. The parser raised on it, which failed the whole file, which meant that
machine could never be backed up at all. Read it as an empty string so the
value name is still preserved.
The machine map preview was a bespoke popover. LocationMapTooltip already
existed and is what the machine detail page uses, so the warranty tables now
show the same thing.
That is what was asked for - the preview zooms, as it does on the machine page -
and it comes with behaviour the bespoke one did not have: scroll-wheel zoom, and
staying open while the pointer is on the tooltip itself so it can actually be
read and panned.
It also solves the clipping properly. The previous commit reached for
position: fixed with a hand-rolled flip because .table-container's overflow-x
clipped an absolute child; LocationMapTooltip teleports to body, which avoids
the clipping context altogether rather than escaping it. All of that
positioning code, and the blueprint rendering, is deleted - 64 lines from 140.
An unplaced machine keeps its chip and now says why in the native tooltip
rather than showing an empty panel: 17 of the 142 linked machines have no map
position, and those rows still need their number.
The bottom row's preview was cut off behind the pagination controls.
.table-container sets overflow-x: auto, and ANY non-visible overflow makes an
element a clipping context, so the absolutely-positioned preview was cropped at
the container's edge. The last row had nowhere to open into.
The preview is now positioned fixed from the chip's bounding rect, which
escapes the clip entirely, and flips ABOVE the chip when there is not room
below. It also clamps horizontally, since the column sits well to the right on
a wide table and the panel is nearly 300px.
Height is measured from the rendered element rather than assumed: it depends on
the blueprint's aspect ratio, which differs per site. That means one tick where
the element exists but is unplaced, so it starts hidden and is revealed once
positioned - otherwise it flashed in the corner of the screen.
Also sets white-space: normal. The table sets nowrap for its cells, which the
preview inherited and which ran the location line off the panel.
The Add User dialog was see-through over its own overlay in dark mode. Both it
and the audit-log detail dialog set background: var(--bg-card), which is
translucent in dark mode - frontend/CLAUDE.md already names this trap and says
a modal panel must use var(--bg-card-solid). The shared Modal.vue and the
.modal rule in style.css were already correct; these two hand-rolled panels
were not.
Audited every modal-ish panel in the app while here. The remaining
var(--bg-card) uses are correct and deliberate: .relationships-section and
.permission-category are cards, and the audit-log .filters rules are form
controls - translucency is the intended look on all of them.
One more was worth changing: the warranty asset-search dropdown floats above
the form INSIDE a modal, so a translucent panel let the fields show through the
suggestions. That one is now solid too.
A shopfloor PC is bought, warranted and replaced as a PC, but it is FOUND by
the machine it drives - nobody walks the floor looking for an asset number. The
warranty tables listed the covered asset and left the reader to work out where
that is.
Both tables gain a Machine # column. The payload resolves it by walking the
asset relationship graph in BOTH directions: the canonical edge is
PC --controls--> machine, but a dual-bay pair carries controls on both bays and
hand-made links are not reliably oriented.
Hovering the chip shows the floor map with the machine marked, so the row
answers "where do I go" without opening anything. The blueprint follows the
viewer's theme and the marker is placed from mapx/mapy as a percentage of the
configured map dimensions, since the preview is a few hundred pixels wide
rather than the full plan. It renders only while hovered, so a long table does
not build a blueprint per row.
A machine with no map position still gets its chip and says so, rather than
being dropped: against real data 142 PCs resolve to a machine and 125 of those
are placed, so 17 rows would otherwise have silently lost their number.
The chip is deliberately not a link. /machines/:id is keyed by machineid, not
assetid, and resolving one to the other here would make the warranty plugin
import the machines plugin (ADR-014). Worth noting separately: the existing
assetLink() in these tables already sends machine-type assets to
/machines/<assetid>, which is that same mismatch and predates this change.
Reported from a real display: Edge closed on the desktop but still listed in
Task Manager, and the watchdog never relaunched it.
That is this watchdog's own bug. It asked only whether a --kiosk process
existed. After an Edge update the window can be gone while the process lingers,
so the check said "kiosk is up" and returned - every cycle, forever. Matching on
the command line was chosen to stop a stray renderer masking a dead kiosk; it
does not help when the orphan is the parent.
A kiosk now counts as up only when a --kiosk process still owns a visible
window (MainWindowHandle). Windowless ones are killed BEFORE relaunching:
leaving them would satisfy the next cycle's check again, and a second browser
would fight the first for the display.
Also refuses to run as SYSTEM. MainWindowHandle is session-scoped, so a SYSTEM
caller reads 0 for a perfectly healthy kiosk and would kill and relaunch it on
every cycle. The task uses an interactive Users principal so this cannot
normally happen; the guard makes a mis-registered task fail loudly instead of
thrashing a display in a hallway.
Verified on Windows this time by letting the SCHEDULED TASK do the work rather
than invoking the script by hand - which is what the first version was missing:
- task fired unattended (rc=0) and launched the kiosk into session 1
- the next cycle saw the healthy kiosk and did nothing, no relaunch loop
- a windowless --kiosk process was killed and replaced
- run as SYSTEM, it refused and the healthy kiosk survived
Every time on the board was wrong, from two faults stacked.
The shopfloor feed serialised starttime/endtime with a bare isoformat(). Those
columns are stored NAIVE but hold UTC, so an untagged string is read by the
browser as LOCAL and every card shifted by the tz offset. The model's to_dict
already learned this - its _utc_iso helper documents the exact symptom, a 14:34
notification showing 18:34 - but the feed had not, so the feed now uses it too.
The dashboard then formatted with toLocaleString, i.e. the VIEWER's zone. A
board hangs on a wall in the plant: it has to read plant time whatever the
machine driving it is set to, and a kiosk with a wrong system timezone would
otherwise show wrong times to the floor with nothing to reveal it. It now
loads site_timezone and formats through formatInZone, the wall clock included -
a header disagreeing with the cards beneath it is worse than either being
wrong alone.
startsWhen was worse still: it decided TODAY/TOMORROW from browser-local
calendar days, so the wording itself could differ between the board and a
remote admin looking at the same card. That arithmetic now runs on the site's
calendar day.
Separately, the type chip carried a margin-bottom while the state chip beside
it did not. .chip-row centres each item's MARGIN box, so that margin lifted the
type chip about 4px and left "Starts Thu, Aug 13 8:00 PM" looking low. The row
already provides the spacing, so the chip's own margin is gone.
The accessprotocols / computeraccess tables replaced the old isvnc/iswinrm
booleans and the PC page already badges what a machine exposes, but nothing
kept them current: the 574 rows in place all came from the legacy migration and
have not moved since. The collector schema had no field for them.
Adds 'accessprotocols', a list of catalog names, synced with the same
discipline as the printer links. A reported protocol is activated; a
catalogued one the PC did NOT report is deactivated rather than deleted, so a
manual portoverride survives a service being briefly down. An unknown name
warns and is skipped: the catalog is admin-managed, and a typo on one bay must
not invent a protocol for the whole site.
Presence of the key is what drives the sync. A payload without it leaves every
existing row untouched, which is what protects the migrated rows from a
collector that does not report protocols yet.
Six tests cover recording, case-insensitive matching, deactivation on removal,
the omitted-key no-op, an explicit empty list meaning "exposes nothing", and
that an unknown name never creates a protocol.
Kiosks were updating, closing Edge, and never coming back - the display stayed
dead until the next logon or reboot.
The kiosk is launched by an all-users Startup shortcut, which runs ONCE at
logon, and nothing supervised the browser afterwards. RelaunchNotification=2
was meant to cover the update case and does not: that policy drives Edge's own
update-restart, which depends on session restore to return to where it was.
Kiosk mode restores no session and has no UI to show the notification in, so
Edge honours the close and never the relaunch. The same gap swallowed crashes
and anyone closing the window.
Adds a scope entry that registers a scheduled task in the INTERACTIVE session -
SYSTEM cannot launch a visible browser, which is why the dispatcher writes a
shortcut rather than calling Start-Process. The task relaunches from that same
shortcut, so the target URL keeps one source of truth: retarget a subtype in
DISPLAY_TYPE_TARGETS and the watchdog follows unchanged.
Two details that matter. It matches on the COMMAND LINE, not the image name:
Edge runs a crowd of msedge.exe children and only the parent carries --kiosk,
so testing "is msedge running" would let a stray renderer mask a dead kiosk
forever - verified against a real kiosk PC showing 7 processes and 1 match. And
it avoids -RepetitionDuration [TimeSpan]::MaxValue, which serialises out of
range and is rejected, exactly as the kiosk installer documents.
A launch debounce stops a display that fails to start from spawning a browser
every cycle, the log is size-bounded because this runs forever on a PC nobody
watches, and it does nothing at all when no kiosk shortcut is present so it
cannot put Edge on a PC that never asked for one.
Verified on Windows: registers with the right principal and triggers, relaunches
when the kiosk is gone, debounces an immediate re-run, and is idempotent across
cycles (the staged script compare is trimmed - Set-Content adds a trailing
newline the here-string lacks, so an untrimmed compare rewrote it every cycle).
The link had a custom hover tooltip that took the hidden events with
dayEvents.slice(3) - correct while dayMaxEvents was a fixed 3, wrong the moment
that cap started varying with the row height. On a short row showing one chip,
'+4 more' sliced from index 3 and listed the wrong events; on a day with three
events it sliced to nothing, hit the empty-list guard and rendered no tooltip at
all. A link with nothing behind it.
FullCalendar's own popover replaces it rather than the arithmetic being fixed:
no index to drift out of step with the cap, a header and a close button, every
event for that day listed, and it works on a touch screen - which a hover
tooltip never did on a kiosk. Clicking an event in the popover still opens the
detail modal.
That takes the hover handlers, the container mouseenter/mouseleave delegation,
the tooltip markup and styles, and the by-date index that existed only to feed
them: 108 lines out, 25 in. The popover is themed through the CSS variables,
since FullCalendar ships it light.
An audit of every scrollable element on the page, rather than the document
alone, found what the previous two attempts kept missing: FullCalendar's
day-grid scroller sits 16px short of its own content at every window size.
It is phantom. The table inside measures exactly the scroller's height, the
last week sits flush with the bottom, and the horizontal axis is clean - there
is nothing to scroll to, but the browser paints a bar for the gap. Shrinking
the calendar to close it does not work either: the gap stays 16px however small
the grid gets, so the previous pass had started shrinking the month for
nothing. That correction is reverted to page overflow only.
The grid is sized to fit its six rows, so the scroller has nothing to reveal;
hiding it is safe and is what the earlier arithmetic was failing to buy.
Audited with 60 events across 12 days at 1920x1080, 1600x900, 1366x768,
1280x720 and 1920x1200: no scrollable element left in the calendar, six weeks
visible, last week fully in view, badges shown with a '+N more' where a short
row cannot hold them all.
The nav sidebar still scrolls on a short window - 100vh of viewport against
about 1384px of menu - but that is every page, not this one.
The previous attempt at this was verified against an empty month, which is why
it measured clean and still scrolled: put events in the month and the fixed
height clipped the grid, leaving the last week behind FullCalendar's own
scroller. The media-query floors it added are gone.
Two things were guessed and are now measured. The height came from
calc(100vh - 230px), a stand-in for chrome that is not 230px tall; it is taken
from the container's real top at mount and on resize, then corrected against
whatever the page still overflows by. And dayMaxEvents was a fixed 3, which
does not fit an 80px row - hence the overflow. It is `true` now, so
FullCalendar shows as many chips as the row genuinely holds and rolls the rest
into a '+N more'; that is the only setting that cannot outgrow the box. The
chips and the day-number strip are tighter, so more fit before that happens.
With 44 events across 11 days, at 1920x1080, 1600x900, 1366x768 and 1280x720:
all six weeks visible, no page scrollbar anywhere. A full-height display shows
every chip; a laptop shows one and a link.
The floors that keep the month grid from squashing - 620px on the calendar and
110px per day cell - add up to more than calc(100vh - 230px) yields on a 768 or
720 tall window. The floors won, the page overflowed, and a month that used to
fit needed scrolling to see.
They relax below 900px tall: 78px cells, which still show the day number and
three events, so the grid stays readable rather than collapsing to the strips
the floors were guarding against. Taller windows keep the roomy cells.
Measured at 1920x1080, 1600x900, 1366x768, 1280x720 and 1080x1920 portrait:
six rows and no page scrollbar in all five.
Two of the board's colours were still hardcoded: carousel headings were gold
and grid headings blue, left over from when carousel meant Recognition and grid
meant Recertification. A type wearing another type's colour is worse than no
colour at all, and it was the whole of the "why did my category turn blue"
puzzle - it had turned from one style's hardcoded colour to another's.
Headings now take a colour from the types beneath them. Where a row holds
several types, the one with the most cards wins, ties going to whichever sorts
first on the board. Live cards decide it; the upcoming set only gets a say when
nothing on the row has started yet, so a heading always describes what is
happening now. That covers the standard rows too - the old getSectionClass
painted the list red off a legacy 'danger' keyword.
Upcoming cards say so plainly: STARTS TOMORROW 2:06 PM, or the weekday and date
beyond that, on an amber chip, with a dashed border and a hollow carousel pip.
Shape as well as colour, so it reads from across the floor and does not depend
on telling amber from white.
Cards and tiles also carry the type's colour as a bar down the left edge, the
way the legacy board marked them; a 2px border reads as chrome at distance.
Rows ran in display-style order and then alphabetically, so what led the screen
was an accident of styling and the alphabet - a new type called Awareness
landed above Recertification for no better reason than the letter A.
Each type now carries a board position, lowest first, set on the Notification
Types page. The migration seeds Recognition at 10 and Recertification at 20 and
leaves everything else at 100, so an existing board keeps the order sites
already expect. Steps of ten leave room to slot a row in without renumbering
the rest.
A row shared by several types sits wherever its earliest-ordered type puts it,
so a category moves as a unit.
Backup timestamps read wrong because of two faults stacked, which is why it
looked like a single offset.
The API serialised naive ISO ("2026-08-07T12:00:00"), with nothing saying the
value was UTC. JavaScript's new Date() parses that as BROWSER-LOCAL, so every
timestamp shifted by the viewer's offset before any timezone formatting ran.
Every datetime this plugin stores is naive UTC, so the wire format now carries
a trailing Z.
The history view then formatted with toLocaleString(), i.e. the viewer's zone,
ignoring the site_timezone setting entirely. It now loads that setting and
formats through the shared formatInZone helper, matching NotificationsList.
The panel list label is built server-side with strftime, so a client cannot
correct it afterwards. It now converts to the site zone using the same Setting
lookup the notifications plugin uses - without that it showed UTC, four hours
out at West Jefferson.
Tests cover the wire format and that 16:30Z renders as 12:30 in
America/New_York.
DNC Info becomes a single tabbed card - General, eFocas, Serial, NTSHR and
MARK - instead of a flat wall of every value. On 3204 that is 11 rows visible
rather than 22, and on 0600 eleven rather than 27, which also stops the card
unbalancing the detail page's two-column layout.
Everything DNC now lives on that one card, so the Part Marker panel is gone:
its settings are the MARK tab. The partmarker KIND is untouched and still
stores, dedupes and serves revisions - they are listed on the backup history
page - it simply contributes no card of its own, which on 145 of 147 machines
would have been an empty box.
Downloads are named for the machine: 3204.reg, and 3204-wow6432node.reg for
the dialect that imports outside NTLARS. The view had been rebuilding the name
from sourcefilename and producing 3204.reg-wow6432node.reg, so the revision now
carries assetnumber and both ends agree. That needed a viewonly relationship to
Asset - no backref, so the core asset side gains no dependency on this plugin.
The history page was hardcoded to light colours (#e0e0e0, #f4f9ff, #666) and
rendered as a white table on a dark page. It now uses the palette variables
throughout, per frontend/CLAUDE.md. The current-revision tint is a color-mix
against --primary so it reads in both themes rather than a baked light blue
that disappears on dark, and the diff columns are headed as well as red/green,
since colour alone does not survive a colourblind reader.
A panel with many fields was unusable. The detail page is a two-column
multicol (.content-column is display:contents, so the left/right wrappers are
dissolved and cards flow into one balanced flow), and a card cannot be split
across a column break. One 30-row panel therefore became a single unbreakable
block that dragged its column far past the other.
The tabs mode renders sections one at a time, so such a panel stays the height
of its largest section. Payload is {sections: [{label, fields}]}; sections that
end up with no fields are dropped rather than left as empty tabs. The active
tab is marked by an underline AND weight, not colour alone.
keyvalue rows now sit in two aligned columns instead of space-between: a ragged
right edge is hard to scan, and one long value (a UNC path) otherwise pushed
its label away from every other row.
Four specs cover the new mode, including that it drives panel visibility.
Every machine was getting a Part Marker Configuration panel, and only two of
the 147 known machines are part markers. A kind applies to an asset TYPE, but
whether a given asset ever carries that kind of backup is a property of the
individual machine, so type alone cannot decide what to show.
The generic renderer already handled this: a list panel is visible when it has
rows OR declares empty text. Declaring emptytext on both kinds defeated it and
forced them to render everywhere. emptytext now defaults to None on the base
class, neither bundled kind sets one, and the panel builder OMITS the key
rather than emitting null - a present-but-null 'empty' would still have kept
the panel on screen.
The DNC Info card carried empty text that could never be displayed, since
keyvalue visibility is decided purely on field count. Removed rather than left
to mislead.
A lathe now shows DNC Info and NTLARS history; 0600 and 0614 additionally show
Part Marker once something collects for them; a machine with no NTLARS data
shows no backup panels at all. The emptytext mechanism stays available for a
kind that genuinely wants to say "expected here, nothing yet".
site-profile-universal.json is what the released Windows installer is built
from, so a bundled plugin missing from it is invisible to the install wizard.
Worse, `flask plugin prune-schema` drops the tables owned by plugins the site
did not install (ADR-014), so backuprevisions would have been dropped at
provisioning on every new site - a table that shipped in the build, removed
because the profile never named it.
The same omission explains why `flask plugin upgrade-all` skipped backups:
upgrade_all_plugins iterates the REGISTRY, not the plugins directory, on
purpose - a plugin folder merely sitting on disk unadopted must not have its
DDL run as a side effect of a deploy. instance/ is gitignored, so any machine
that never ran `flask plugin install backups` has it on disk but unadopted.
Nothing in the suite caught the stale profile, so this adds two guards: every
bundled plugin carrying a manifest must appear in the universal profile, and
the profile must not name a plugin that does not exist. Verified the first one
fails with the profile as it was.
The view linked to /assets/<assetid>, which is not a route. Each asset type
has its own detail route keyed by its own id (/machines/<machineid>), and
there is no generic asset detail page to return to, so the link 404'd.
Uses router.back() instead: this page is only ever reached from an asset
panel, so history is the correct destination regardless of the asset type,
and it needs no per-type route table. Falls back to the machines list when
opened directly from a pasted URL.
The /api/assets/<id> call that fetches the heading name is unaffected - that
endpoint does exist; it was only the frontend route that did not.
Scheduled notifications with a special display style rendered nowhere at all.
styledGroups only read notifications.current, so anything upcoming fell through
to the standard Upcoming list, which drops banner/carousel/grid styles. A
scheduled banner was therefore invisible until the moment it went live. Upcoming
items now join the styled groups, tagged so they can be told apart, and each row
sorts live cards ahead of not-yet-started ones.
State chips name what a card is doing: UPCOMING with its start time, or
RESOLVED for the tail a type's grace window buys it. Pending cards also dim to
0.72 so a glance never mistakes one for live. The dim rule is scoped to
.recognition-card.active.pending on purpose: inactive carousel cards sit at
opacity 0, and an unscoped .pending outranks that, which stacks the whole
carousel back into view at once.
Rows keyed by board category can hold more than one type, and the heading then
speaks for none of them. Such rows keep a neutral heading instead of wearing
the first type's colour, and each card carries its own type chip. The chip is
named rather than only coloured, since colour alone does not survive a
colourblind reader at board distance.
Carousel rows gain one pip per card, so a reader can see how many are in the
rotation and how long until theirs returns.
Adds a kind-pluggable backups plugin. Configuration captured from a PC is
filed against the MACHINE it controls, with a revision history and download
back to the native format.
NTLARS/DNC is the first kind. Settings live in the controlling PC's registry
but describe the machine, so revisions attach to the machine's asset and carry
no foreign key to the PC: history survives a PC being replaced or deleted, and
sourcehostname records the handover.
Storage splits by kind. Parseable kinds store a dialect-neutral JSON
projection in ShopDB and re-render on download; opaque vendor formats (part
marker and similar) keep their bytes on the SFLD share with ShopDB holding
metadata and the UNC pointer.
Two .reg dialects exist in the wild: NTLARS's own Save... export omits the
WOW6432Node path segment, scripted exports include it. Parsing strips whichever
root matched, so a stored revision commits to neither and download offers both
(NTLARS Load... by default, WOW6432Node for direct reg import). Getting this
backwards is silent, so the dedup hash deliberately excludes sourcedialect and
both dialects of one config dedup to a single revision.
Dedup is load-bearing: the collector runs every GE-Enforce cycle across the
fleet, so a revision is inserted only when the content hash differs from that
asset's latest for that kind.
A freshly imaged PC opens NTLARS with a blank General tab. Recording that would
make an empty config the newest revision exactly when someone needs the last
good one, so a blank MachineNo is rejected rather than accepted as a change.
Two of the 320 known-good backups on the share already have that shape.
DNC Info card summarises the latest revision on the machine page: General
(Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147
machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER
or the ShopDB machine type, not on the MARK key having content: MARK carries
serial defaults on 145 of 147 machines and DncPatterns reads YES on 103
including ordinary lathes, so neither identifies a marker.
The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served
by a generic endpoint, so the expected successor to DNC ships its own card by
adding a class rather than changing the plugin or the panel wiring.
Also: schedule and retention settings with a prune that never drops the newest
or the oldest revision, and scripts/import_ntlars_backups.py to seed history
from the existing per-machine .reg files (144 of 147 resolve to assets).
Codec verified against all 320 real backups: round-trips clean through both
dialects. Bay-side generation verified on Windows against reg.exe export.
The carousel gold and the grid blue were hardcoded in the dashboard's CSS,
which was fine while carousel meant Recognition and grid meant Recertification.
Now that any type can take either style, a type arrived wearing somebody else's
colour: a green Kudos row came out gold, and every grid row came out
Recertification blue regardless of what was picked in the type settings.
Row headings now take the type's colour, and cards and tiles take theirs
through a --accent property, so a shared category row shows a red incident tile
beside a purple change tile. Heading text colour is computed from the
background's luma rather than listed per colour, since a site picks its own
hexes - gold needs dark text, navy needs white.
The card backgrounds were gold- and blue-tinted gradients; they are neutral
dark now so the accent is what carries the colour. The old values remain as CSS
fallbacks for a card that somehow arrives with no type.
--bg-card is deliberately translucent in dark mode (rgba(0,0,61,0.4)) so cards
glass over the page; --bg-card-solid exists for the things that must not. Two
hand-rolled modal panels used the former, leaving the notification-type and
access-protocol editors transparent over the overlay with the table legible
through them. The shared .modal in style.css already got this right.
Also writes down the page-vs-modal rule the codebase already follows, since
nothing stated it: a record with a detail page gets a routed form page, a lookup
row that only exists inside its list gets a modal over that list. Plus the modal
rules from the overlay-close fix - data entry never closes on a stray click,
confirmations may, and panels are painted solid.
The network device form had no IP field, so the one thing people look up a
switch for could not be entered. The API had accepted an ipaddress since the
legacy import work - create, update and every read already carried it - but
nothing in the UI ever sent one, which also left the device off the "Devices on
this network" list, since that matches on the IP a device does not have.
The field sits next to Hostname and round-trips through the Communication row
the platform keeps IPs in, the same way the PC and printer forms do.
Clearing it now clears the stored address. The upsert helper returned early on a
blank value despite a docstring promising it cleared - harmless while no form
could submit one, wrong the moment this field existed.
The shopfloor board grouped cards by display style alone, so every type set to
grid landed inside the Recertification row and every carousel type inside
Recognition's - under a heading naming somebody else's type. Setting Awareness
to grid put awareness messages under "Recertification Required".
Each type now gets a row of its own, titled by its own name, and rotation state
is per row: two carousel rows advance on their own indexes instead of sharing
one counter, and two grid rows page independently.
For the other direction there is notificationtypes.boardcategory. Types sharing
a category share one row under the category name, so Change, Awareness and
Incident can sit together while Recognition and Recertification keep their own.
Blank - the default - means a row of its own. The category is part of the
grouping key along with the display style, since a category cannot merge a
banner with a row of tiles.
A card that names no employee now renders its message as the tile or card,
rather than a placeholder face above a blank name, which is what a grid type
like Awareness looked like before.
The layout fingerprint that makes open kiosks reload now covers the category
and the grace window, so a re-grouped board reaches screens that are already up.
Eight tiles across left each one too narrow to read at floor distance. The row
now shows four per page in a 2x2 block, cycling on the same seven-second timer,
so a name is legible from where people actually stand.
The tiles lay the photo beside the name rather than above it. Two rows of
stacked tiles would roughly double the section's height, and the board is
scaled to fit one screen, so height spent here shrinks everything else. The
photo grows to 120px and the name to 26px in the space that buys.
The shopfloor feed kept every ended notification up for a hardcoded 30 minutes,
flagged resolved. A card with an 8:00 end time was still on the board at 8:29,
which reads as an expiry that did not work - and in the carousel, grid and
banner sections it read that way with no visual sign at all, since only the
standard cards render the resolved state.
The tail is now notificationtypes.gracewindowminutes, set per type on the
Notification Types page and defaulting to 0, so an end time means what it says.
A type whose cards are worth acknowledging after they clear - an incident, say
- opts into a tail, and only that type's cards get one.
The feed widens its query to the largest configured tail and then holds each
row to its own type's window. That keeps one portable query rather than a
per-type interval expression in SQL, and with every type at 0 it collapses to
"still showing".
Also fixes resolved serializing as null rather than false for a card with no
end time, which the and-chain produced.
The kiosk runs full-screen on a wall-mounted display, where the interface sat
against the top edge with the rest of the screen empty below it. It now
centres as one block, vertically and horizontally.
The title was the other half of it: the header was space-between, which reads
as centred only on the steps that show the Start over button. On the first
step, with no button to balance it, the title sat alone at the left edge of
the column. The title is centred and Start over is taken out of the flow so it
keeps its corner without shifting the title on the steps that have it.
Centring uses `safe center`, with plain `center` as the fallback line. On a
screen too short for the content, plain `center` overflows in both directions
and the header ends up above the scroll origin, unreachable. `safe` falls back
to top-aligned there and the page scrolls normally.
Three defects, all found on printedparts_label_prefix, all one root cause:
nothing in the framework knew that setting existed.
The parts kiosk runs logged out. An unauthenticated read of a setting is
limited to an allowlist, the key was not on it, so the kiosk got a 404 and
fell back to no prefix. An admin previewing the same page while logged in saw
the prefix, which is why it looked like it worked.
The same setting also looked like it would not save. The row did not exist on
a site that installed the plugin before the setting was added, so the first
save created it - under the placeholder category the settings API uses for
keys it does not recognise, where the plugin's settings page, which lists by
category, could no longer see it. The value was in the database the whole
time.
And the row was missing in the first place because seeding ran from
on_install / on_enable, which fire only on a state transition. Neither runs
again on an upgrade, so a setting added in a later plugin version never
reached a site that installed an earlier one. The comment claiming enable ran
every upgrade cycle was simply wrong.
A plugin now declares the settings it owns in get_settings_defaults(): key,
default, type, category, description, and whether a logged-out page may read
it. The framework seeds declared keys at install, at enable, and on every
flask plugin upgrade-all; files a first-time write under the declared
category; re-homes any row left in the placeholder category, value untouched;
and answers an anonymous read for keys marked public. Core carries no list of
any plugin's keys.
Contract 0.16.0 (additive optional hook). printedparts and printers move to
the hook and floor their core_version at 0.16.0. The dev database had two rows
in the misfiled state (printedparts_alert_email, employee_db_host); the first
repairs itself on the next upgrade pass.
Reordering meant clicking the up arrow repeatedly - moving a slide from the
bottom of a long playlist to the top was a dozen clicks and a page of
re-rendering. Rows are now draggable, with a grip so it looks it.
Applies to BOTH surfaces: the manager already switches between Lobby Display and
Shopfloor Screensaver, so one change covers the lobby TV and the EventSaver
playlist.
The drop target is shown as a line on the row being dropped against rather than
by shuffling rows under the cursor, which reads as the list fighting the drag.
The hover preview is dismissed when a drag starts, or it would sit over the list
for the whole gesture.
Drag and the up/down buttons now share reorderTo(), so both persist through the
same call and both recover the same way: a failed save reloads from the server
rather than leaving an order on screen that looks saved and is not.
dataTransfer.setData is set because Firefox starts no drag at all without it.
The thumbnails are 120x68 and cropped with object-fit: cover, so any text on a
slide is unreadable and the edges are cut off. Picking the right slide to
reorder or delete meant opening images by hand to tell them apart.
Hovering a thumbnail now shows the whole slide, bounded by the viewport rather
than the image so a 3300x2550 upload does not fill the screen, with the filename
underneath.
Fixed position rather than inside the row: the list scrolls and a
relatively-positioned parent would clip it. pointer-events: none so the preview
can never sit between the cursor and the move or delete buttons.
Somebody standing at a display who wants to hold a slide, or go back to one that
has just passed, had to wait for the whole rotation to come round again.
Left and right arrows step back and forward; PageUp and PageDown do the same, so
a presenter remote works without being told about it. The listener is on window
rather than an element because a display has nothing focused.
Stepping restarts the rotation timer instead of leaving it running. Advancing by
hand and then having it move again a second later, because the existing timer
was nearly up, reads as the display ignoring the keypress.
Ignored entirely when there is less than one slide to move to.
The viewport-relative height can resolve smaller than the grid needs on a short
window - a laptop with a docked browser, or a display in portrait - which would
have reintroduced the squashing it was meant to fix. Three floors now: the
container, the calendar root, and each day cell.
The day cell is the one that matters: six week rows cannot render shorter than
about 660px however short the window, so the grid stays readable rather than
collapsing back to strips.
The calendar was set to height 'auto', which sizes each week row to its own
content, so a month of mostly empty days collapsed into thin strips. It now
takes a viewport-relative height and expandRows shares that evenly across the
weeks, with a floor under each day cell so a short window squeezes the grid back
down rather than the rows vanishing.
New notifications now have "Show on Shopfloor Dashboard" ticked. The board is
where these are meant to be read, and starting unticked meant most were written
and then never appeared on it.
Only the default for a NEW notification. Editing an existing one still loads its
stored value, so nothing that was deliberately turned off gets flipped back on,
and the column default is left alone so an API or import caller that omits the
field keeps the behaviour it has today.
The installer on the share was stamped 0.8.0 and contained sixteen commits that
were not in the v0.8.0 tag. A build that misreports its own version is exactly
what the version check in docs/RELEASING-WINDOWS.md exists to prevent, and it
would have left two sites unable to say what they were running.
Everything in 0.8.1 landed after v0.8.0 was tagged this morning, driven by two
sites entering real data for the first time: the blank-code 500, the two-slide
display that never rotated, modals discarding a part-filled form, filters
returning an empty page, model photos that could not be saved, and the shared
equipment catalog that lets a new site start with vendors, models and printer
supply part numbers already present.
CHANGELOG gains a 0.8.1 section, and the OpenAPI document follows __version__
rather than being restated.
CustomFieldsInputs renders nothing when a site has defined no custom fields,
which is the normal state. The network form was the only one wrapping it in a
fieldset, and a fieldset draws its border whether or not anything is inside, so
an empty bordered box sat below Notes on every new network device.
Machines, PCs, printers and measuring tools all place the component bare. This
now matches them.
Operators reported losing a part-filled form by clicking slightly outside it.
Every data-entry modal closed on a backdrop click with no warning and no way
back - the worst possible response to a misplaced click, and it happens most to
someone adding their first records at a new site.
Close-on-overlay is removed from 35 modals across 30 files: anything containing
an input, textarea, select or v-model. They still close by Cancel or the X.
Confirmation dialogs keep it, because a delete prompt holds nothing to lose and
dismissing one by clicking away is the behaviour people expect. VendorsList
shows the distinction - its edit form no longer closes that way, its delete
confirmation still does.
The shared Modal component now defaults closeOnOverlay to FALSE. Every current
caller holds a form, a checkout, a stock adjustment or a map position being
picked, and not one passed the prop, so all of them had the same fault. A modal
that genuinely wants dismissing that way opts in explicitly.
Also regroups the operator console menu, which had grown to numbers 1-9 plus
three letters bolted on with no order to them. Actions are now grouped by what
they touch, keyed by their first letter, and the old numbers still work so
nobody who has used it for months is stopped by a rearrangement.
The menu also warns when the server is not fully provisioned and names the key
that fixes it, instead of reporting it as ordinary status lines that read as
normal unless you already knew what to look for. That check is cached for the
session because it shells out to flask twice and the answer does not change
while somebody reads the screen.
The installer offers the catalog as a tick-box, which covers new sites and
nobody else. A site that installed before it existed, or unticked the box and
later changed its mind, had to be talked through an RDP session and a flask
command - which is exactly the sort of thing the console exists to avoid.
`shopdb-admin.ps1 catalog`, and C on the menu. It runs the dry run FIRST and
prints what would be added, then asks before writing: somebody running this on a
site that has been live for years deserves to see what it would touch before it
touches anything. Answering anything but yes leaves it alone.
An older build without the seed-catalog command is reported as such and told to
update, rather than the failure being read as an empty catalog.
Repair is now on the menu too. It was reachable only by typing the verb, which
is little use to the operator most likely to need it.
`flask seed reference-data` wrote a dozen generic model types and no vendors or
models at all, so adopting this platform began by retyping a catalog another
site had already spent a year building. That is the largest single obstacle to
standing a new facility up.
scripts/export_catalog.py dumps the catalog from a live instance to
shopdb/data/catalog.json, and `flask seed catalog` loads it. What travels:
vendors 53, models 128, modelsupplies 146, modeltypes 35, machinetypes 21,
computertypes 10, printertypes 9, networkdevicetypes 5, locationtypes 11,
operatingsystems 14, measuringtooltypes 8, notificationtypes 3,
accessprotocols 3
The 146 printer supplies are the most useful part after the models themselves:
every toner, drum and maintenance kit with its part number, colour, capacity
tier and page yield, already matched to the right model, instead of somebody
reading them off spent cartridges.
IDEMPOTENT and ADDITIVE. Records match on a natural key - a vendor by name, a
model by vendor plus model number, a supply by model plus part number - so a
second run adds nothing, and it never updates or deletes: a site that corrected
a description or pointed a model at its own photo keeps its version.
Catalog only. No assets, locations, employees, business units or anything with a
serial number: nobody wants one plant's machines appearing at another. Vendor
contact details are excluded too, since a rep's name and number belong to
whoever holds that relationship. supportteams, printerdrivers and customfields
are site-specific and deliberately absent.
Models and supplies reference their vendor by NAME rather than id, because ids
differ between databases and an id-keyed catalog would silently attach part
numbers to the wrong printer.
The installer offers it as a tick-box on a new "Starter data" page, defaulting
to on, passing -SeedCatalog to stage 3. Offered rather than assumed: a site that
machines nothing does not want 21 machine types cluttering its dropdowns.
Verified by loading into an empty database and running twice: every group
populated on the first pass, "Catalog already present, nothing to add" on the
second.
Two things a second site ran into.
The network device form asked for the map position as two raw numbers, so
placing a device meant reading coordinates off another screen and typing them
in. Machines, PCs and printers have had a "Set Location on Map" picker all
along, and the network API already accepted mapx and mapy - only the form was
missing. Same picker, same modal.
The 3D parts kiosk hardcoded 'WJ' as the prefix shown before the number box,
with a comment inviting whoever needed something else to edit the source. That
is West Jefferson's gage-lab tag format and nobody else's, so another site's
operators were told to expect letters that are not on their labels.
It is now printedparts_label_prefix, set in Settings, defaulting to EMPTY - a
site that has not set one sees no prefix rather than inheriting another site's
convention. West Jefferson sets it to WJ once. The kiosk hides the prefix
entirely when unset and falls back to no prefix if the setting cannot be read,
because a cosmetic hint must never stop a kiosk working.
Not to be confused with printedparts_code_prefix, which mints item codes like
3DP0042 and was already configurable. That is the code we generate; this is the
tag already printed on the label.
A site added two slides to the lobby display and it never changed between them.
onMounted awaited fetchSlides, which starts the slideshow itself once it has
more than one slide, and then started it AGAIN unconditionally. Two timer chains
ran, and the second assignment to slideTimer lost the handle to the first, so
nothing could ever cancel it. Both fired about ten seconds later, milliseconds
apart, and each advanced one slide.
With exactly two slides that is 0 -> 1 -> 0 every cycle: the display looked
frozen. With three or more it advanced by two and merely skipped one, which is
why this survived so long - and why adding a third slide would have appeared to
"fix" it.
onMounted no longer starts it; fetchSlides owns that. scheduleNextSlide also
cancels any pending timer before setting a new one, so a future double-call
replaces the chain rather than leaking an untracked one.
While here: the feed has always sent a per-slide duration and the display
ignored it, hardcoding ten seconds, so a slide set to hold for a minute changed
after ten. It now uses the slide's own value, and the progress bar animates over
that same duration instead of finishing early and sitting full.
A server whose migrations or seeds never finished does not fail politely. Most
pages answer 500 and settings endpoints answer 404 for keys that were never
created, which reads as a broken application rather than an unfinished install.
One site spent a morning being debugged that way.
`shopdb-admin.ps1 repair` runs what stage 3 of the installer runs: db upgrade,
plugin upgrade-all, and the three seeds. Every step is idempotent, so running it
on a healthy server changes nothing, and each step runs independently so one
failure does not silently skip the rest.
`check` now says so before anyone has to infer it:
THIS SERVER IS NOT FULLY PROVISIONED
- seed data is missing (permissions, settings or reference data)
Most pages will answer 500 until this is fixed. Run:
shopdb-admin.ps1 repair
That needs a real test to sit on, so `flask db-utils seed-state` reports each
seed group and exits non-zero when any is missing. Verified by emptying the
settings table inside a transaction: MISSING, exit 1, rollback clean. Without it
the console check would have looked reassuring while testing nothing - an older
build with no such command reports UNKNOWN rather than healthy, for the same
reason.
A site reported "internal server error" adding a second business unit. It was
reproducible: create one with a blank code, create another with a blank code,
500.
A column that is unique and nullable accepts any number of NULLs - that is what
makes "optional but unique" work - and exactly ONE empty string. The form sent
'', so the first blank code saved and every one after it collided with it. The
field showed no asterisk because it genuinely is optional; the database just
behaved as though it were not.
This is not specific to business units. A dozen columns across core and the
plugins are unique and nullable - asset numbers, hostnames, item codes, subnet
names, gage-lab tags - and each was one blank form away from the same 500.
Fixing them an endpoint at a time would have left the next to be found by a
user, so a before_flush listener normalises blank to NULL on any unique nullable
text column. Listening on Session rather than on individual mappers covers
plugin models imported later, and avoids mapper-event semantics that differ
between SQLAlchemy versions.
A genuine duplicate is now a 409 with a readable message rather than a bare 500
with a traceback in the log: reusing a code that is taken is the caller's
mistake, not a server fault.
Verified against the development database: three business units with blank codes
all save, the blank stores as NULL, and a real duplicate code returns 409.
Two unrelated things found while looking at blank printer types.
Selecting a filter while past page one returned an empty list. The filter asked
the server for page 5 of a result set that now had one page, and the screen said
nothing matched. useListQuery already resets the page - setSearch and setExtra
both do - but the filter dropdowns bypassed it and called the loader directly.
Nine list pages now route through applyFilter, which calls setPage(1) when it
needs to and loads directly when already on page one, so the composable's URL
watcher does not also fire and fetch twice.
scripts/retype_models.py addresses why printer types cannot be derived. The
catalog types every printer model "Printer": true, and useless, since it does not
say whether the product is a laser, a plotter or a label printer. That answer is
a property of the model - every VersaLink C405 is a laser MFP - but nothing
recorded it, so nothing could derive it. Recording it on the MODEL means the
existing backfill fills every printer by exact name match, and a printer added
later inherits the right type the moment its model is chosen.
It exports the models needing a decision to CSV with a type suggested from the
model number, a person corrects the column, and applying it is a dry run unless
given --commit. A suggested type is refused unless it already exists in that
asset class's own vocabulary, which is what keeps the later name match working.
The suggestion order matters and got this wrong first time: a generic plotter
pattern matched "Zebra ZT411" and filed a label printer as a plotter. Brands now
come before generic patterns, and the review step exists precisely because a
confident wrong guess would type every asset using that model.
Verified on the development database: 24 printer models need a decision, 22 got
a sensible suggestion, applying them let all 42 printers match a printertype by
name, and the transaction rolled back cleanly.
The machine's own type is blank for the 134 machines that came from the classic
ASP database on machinetypeid=1, a LocationOnly placeholder the import refuses
to carry across as a real subtype. The catalog model knows what those machines
are, and its type is populated, so the column reads modeltypename under a
heading that says so.
Where both values exist they are identical - all 262 machines in the development
database match exactly - so nothing is lost by showing the one that is reliably
filled in.
This does not fix the underlying gap. A null machinetypeid also excludes a
machine from the map's subtype filter and drops its marker to the default
colour, and no column heading affects that. Only populating machinetypeid does,
which is what the backfill script is for.
Two fields on the same page were both labelled "Type": the asset's own, and the
catalog model's. Only one of them was vague. "Model type" already says exactly
what it is; the bare "Type" did not say whose.
So the unqualified one is the one that changes. No new vocabulary, and "Model
type" reads correctly against it:
Type -> Machine Type (machines)
Type -> PC Type (computers)
Type -> Printer Type (printers)
Type -> Device Type (network devices)
Left alone everywhere the word is not ambiguous - measuring tools, subnets,
VLANs, notifications, supply types and the manifest editor have no model type on
screen to be confused with.
This is a labelling change only. It does not address the blank type column on
machines imported from the classic ASP database, which is a data gap the
backfill script fills; renaming a column heading was never going to put values
in it.
Adding a "Model type" row next to "Type" put two rows in the Hardware section
that read identically. They come from different tables - modeltypes is the
catalog-wide list spanning every kind of asset, machinetypes is machine-only -
but the names line up in practice: all 262 machines in the development database
match exactly, which is the same fact that makes the type backfill safe.
So the row now appears only when the two disagree, which is the case worth
seeing: a model catalogued as one thing fitted to an asset recorded as another.
When they agree it says nothing and is hidden. Applied to machines, PCs,
printers and network devices, each compared against its own type table.
Correcting an earlier judgement. I said the model's type could not be used to
fill an asset's type, because modeltypes is the catalog-wide list covering every
kind of asset - it holds "Access Point", "Camera" and "Desktop PC" - and only
about two thirds of its names exist as machine types.
That is true across the whole catalog and misleading in practice. Restricted to
the models an asset class actually uses, the picture is different: all 262
machines in the development database map exactly, because the non-machine
entries are never used by machines. The blanks on the machines list are rows
whose type the database could already have supplied.
So the backfill now fills the type as well, under a rule that cannot mistype
anything: exact name match or nothing. A model type with no identically named
entry in the asset's own type table is reported with a count and left untouched,
so somebody can decide rather than have a guess written into their data. The
same shape covers computers, printers and network devices, each against its own
type table.
Verified against the development database by nulling one machine's type inside a
transaction: it was detected as fillable, the proposal read "LocationOnly" ->
"LocationOnly", the update restored exactly the original id, and the rollback
left the row unchanged.
Still a dry run unless given --commit, and a table missing the model column is
skipped, so it runs against a server whose network migration is not yet applied.
An asset that carries a model but no vendor was showing a blank the database
could already answer: the model records its vendor, and both sides reference the
same vendors table. Machines, PCs, printers and network devices now fall back to
it.
The fallback is FLAGGED, not merged silently. to_dict sets vendorfrommodel and
the detail pages render "(from model)" beside the value, because the record
itself is still empty: the edit form shows an empty vendor box, and a page
implying the vendor is stored would be lying about where it came from.
The model's type is exposed under its own name, modeltypename, and shown as a
separate "Model type" row. It is deliberately NOT used to fill in the asset's
own type. modeltypes is the catalog-wide list covering every kind of asset - it
holds "Access Point", "Camera" and "Desktop PC" alongside the machine entries -
so it is a different taxonomy from machinetypes. Only about two thirds of the
names overlap, and mapping one onto the other would mistype the remainder, with
the failure mode being a machine labelled "Desktop PC".
scripts/backfill_vendor_from_model.py writes the derived vendor down for real,
since the display fallback leaves reports that read vendorid still seeing
nothing. It is a dry run unless given --commit, fills only rows where the
asset's vendor is NULL and the model names one, and never overwrites a vendor
somebody chose. It skips a table lacking either column, so it runs against a
server whose network migration has not been applied yet.
Verified against the development database by nulling one machine's vendor inside
a transaction: it was detected as fillable, restored to exactly its original
value, and the rollback left the row untouched.
FLASK_ENV is not forced by the script. The app already reads it from .env, and
overriding it demanded a SECRET_KEY the environment had no reason to supply.
Three faults around vendor-model photos, found while looking at why an uploaded
image did not appear.
Saving a model was blocked after uploading a photo. The Image URL field was
type="url", and an upload sets it to an application path such as
/api/models/image/model-120.png. Native url validation demands an absolute URL
with a scheme, so the browser refused to submit the form with "Please enter a
URL" for a value the page had just written itself. The field is now type="text",
which is what it always needed to be: it holds either a full web address or a
path on this server. documentationurl stays type="url".
The upload button did not appear when adding a model, only when editing one.
That was deliberate - the photo is stored as model-<id>.<ext>, so it cannot be
sent before the record has an id - but it reads as a missing feature, and the
hint explaining it was easy to miss. A photo chosen while creating is now held
and uploaded as soon as the model is saved, and it is dropped if the dialog is
cancelled, so it cannot land on the next model created in the same session.
Network devices could never show a photo. NetworkDeviceDetail.vue binds its hero
image to networkdevice.imageurl, but networkdevices carried only vendorid, with
no link to a catalog model, so nothing could populate it - a feature that looked
present and could not work. Machines, PCs and printers have carried
modelnumberid since July. This adds the same column and relationship, the
to_dict branch that exposes modelname and imageurl, the field on the API, and a
Model selector on the form so the link can actually be set.
The migration is guarded the same way employees0002photo is: on a fresh database
the tables come from the SQLAlchemy models, which already declare the column, so
an unconditional add fails with "duplicate column name". The foreign key is
created only on databases that can add one by ALTER; routing it through
batch_alter_table made Alembic's column sort raise "Circular dependency
detected" on the fresh-database test.
Deploying this needs `flask db upgrade` and `flask plugin upgrade-all` on the
server, not just a file copy.
The exe is attached to a release, never committed: most forges reject files over
100 MB inside a repository while allowing release assets far larger, and a
committed binary would sit in every future clone forever.
Also records that tags must be pushed explicitly. A plain push of the branch
does not carry them, so a release had nothing to hang off.
2026-08-05 07:44:57 -04:00
847 changed files with 145040 additions and 12259 deletions
@@ -4,6 +4,16 @@ Follow these when suggesting code. They are enforced by a naming/style hook and
by CI (`.github/workflows/ci.yml`) - suggestions that break them fail the build.
by CI (`.github/workflows/ci.yml`) - suggestions that break them fail the build.
`CONTRIBUTING.md` is the full authority; this is the short version.
`CONTRIBUTING.md` is the full authority; this is the short version.
## General Guidelines
- Finish the in-flight task before starting new workstreams.
- When asked to "take notes" on a future idea, capture it in the notes file only - do not implement it, and do not expand on it at length in replies.
- Use a temp file for multi-line git commit messages instead of heredoc syntax: `git commit -F <file>`.
- Avoid using `powershell.exe -File` for launching scripts, as it fails if the script file is missing or quarantined.
## Installer Logic
- The endpoint agent installer shipped to endpoints is generated by `_install_script()` in `plugins/endpoints/api/siteagent.py`, NOT `agent/installer/install-shopdbagent.ps1`. Any installer logic change must be made in `siteagent.py`, followed by rolling the containers.
- The user's workflow: the agent hands over only the built EXEs (`shopdbagent.exe`, `shopdb-endpoint-tray.exe`); the user uploads them via the "Upload New/Legacy Agents" field in the Build Site Agent UI and builds the package with the newest version selected at the top.
## Naming (LOCKED - the hook rejects violations)
## Naming (LOCKED - the hook rejects violations)
- **DB tables**: lowercase, concatenated, plural. No underscores, no dashes.
- **DB tables**: lowercase, concatenated, plural. No underscores, no dashes.
with quantities are standalone tables, not assets.
with quantities are standalone tables, not assets.
- **Ledger pattern**: a cached `quantityonhand` moves in the SAME commit as the
- **Ledger pattern**: a cached `quantityonhand` moves in the SAME commit as the
signed transaction row it reflects.
signed transaction row it reflects.
- **Configuration for endpoint agent**: The endpoint agent's live collector key and base URL are provisioned in the Windows registry `HKLM\SOFTWARE\GE\ShopDB` (values `CollectorKey`, `BaseUrl`), not committed in `appsettings.json`; `appsettings.json` holds only non-secret runtime settings.
`CONTRIBUTING.md`defines naming rules (DB tables, columns, Python, JS, Vue, API). Pre-commit hook at `scripts/check-naming-and-style.sh` enforces them. Read `CONTRIBUTING.md` before naming any new identifier.
`CONTRIBUTING.md`is authoritative: lowercase concatenated DB columns, snake_case
pure-code identifiers, a closed acronym list, banned shorthand. The pre-commit
## Current state (as of 2026-07-13)
hook `scripts/check-naming-and-style.sh` enforces what it can.
Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely complete; the last big milestone is the legacy-data import + a production pilot.
- **Phase 5**: ADRs moved to `docs/adr/`, Alembic baseline migration, per-site deploy artifacts (`Dockerfile`, `docker-compose.yml`, `docs/DEPLOY.md`)
### Active state
- 1159 tests, naming/style check green, Gitea Actions CI (backend + naming + frontend build + a lean-build job + a migrations-mysql job that runs the real fresh upgrade on utf8mb4 MySQL 8)
- GE-Enforce HTTPS cutover: the displays/kiosks cohort now fetches manifest + inline payloads entirely over HTTPS (share-less); the `gea-shopfloor-display` scope is authored in code (`plugins/geenforce/seed_display_scope.py`) and published via `seed_display_scope(publish=True)`. Other fleet PC types still enforce from the SMB share and only report. See `docs/geenforce-api-cutover.md`.
- Core Alembic chain: baseline `68b3947ae14f` -> head `7d26_settings_description_text` (33 core migrations). Each plugin owns its own chain (ADR-008); deploy runs `flask db upgrade` then `flask plugin upgrade-all`. Reproducible + idempotent from empty (env.py relaxes session sql_mode so the chain runs on strict MySQL 8).
- Lean per-site builds (ADR-013 + ADR-014): `scripts/build-site.sh` (backend) + `SITE_PLUGINS` via `scripts/stage-frontend.mjs` (frontend) ship only chosen plugins; `flask plugin prune-schema` drops non-installed plugins' tables at provisioning. Sidebar nav / settings / Displays all gate on staged routes. Manifest-less `plugins/<name>/frontend/` dirs (e.g. `applications`) are core and always ship.
- Windows sites install from a single air-gapped installer `.exe` built per site from its plugin profile (`deploy/windows/installer/`, built by `build-installer.sh` or `build-installer.ps1`). Operator docs: `docs/INSTALL-WINDOWS.md` + `docs/OPERATE-WINDOWS.md` - these are canonical for a NEW site. `docs/INSTALL-WINDOWS-IIS.md` and `docs/DEPLOY-WINDOWS-IIS.md` are the MANUAL procedure, kept for hand-built servers only. The installer verifies its third-party payload against `bundle-lock.json` and installs wheels with `pip --require-hashes`; every build stages a CycloneDX SBOM (`sbom.cdx.json`) onto the server.
- Legacy import: `docs/IMPORT-API.md` is the schema-agnostic import contract; `docs/IMPORT-ADOPTION.md` + `docs/PILOT-DEPLOY.md` cover adopting a site; `scripts/site_imports/wjf/` is the West Jefferson reference loader (all 16 stages, validated end-to-end including on a Windows + MySQL 8 VM).
- API is migration-complete: an admin PAT + docs/IMPORT-API.md let a script import the whole legacy DB (X-Import-Mode preserves timestamps).
- Pre-1.0 framework; sister sites should pin tight `core_version` ranges until contract reaches 1.0
### Deferred
- Equipment data migration (one-shot script for legacy ASP shopdb -> assets). Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates. Skill `migrating-asset-schema` documents the pattern; the actual one-shot script lives in `scripts/migration/` when run.
- Printers retirement: legacy `PrinterData` model + frontend changes. Coordinated with the equipment data migration.
- (DONE 2026-07-11) `measuringtools` plugin (ADR-005) is built and bundled; docs/PLUGIN-GUIDE.md narrates its construction as the plugin tutorial.
- MySQL 5.7+ database (5.6 works with extra utf8mb4 config; see docs/DEPLOY.md)
- MySQL 5.7+ database (5.6 works with extra utf8mb4 config; see the DEPLOY wiki page)
- JWT authentication
- JWT authentication
- Plugin architecture for extensibility
- Plugin architecture for extensibility
@@ -100,7 +100,7 @@ To maintain consistency with the legacy ShopDB database and codebase, the follow
- Python 3.14
- Python 3.14
- Node.js 18+
- Node.js 18+
- MySQL 5.7+ (5.6 works with extra utf8mb4 config; see docs/DEPLOY.md)
- MySQL 5.7+ (5.6 works with extra utf8mb4 config; see the DEPLOY wiki page)
ShopDB Flask uses MySQL as the canonical database. SQLite is used only for the
ShopDB Flask uses MySQL as the canonical database. SQLite is used only for the
test suite (`TestingConfig` in `shopdb/config.py` points at an in-memory
test suite (`TestingConfig` in `shopdb/config.py` points at an in-memory
@@ -108,8 +108,10 @@ SQLite). Do not run dev or production against SQLite.
### Distribution
### Distribution
The application is distributed through the internal GE Aerospace git server. Clone
Source lives here. The Windows installer ships as a release asset rather than a
it from there; there is no public package or image registry.
file in therepository - it is around 240 MB, well past what a repository takes
and well inside what a release asset does. There is no package or image
registry; a site installs from that `.exe` or builds from source.
### Fast path (Docker)
### Fast path (Docker)
@@ -119,7 +121,7 @@ so a container deploy needs no separate Node build step.
```bash
```bash
cp .env.example .env
cp .env.example .env
# Edit .env: set SECRET_KEY, JWT_SECRET_KEY, DATABASE_URL, CORS_ORIGINS,
# Edit .env: set SECRET_KEY, JWT_SECRET_KEY, DATABASE_URL, CORS_ORIGINS,
# and the MYSQL_* passwords (see docs/CONFIG.md for every variable).
# and the MYSQL_* passwords (see the CONFIG wiki page for every variable).
docker compose up -d --build
docker compose up -d --build
@@ -170,19 +172,43 @@ including VS Code and troubleshooting, is the DEVELOPMENT-SETUP page in the
project wiki.
project wiki.
To import a site's legacy data, use the HTTP import surface: an admin API
To import a site's legacy data, use the HTTP import surface: an admin API
token plus [docs/IMPORT-API.md](docs/IMPORT-API.md) drive the whole migration
token plus the IMPORT-API page in the project wiki drive the whole migration
through documented endpoints (`X-Import-Mode` preserves original timestamps).
through documented endpoints (`X-Import-Mode` preserves original timestamps).
`scripts/site_imports/wjf/` is the West Jefferson reference loader.
`scripts/site_imports/wjf/` is the reference site's loader, kept as a worked
example of the whole sequence.
### Which deployment route
### Which deployment route
| Target | Use |
| Target | Use |
|---|---|
|---|---|
| **Windows Server + IIS** (how sister sites run) | **[docs/INSTALL-WINDOWS.md](docs/INSTALL-WINDOWS.md)** - one installer `.exe`, offline, no manual IIS work. Day 2: [docs/OPERATE-WINDOWS.md](docs/OPERATE-WINDOWS.md) |
| **Windows Server + IIS** (how sister sites run) | **the INSTALL-WINDOWS page in the project wiki** - one installer `.exe`, offline, no manual IIS work. Day 2: the OPERATE-WINDOWS page in the project wiki |
| Linux / Docker, air-gapped | [docs/DEPLOY-AIRGAP.md](docs/DEPLOY-AIRGAP.md) |
| Linux / Docker, air-gapped | the DEPLOY-AIRGAP page in the project wiki |
| Linux / Docker, connected | [docs/DEPLOY.md](docs/DEPLOY.md) |
| Linux / Docker, connected | the DEPLOY page in the project wiki |
For every environment variable and Setting key see [docs/CONFIG.md](docs/CONFIG.md).
For every environment variable and Setting key see the CONFIG page in the project wiki.
### What the Windows installer requires
The installer checks all of this before it changes anything and refuses rather
than half-installing.
| | Minimum | Notes |
|---|---|---|
| Windows Server | **2019** (build 10.0.17763) | Ships **IIS 10.0**, which is therefore the earliest IIS supported. Server 2016 is also IIS 10.0 but falls below the build floor and is refused. |
| Windows client (test boxes) | **10 22H2** (build 10.0.19045) or 11 | **Pro or higher** - Home has no IIS at all. |
| Architecture | 64-bit | The payload is cp314 win_amd64. |
| IIS Web Server role | installed beforehand | `Install-WindowsFeature -Name Web-Server -IncludeManagementTools`. Needs no internet. |
| Free disk | 5 GB | Refused below this. 40 GB is comfortable once uploads and config backups accumulate. |
| Database | bundled MySQL 8.4 LTS, or your own | Bring credentials if you use an existing server. |
The binding constraint is the operating system rather than IIS: the payload is
Python 3.14 plus HttpPlatformHandler and URL Rewrite, and the handler itself runs
on older IIS. An older host is untested rather than known-broken.
If you must deploy onto something older, the INSTALL-WINDOWS-IIS page in the project wiki
is the manual procedure and has no OS gate - it even documents MySQL 5.6, which is
that era of machine. The trade is stated there: it produces a server the
installer will not subsequently upgrade.
## Configuration
## Configuration
@@ -220,12 +246,22 @@ Query parameters for list endpoints:
-`search` - Search term
-`search` - Search term
-`assettype` - Filter by asset type (computer, printer, machine, networkdevice, measuringtool)
-`assettype` - Filter by asset type (computer, printer, machine, networkdevice, measuringtool)
## Where to start
the START-HERE page in the project wiki routes by what you are here to do -
standing up a site, deploying the shop-floor tools, writing a plugin,
integrating with the API, or diagnosing something. On the published mirror the
same page is the wiki's front door.
## Plugin System
## Plugin System
ShopDB supports plugins for extending functionality. See `CONTRIBUTING.md` for plugin development guidelines.
ShopDB supports plugins for extending functionality. See `CONTRIBUTING.md` for plugin development guidelines.
The image bundles thirteen plugins; only the ones a site installs are loaded:
The image bundles these plugins; only the ones a site installs are loaded. The
current count and each plugin's version and migration head are in
the PROJECT-MAP page in the project wiki, which is generated:
- **backups** - Machine configuration backups (NTLARS/DNC, CMM, part marker, UDC) with revision history and diffs
- **computers** - Shopfloor PCs and workstations, collector fleet ingest
- **computers** - Shopfloor PCs and workstations, collector fleet ingest
- **employees** - Employee directory
- **employees** - Employee directory
- **geenforce** - GE-Enforce imaging/software manifests and fleet compliance
- **geenforce** - GE-Enforce imaging/software manifests and fleet compliance
@@ -237,6 +273,7 @@ The image bundles thirteen plugins; only the ones a site installs are loaded:
- **printedparts** - 3D-printed part catalogue, kiosk issue tracking and stock alerts
- **printedparts** - 3D-printed part catalogue, kiosk issue tracking and stock alerts
- **printers** - Extended printer management with Zabbix integration
- **printers** - Extended printer management with Zabbix integration
- **slides** - TV/kiosk slideshows
- **slides** - TV/kiosk slideshows
- **tools** - Tech Tools: barcode and QR generation laid out for real label stock
- **usb** - CMMC USB check-in/out tracking
- **usb** - CMMC USB check-in/out tracking
- **warranty** - Warranty records with Dell lookups
- **warranty** - Warranty records with Dell lookups
# Before the file goes in: this may be the first thing that creates AppRoot,
# and the file below holds the root and app database passwords.
Protect-InstallRoot$AppRoot
New-Item-ItemTypeFile-Force-Path$sqlPath|Out-Null
New-Item-ItemTypeFile-Force-Path$sqlPath|Out-Null
Protect-File$sqlPath
Protect-File$sqlPath
$sql=@"
$sql=@"
@@ -1146,8 +1202,17 @@ function Invoke-Stage2 {
# Same class of failure as MySQL's --defaults-file. Never put a space in a
# Same class of failure as MySQL's --defaults-file. Never put a space in a
# path this installer controls.
# path this installer controls.
$pyTarget='C:\Python314'
$pyTarget='C:\Python314'
# Stage 4 needs it to grant the app pool read access; a venv's python.exe is
# a copy, but python314.dll and the stdlib still come from here.
$script:PyRoot=$pyTarget
if(Test-Path(Join-Path$pyTarget'python.exe')){
if(Test-Path(Join-Path$pyTarget'python.exe')){
Write-Log"Python already present at $pyTarget"'OK'
Write-Log"Python already present at $pyTarget"'OK'
# NOT hardened: something else installed it and may depend on the ACL it
# has. Say so instead - the app would otherwise run on an interpreter
# any logged-on user can replace.
if(Test-UserWritableRoot$pyTarget){
Write-Log"$pyTarget is writable by non-administrators; this installer did not create it, so it is left alone. Harden it or reinstall Python under an ACL only administrators can write."'WARN'
The commands below copy files. This is where each one is obtained, because they
do not all ship from the same place and two of them are not in this repository
at all yet.
| Artifact | Where it is today |
|---|---|
| GE-Enforce client (`Install-GEEnforce.ps1`, `Invoke-ShopdbEnforce.ps1`, `ShopdbEnforceClient.psm1`) | This repository, `plugins/geenforce/client/`. Present on any installed server under the install directory. |
| `Report-AssetToShopDB.ps1` | `plugins/computers/client/`, so it versions with the collector contract it implements. It names no site: the server comes from `HKLM:\SOFTWARE\GE\ShopDB``BaseUrl` (which Install-GEEnforce.ps1 writes) or `-ApiUrl`, and the NIC it reports is the one carrying the default route unless the site names its ranges via `-AllowedRanges` or the `CollectorRanges` registry value. |
| EventSaver (`EventSaver.cs`, `EventSaver.ini`) | `plugins/slides/client/`. The source is a single C# file that builds with the in-box .NET Framework compiler, so a site can rebuild it rather than trust a binary. The compiled `EventSaver.scr` is a release asset, not a file in the repository. Neither file names a site. See [EVENTSAVER.md](EVENTSAVER.md). |
A site that would rather not run a binary it cannot rebuild should take
EventSaver's source and compile it locally - the build needs no SDK and is one
command.
---
## Part 1: the asset reporter
### What it does
Runs as SYSTEM, collects hostname, BIOS serial, PC type, logged-in user, IP
addresses and the machine number where one is configured, and POSTs them to:
```
POST https://<your-shopdb>/api/collector/computers
X-API-Key: <collector token>
Content-Type: application/json
```
The payload contract is [COLLECTOR-INTEGRATION.md](COLLECTOR-INTEGRATION.md) and
[ADR-006](adr/ADR-006-collector-contract.md). A field your site does not collect
is simply absent; the server upserts on `hostname` and leaves the rest alone.
### What you must provide
| Input | Where it comes from |
|---|---|
| Server URL | `-ApiUrl`, or `HKLM:\SOFTWARE\GE\ShopDB` value `BaseUrl`. A script downloaded from Settings has your URL already in it |
| API key | `-ApiKey`, or `HKLM:\SOFTWARE\GE\ShopDB` value `CollectorKey`. Never stamped into the script |
| Routable ranges | Optional. `-AllowedRanges '10.20.0.0/23,10.21.4.0/26'`, the `CollectorRanges` registry value, or the `computers_routableranges` setting. Unset reports the NIC carrying the default route, which is right at most sites |
**Shortcut: download it pre-configured.** Settings > Computers > Asset reporter
generates this script with your server's URL and ranges already in the parameter
defaults, and shows its SHA-256. The key is deliberately not included.
### Minting the key on your ShopDB
Create a **managed personal access token scoped to `collector.ingest` and
nothing else**, and use it as the `X-API-Key` value. That scope authorises the
collector ingest API and no normal route, so a token recovered off a shop-floor
PC cannot read your asset register. A shared `COLLECTOR_API_KEY` environment
variable also works and is simpler for a pilot, but it cannot be rotated per
fleet or revoked individually.
### Deploying it
The three paths below are alternatives - pick one. All of them end with the same
two facts on the PC: the script exists somewhere it can run from, and the key is
in the registry.
#### Option A: Microsoft Intune
**A1. The key, as a platform script.** Devices > Scripts and remediations >
Platform scripts > Add > Windows 10 and later. Run as SYSTEM, do not run in the
64-bit context only.
```powershell
# Set-ShopdbCollectorKey.ps1 (Intune platform script, runs once per device)
Provision the key with a second entry, ordered **before** it, so the key is
present the same cycle the first report fires:
```json
{
"_comment":"Writes the collector key from an ACL'd share file. The token is NOT in this manifest - manifests sync broadly. Rotate by replacing configs/collector-key.txt on the share; every PC picks it up next cycle.",
"Name":"Provision the ShopDB collector key",
"Type":"PS1",
"Script":"scripts/Set-CollectorKey.ps1",
"DetectionMethod":"Always"
}
```
Keep the token in `configs/collector-key.txt` on your share, locked to SYSTEM
and administrators - not in the manifest JSON.
#### Option C: manual
For a pilot, a single bay, or an estate with no management plane:
@@ -22,7 +22,7 @@ The repo ships hosted, generated API docs. Start there:
The MCP server exposes a curated set of GET endpoints as tools for an agent to
The MCP server exposes a curated set of GET endpoints as tools for an agent to
query the asset database over HTTPS with a scoped read token; it never runs on
query the asset database over HTTPS with a scoped read token; it never runs on
the prod box. Set it up on a work PC with
the prod box. Set it up on a work PC with
`pxe-images/github/setup-mcp.cmd`.
`<imaging-share>/github/setup-mcp.cmd`.
The docs blueprint is `shopdb/core/api/docs.py` (a core blueprint, always
The docs blueprint is `shopdb/core/api/docs.py` (a core blueprint, always
mounted regardless of which plugins are staged into a site build).
mounted regardless of which plugins are staged into a site build).
@@ -75,8 +75,8 @@ keying) are covered there as well.
Everything else is the core UI API: the endpoints the Vue frontend calls. As a
Everything else is the core UI API: the endpoints the Vue frontend calls. As a
rule these are JWT-authenticated (a login token or a managed Personal Access
rule these are JWT-authenticated (a login token or a managed Personal Access
Token) and versioned by the plugin contract (`__contract_version__`, currently
Token) and versioned by the plugin contract (`__contract_version__`; the
0.15.0). Behavior and stability guarantees are in **CONTRACT-STABILITY.md**;
current value is in [PROJECT-MAP.md](PROJECT-MAP.md), which is generated). Behavior and stability guarantees are in **CONTRACT-STABILITY.md**;
sister sites should pin tight `core_version` ranges until the contract reaches
sister sites should pin tight `core_version` ranges until the contract reaches
1.0.
1.0.
@@ -96,6 +96,25 @@ Two auth patterns dominate the reads:
Every mutating endpoint (POST / PUT / PATCH / DELETE) requires a JWT and is
Every mutating endpoint (POST / PUT / PATCH / DELETE) requires a JWT and is
gated by `require_role` or `require_permission`; none are public.
gated by `require_role` or `require_permission`; none are public.
The map write surface splits along that line. Repositioning markers
(`/api/mappositions/*`) needs `assets.edit` - the same permission the old
per-asset `PUT /api/assets/<id>` always needed, so bulk editing grants nobody
anything new. Creating buildings and levels or uploading a blueprint
(`POST|PUT|PATCH|DELETE /api/maplevels/*`) needs `role:admin`, because a level's
dimensions are the coordinate space every marker on it is expressed in.
Two GETs deliberately break the optional-auth convention and require a
permission, because what they return is not asset metadata but a file that
carries value or liability of its own:
| Endpoint | Requires | Why |
|---|---|---|
| `GET /api/applications/package/<filename>` | `applications.download` | Licensed vendor installers. A separate permission from `applications.view` on purpose: seeing that an application exists is not the same as being handed the vendor's installer, and an anonymous URL is a redistribution channel. The script-body download is gated the same way. |
| `GET /api/warranty/proof/<filename>` | `warranty.view` | Invoices and certificates carry pricing and a service tag. |
Their sibling upload/delete routes are `applications.edit` / `warranty.edit`
like any other mutation.
### Fully public endpoints (auth = none)
### Fully public endpoints (auth = none)
| Endpoint | Purpose |
| Endpoint | Purpose |
@@ -103,22 +122,36 @@ gated by `require_role` or `require_permission`; none are public.
| `POST /api/auth/login` | Obtain a JWT. |
| `POST /api/auth/login` | Obtain a JWT. |
| `GET /api/setup/needs-admin` | First-run check: does the instance have zero users. |
| `GET /api/setup/needs-admin` | First-run check: does the instance have zero users. |
| `POST /api/setup/create-admin` | First-run only; creates the first admin, then 403s forever. |
| `POST /api/setup/create-admin` | First-run only; creates the first admin, then 403s forever. |
| `GET /api/settings/map-blueprint/<filename>` | Serve the floor-map blueprint image. |
| `GET /api/settings/map-blueprint/<filename>` | Serve a floor-map blueprint image. Pre-0.11.0 single-map path, retained; new uploads are served per level below. |
| `GET /api/maplevels` | Buildings and their levels, each with blueprint paths, native pixel size and marker count. Public because the printer-installer map draws a floor plan before anyone logs in; it exposes level names and image paths, nothing about assets. |
| `GET /api/maplevels/<levelid>` | One level's name, building, blueprints and size. |
| `GET /api/maplevels/<levelid>/blueprint/<filename>` | Serve a level's blueprint image, with sandbox headers so an SVG floor plan cannot execute as script. |
| `GET /api/settings` and `GET /api/settings/<key>` | Read-only, and only the public allowlist: the `branding` and `map` categories, a few named site keys, plus any key a plugin declares `public` in `get_settings_defaults` (e.g. `printedparts_label_prefix`, which the logged-out parts kiosk renders). Every other key answers 404 to an anonymous caller. |
| `GET /api/models/image/<filename>` | Serve a model image. |
| `GET /api/models/image/<filename>` | Serve a model image. |
| `GET /api/applications/image/<filename>` | Serve an application image (tiles render before login). |
| `GET /api/dashboard/navigation` | Public navigation tree. |
| `GET /api/dashboard/navigation` | Public navigation tree. |
| `GET /api/dashboard/health` | Liveness / health probe. |
| `GET /api/dashboard/health` | Liveness / health probe. |
| `GET /api/plugins/enabled` | List enabled plugins (no claims used). |
| `GET /api/plugins/enabled` | List enabled plugins (no claims used). |
| `GET /api/dashboarddefaults/display-role` | Display role resolution (see section 1). |
| `GET /api/dashboarddefaults/display-role` | Display role resolution (see section 1). |
| ~~`GET /api/employees/search`, `/lookup/<sso>`, `/lookup`~~ | NOT public. These need `employees.view` and answer 401 anonymously. A kiosk resolves a badge through its own gated endpoint, not through these. |
| `GET /api/employees/photo/<filename>` | Serve an employee photo. |
| `GET /api/employees/photo/<filename>` | Serve an employee photo. |
| `GET /api/notifications` (and `/types`, `/<id>`, `/active`, `/calendar`, `/dashboard/summary`, `/employee/<sso>`, `/shopfloor`) | Read shop-floor notifications for the public display and kiosks. |
| `GET /api/notifications/shopfloor`, `GET /api/notifications/employee/<sso>` | The two the public display and kiosks actually read. The rest of `/api/notifications` (the list, `/types`, `/<id>`, `/active`, `/calendar`, `/dashboard/summary`) needs `notifications.view` and answers 401 anonymously. |
| `GET /api/slides/feed` | Slide feed for the lobby display and screensaver. |
| `GET /api/slides/feed` | Slide feed for the lobby display and screensaver. |
| `GET /api/slides/img/<surface>/<filename>` | Serve a slide image. |
| `GET /api/slides/img/<surface>/<filename>` | Serve a slide image. |
| `GET /api/printedparts/image/<filename>` | Serve a printed-part image. |
| `GET /api/printedparts/image/<filename>` | Serve a printed-part image. |
| `GET /api/printedparts/kiosk/item/<itemcode>` | Kiosk part lookup (deliberately open; a kiosk carries no JWT). |
| `GET /api/printedparts/kiosk/item/<itemcode>` | Kiosk part lookup. Carries no JWT, but is NOT open: the routes 404 unless `printedparts_kiosk_enabled` is on AND the request carries the site's `X-Kiosk-Token`. |
| `POST /api/printedparts/kiosk/take` | Kiosk part checkout (deliberately open, per decision record). |
| `POST /api/printedparts/kiosk/take` | Kiosk part checkout, behind the same flag and device token. |
| `GET /api/usb/kiosk/device/<device_id>` | USB kiosk device lookup, behind `usb_kiosk_enabled` plus `X-Kiosk-Token`. |
| `POST /api/usb/kiosk/checkout`, `/checkin` | USB kiosk custody moves, same gate. |
**"No authentication" is not "open."** Both kiosks are off by default and refuse
every request without the site's device token; with the flag off they answer 404
rather than 403, so a kiosk a site has not turned on does not announce itself.
The token is a credential for a screen, not authorization for a person: it says
which cabinet is asking, and the badge on the request says who is standing at
Subclass `BackupKind` and override what applies. `NtlarsKind` is the fullest
example (parseable, renderable, has an info panel); `PartMarkerKind` is the
lean one.
| Member | What it is |
|--------|------------|
| `key` | Wire value. This is what lands in `backuprevisions.backupkind` and what the fleet table shows on the badge. Keep it short and lowercase. |
| `displayname` | Human label for the UI. |
| `storagebackend` | `'shopdb'` (bytes live in the DB, deduped on a semantic hash) or `'share'` (file lives on the SMB share, deduped on a raw byte hash). |
| `assettypes` | Which core asset types this kind can attach to; `['*']` for any. |
| `emptytext` | Text when an asset has no revisions of this kind. `None` HIDES the panel, which is the right default - a kind applies to an asset TYPE, but whether a given machine ever has that backup is per-machine. A part-marker panel on all 144 machines is noise. |
| `resolveassetid(payload)` | Map a collector payload to the asset the backup belongs to. |
| `infopanel()` / `buildinfo(...)` | Optional at-a-glance card (ADR-010). Declared by the KIND, not hardcoded in the plugin, so a successor technology ships its own card by adding a class. |
| `sharedir(...)` | Conventional UNC directory for a `share` kind. Advisory - the authoritative path is whatever the collector reported, because the PC is what actually wrote the file. |
### What you do NOT have to touch
The GE-Enforce **Enforcement Reports** Backup column needs no change. It reads
`backuprevisions` generically: newest revision per host, whatever the kind, and
shows `backupkind` on the badge. A new kind inherits the badge, the colour and
the tooltip with no work.
Same for the staleness rule below - it is time-based and kind-agnostic.
---
## How the Backup badge decides good vs stale
### The trap this is built around
**A revision is only written when the config CHANGES.** Dedup compares against
the latest revision for the chain `(asset, kind, sourcehostname)`, so a machine
whose config has been stable for six months has a six-month-old newest revision
and is perfectly healthy. The question worth answering is not "when was the last
backup taken" - it is "is this still being checked".
So the timestamp the fleet table uses is `backuprevisions.lastseenat`: the last
time the collector CONFIRMED this config, whether or not anything changed. It
moves on every successful collection; the revision does not.
That was shown as a raw date at first, and it read as neglect. At the default
`backups_intervalhours` of **24**, the collector only attempts once a day, so a
day-old confirmation IS the healthy steady state. The date made a working system
look like a stalled one, and made the reader do arithmetic against a setting
they would have to go and look up.
### What it does now
`GET /api/geenforce/reports` returns, per host:
| Field | Meaning |
|-------|---------|
| `backupkind` | The kind the verdict is about - the STALEST measurable one, not the freshest. `null` = no backup at all. |
| `backuplastseen` | When it was last CONFIRMED still current (ISO). Tooltip only. |
| `backupcollectedat` | When the config was last CAPTURED (ISO). Only moves on a real change. Tooltip only. |
`check -Json` reports version, publishing method, IIS and pool state, HTTP
reachability, database reachability, Python version and installed plugins. If it
passes, the restore worked; if the version it reports is not the version you
expect, see "Rolling back a release" in [UPDATES-WINDOWS.md](UPDATES-WINDOWS.md),
because a restored database and newer code is the one combination the installer
cannot fix for you.
### What is lost if `.env` is lost
`.env` is not in the database dump, and rebuilding it is not simply retyping it:
| Value | If it is lost |
|---|---|
| `SECRET_KEY`, `JWT_SECRET_KEY` | Every issued token and session becomes invalid. Users log in again; managed API tokens must be reissued, which means every collector and GE-Enforce client needs its key replaced. Recoverable, but it is a fleet-wide job. |
| `DATABASE_URL` password | Recoverable: reset the MySQL user's password and write the new one in. |
| `MYSQL_ROOT_PASSWORD` | Recoverable through MySQL's own reset procedure, which requires stopping the server. |
| `ZABBIX_TOKEN` and similar integration tokens | Reissue at the far end. Nothing else breaks. |
So back it up with the database, to somewhere as protected as the dump - it is
ACL'd to Administrators and SYSTEM on the server for the same reason. A dump
without its `.env` restores the data and locks everyone out of it.
@@ -80,7 +82,7 @@ owner is an admin - it cannot act with admin authority anywhere.
1. Settings > API Tokens > New Token.
1. Settings > API Tokens > New Token.
2. Check **Restrict permissions**, then in the permissions grid tick only
2. Check **Restrict permissions**, then in the permissions grid tick only
**Submit collector payloads (fleet reporting)** (the `collector.ingest`
**Submit collector payloads (fleet reporting)** (the `collector.ingest`
permission under the Collector category). Name it (e.g. `wj-fleet-collector`),
permission under the Collector category). Name it (e.g. `fleet-collector`),
optionally set an expiry, Create.
optionally set an expiry, Create.
3. Copy the `shopdb_pat_...` secret (shown once) and deploy it to the fleet the
3. Copy the `shopdb_pat_...` secret (shown once) and deploy it to the fleet the
same way as the env key: the `collectorApiKey` field in per-site
same way as the env key: the `collectorApiKey` field in per-site
@@ -138,7 +140,7 @@ Response body (HTTP 200), wrapped in the standard envelope
"status":"ok",
"status":"ok",
"action":"created",
"action":"created",
"assetid":12345,
"assetid":12345,
"identityvalue":"WJRP2335",
"identityvalue":"SHOPPC2335",
"warnings":["unknown operating system: Microsoft Windows 11 Enterprise 23H2 (build 22631)"]
"warnings":["unknown operating system: Microsoft Windows 11 Enterprise 23H2 (build 22631)"]
},
},
"message":"computers collector created",
"message":"computers collector created",
@@ -198,7 +200,9 @@ a column.
| Payload field | Type | Server behaviour (`apply_collector_payload`) |
| Payload field | Type | Server behaviour (`apply_collector_payload`) |
|---|---|---|
|---|---|---|
| `hostname` (required) | string | Identity. Matches `Computer.hostname` (case-insensitive), then falls back to `Asset.assetnumber`. New asset created if no match. |
| `hostname` (required) | string | Identity. Matches `Computer.hostname` (case-insensitive), then falls back to `Asset.assetnumber`. New asset created if no match. |
| `machinenumber` | string | Business tag -> `Asset.assetnumber`. The placeholder `9999` and empty string are skipped; when skipped a new PC falls back to `assetnumber = hostname`. On an existing PC a real value updates `assetnumber`. |
| `machinenumber` | string | Identifies the MACHINE this PC drives, never the PC. A new PC always takes `assetnumber = hostname`, and an existing PC's `assetnumber` is left alone. The number resolves a machine asset and builds a PC -> machine `controls` relationship; an unknown number warns rather than creating a machine. The placeholder `9999` and empty string link nothing. See "Machine links" below. |
| `deviceid` | string | Identifies the DEVICE hanging off this PC, read from `C:\Enrollment\asset-id.txt`. ONE file for every bay with no NTLARS/eDNC `MachineNo` registry to identify it. It names the asset and nothing else - it does NOT say what kind of device it is, because the pc-type already does - so the same file serves a part marker, a Keyence, a Genspect or anything declared later. SEPARATE from `machinenumber` on purpose: "which bay is this" and "which device is this" are different facts, and `machinenumber` is what GE-Enforce `TargetMachineNumbers` gates on, so naming a device there would silently stop every bay-gated manifest entry from matching. An unknown value warns rather than minting, and a value resolving to the wrong KIND of asset is refused with the asset named. See "PC -> device links" below. |
| `measuringtoolid` | string | The 0.12.0 name for the same thing, before the file was generalised, read from `C:\Enrollment\measuringtool-id.txt`. Accepted as an alias so a bay already staged with that file keeps reporting; `deviceid` wins when both arrive. Prefer `asset-id.txt` for anything new. |
| `pctype` | string | `gea-shopfloor-*` imaging type -> `Computer.computertypeid` via the configurable `pctypemap` settings. Unmapped value -> warning, not error. |
| `pctype` | string | `gea-shopfloor-*` imaging type -> `Computer.computertypeid` via the configurable `pctypemap` settings. Unmapped value -> warning, not error. |
| `pcsubtype` | string | Accepted but not stored yet -> warning. |
| `pcsubtype` | string | Accepted but not stored yet -> warning. |
| `modelnumber` | string | `Computer.modelnumberid`, scoped to the vendor when known. Model row auto-created if missing. |
| `modelnumber` | string | `Computer.modelnumberid`, scoped to the vendor when known. Model row auto-created if missing. |
| `osname` | string | `Computer.osid`. Controlled vocab: looked up in `operatingsystems`, NOT auto-created. Unknown value -> warning (row still written, `osid` left unset). |
| `osname` | string | `Computer.osid`. Controlled vocab: looked up in `operatingsystems`, NOT auto-created. Unknown value -> warning (row still written, `osid` left unset). |
| `installedsoftware` | array of `{name, version}` | `ComputerInstalledApp` rows for applications shopdb already tracks. Unknown app name -> warning, skipped. |
| `installedsoftware` | array of `{name, version}` | `ComputerInstalledApp` rows for applications shopdb already tracks. Unknown app name -> warning, skipped. |
| `defaultprinter` | string | The default printer's identifier (windows name / share / hostname / port IP). Resolved to a printer asset and linked PC -> printer as a `defaultprinter` relationship. Unresolved -> warning. |
| `printers` | array of strings | All installed network printer identifiers. Each resolves to a printer asset and is linked PC -> printer as a `connectedto` relationship (the default is skipped here since it already links as `defaultprinter`). Unresolved entries -> warning. |
Schema source of truth: `get_collector_schema` in `plugins/computers/plugin.py`.
Schema source of truth: `get_collector_schema` in `plugins/computers/plugin.py`.
If you change the payload, change it there and re-check this table.
If you change the payload, change it there and re-check this table.
### PC -> printer relationship sync
### PC -> machine links, and part markers
When a payload carries `defaultprinter` and/or `printers`, the collector syncs
The reported `machinenumber` says which machine the PC drives. It is not the
`AssetRelationship` rows so a PC page shows its printers and a printer page shows
PC's own identifier, and writing it as one is a defect this contract used to
the PCs that use it (both render in the shared Relationships card).
describe: `assets.assetnumber` is uniquely indexed and the machine already holds
that value, so the insert failed and the bay got a 500 on every report, forever.
A PC takes its hostname as its asset number and keeps it.
-Resolution: each identifier is matched, first hit wins, against the printer's
-Link: PC -> machine as `controls`, tagged
`windowsname`, `hostname`, `sharename`, its asset number/name, then any active
`assetrelationships.label = 'collector:machine'`. Only tagged rows are ever
printer communications IP. Case-insensitive except the IP (exact). An
archived by a push, so a link made by hand is never touched.
identifier that resolves to nothing adds a warning and is skipped; it never
- Unknown number: warns and links nothing. A machine is never invented, because
fails the whole push.
a mistyped number would create equipment nobody can account for.
-Link types: the default printer links with `defaultprinter` (directional, PC
-A second PC reporting a machine another PC holds is a CLAIM, not a handover.
is the source); every other reported printer links with `connectedto`
A PC imaged for a machine carries that number before it reaches the floor, so
(symmetric). A printer that is both default and in `printers` links only as
the PC currently running the machine keeps the link while it is still alive -
the default.
reporting within 24 hours (`MACHINE_CLAIM_QUIET_HOURS`) and still `In Use`.
- Idempotent: re-reporting the same set creates no duplicate rows (an existing
The challenger is recorded as a dormant link, which also marks the claim as
matching row is reactivated if it was archived, otherwise left as is).
already announced so it does not alert every cycle.
-Stale-link archive: on every push, collector-created links to printers no
-Handover completes on its own once the incumbent has been quiet past that
longer reported are set inactive. Collector-created rows are tagged in
window, or immediately when someone moves it off `In Use` - which is the
`assetrelationships.label = 'collector:printers'`; only tagged rows are ever
one-step way to force a swap at the moment it happens. The old link is
archived, so links you create by hand in the UI are never touched. A payload
archived, never deleted, so "which PC ran this machine in June" stays
that omits BOTH printer keys leaves all existing printer links untouched
answerable. The retired PC's STATUS is deliberately not changed: the collector
(report an empty `printers: []` to clear the auto links instead).
cannot tell shelved from broken from re-imaged.
-Response: the collector response carries `printerlinkcount` and a
-Alerts for handover and contested claims are gated on the
`printerlinks` list of `{assetid, relationshiptype}` for the links kept.
`computers_machinelink_alerts` setting, which ships OFF. A site may run
several PCs on one machine number legitimately, and there the alerts fire on
correct data. Warnings ride in the collector response either way.
The part-marker case below is one instance of a general pattern - several
devices under one parent. `docs/ASSET-COMPOSITION.md` covers when to use it,
what propagation buys you, and how to declare a new device type without a code
change.
A PC reporting `pctype = gea-shopfloor-partmarker` is handled differently,
because several markers can serve one machine number and an operation holds any
number of them. Such a PC gets its own Part Marker machine asset
(`<HOSTNAME>-PARTMARKER`), `controls` it, and the marker is filed `partof` the
operation the PC reported. Both links are tagged `collector:partmarker`. The PC
does NOT claim the operation directly: `controls` propagates through `partof`,
so control still follows from controlling the marker, without two markers
contesting a link only one can hold. Backups from a marker PC file against the
marker rather than the operation.
### PC -> device links (`asset-id.txt`)
Some pc-types mean the PC drives an attached DEVICE: a CMM, Keyence, Genspect or
wax-trace instrument, or a part marker. `SUBORDINATE_DEVICE_MAP` in
`plugins/computers/pctypemap.py` declares which, and a site can add its own
through a `subordinatedevice_<pctype>` setting without a code change.
`C:\Enrollment\asset-id.txt` names that device. One file for every bay that has
no NTLARS/eDNC `MachineNo` registry to identify it, holding one line: the
device's `assetnumber`. It does NOT record what kind of device it is - the
pc-type already does - so a bay declared later needs no new file and no client
change.
Resolution runs most-stable-identity first, and ADOPTS before it mints:
1.**`deviceid`** - the asset named in `asset-id.txt`.
2. A prior collector link from this PC, reactivated.
3. An existing device this PC already controls that the collector did NOT
create - a legacy `MT-####` row, or one somebody made by hand. Adopting
stamps the label so it is ours from then on.
4. For a measuring tool only: the reported machine number, when it resolves to
one. This is the CMM case, where `cmmid.txt` already reports `CMM4`.
5. Mint, only when none of the above matched.
**Why the file exists, and why it is step 1.** Every other identity here is
derived from the PC. Step 2 looks for a link from THIS PC asset; step 5 mints
`<PC number>-<SUFFIX>`, and a PC's asset number is its hostname. All of them
survive a re-image and NONE of them survives a PC SWAP: a new hostname is a new
PC asset with no prior link and a predicted number that has never existed, so
the same physical device gets a second record while the first keeps its config
and backup history under a dead PC's name. That is how 43 legacy `MT-####`
tools ended up shadowed by minted twins, three records deep in places. Treat
anything minted as a placeholder to be reconciled.
Two guards on the file's contents:
- An **unresolvable** value warns and links nothing, rather than inventing a
phantom device nobody can account for.
- A value resolving to the **wrong kind** of asset is refused, naming the asset
it hit. A machine number pasted into `asset-id.txt` would otherwise file that
MACHINE under a device label the collector also owns - a link that reads as an
instrument, or as a marker, everywhere downstream.
A part marker additionally stays `partof` the operation from `machinenumber`,
because several markers serve one operation. Naming the marker in `asset-id.txt`
changes which marker asset the PC controls; it does not change which operation
that marker files under.
### PC -> printer relationship sync (REMOVED)
**A collector report has never been allowed to set a printer assignment, and as
of 2026-08-24 it no longer can.** `defaultprinter` and `printers` used to be
accepted here and written as `defaultprinter`/`connectedto` relationship rows.
That contradicted the rule stated under "Observed is not assigned" below, and
this document carried both claims at once while the code followed the wrong one.
If you are still sending either key: **your push still succeeds** - they are
ignored, and the response carries a warning naming them. Nothing else about your
payload changed. To report what a PC actually has, use
`POST /api/collector/printers`; to turn that into an assignment, a person calls
`POST /api/printers/assignments/seed-from-observed/<assetid>` after reviewing the
comparison.
### pc-type mapping (configurable per site)
### pc-type mapping (configurable per site)
@@ -261,6 +345,7 @@ The fleet's classic-ASP reporter posts form fields to
|---|---|
|---|---|
| `hostname` | `hostname` |
| `hostname` | `hostname` |
| `machineNo` | `machinenumber` |
| `machineNo` | `machinenumber` |
| `deviceId` | `deviceid` |
| `pcType` | `pctype` |
| `pcType` | `pctype` |
| `serialNumber` | `serialnumber` |
| `serialNumber` | `serialnumber` |
| `loggedInUser` | `loggedinuser` |
| `loggedInUser` | `loggedinuser` |
@@ -276,6 +361,137 @@ classic reporter posts a full `networkInterfaces` array; the collector accepts
only one `ipaddress`, so pick the corp/routable NIC (see the corp-range gate in
only one `ipaddress`, so pick the corp/routable NIC (see the corp-range gate in
the PowerShell below).
the PowerShell below).
### Printers plugin field mapping (observed print queues)
`POST /api/collector/printers`. The other half of the printer story. ShopDB has
always known what a bay SHOULD have - `GET /api/printers/for-host/<hostname>`,
applied by `Set-ShopdbPrinters.ps1` - and has never known what it actually has.
This payload is that missing half. Same dispatcher, same auth, same audit row as
the computers collector; only the payload and the plugin differ.
- Schema + apply: `get_collector_schema` / `apply_collector_payload` in
- Storage: `printerobservedqueues`, one row per observed queue, owned by the
printers plugin. It is a separate table from the assignment on purpose - see
"Observed is not assigned" below.
The identity field is `hostname`. Unlike the computers payload this one is NOT
patch-style: `queues` is required, and the reported set replaces everything
previously recorded for that host.
| Payload field | Type | Server behaviour (`apply_collector_payload`) |
|---|---|---|
| `hostname` (required) | string | Identity of the report. `COMPUTERNAME` or its FQDN; matched case-insensitively. Rows are keyed by the NAME, so a bay that reports before its PC record exists still records everything - the report resolves to an asset the moment that record appears. An unknown hostname is a warning, never an error. |
| `queues` (required) | array of objects | Every real print queue on the host. This REPLACES the host's previous set. An empty array is a valid report meaning "this bay has no queues" and clears the host's rows. A missing key is rejected (400), because one client bug that dropped the field would otherwise erase the fleet's observed state host by host. |
| `queues[].queuename` (required) | string | Windows printer name. A queue with no name is skipped with a warning; a repeated name within one report is dropped with a warning (Windows cannot hold two queues of one name). |
| `queues[].drivername` | string | Driver name verbatim, as the INF spells it. Compared against the assignment's driver to detect drift. |
| `queues[].portname` | string | Windows port name. Also tried as a match key, since a port created outside the client is usually named after the host address. |
| `queues[].portaddress` | string | `PrinterHostAddress` of a TCP/IP port - an IP or FQDN. The PRIMARY key for matching a queue to a printer asset. Omit it for a non-TCP port (USB, WSD, redirected); such a queue is still worth reporting and matches on name alone. |
| `queues[].isdefault` | boolean | True on the one queue that is the user's default. If several arrive true, the first is kept and the rest are cleared with a warning - a bay has exactly one default, and keeping both would leave a seed picking one at random. |
| `queues[].isshared` | boolean | True when the queue is shared off this PC. Recorded, not acted on. |
| `observedat` | ISO-8601 datetime | Accepted and ignored, as is any other client timestamp. The server stamps `observedat` at ingest, one value for the whole report, so a bay with a wrong clock cannot report itself fresh or stale. |
Response `data` is the standard collector shape: `action` is always `updated`
(this endpoint replaces rows and creates no asset - calling an identical
re-report `noop` would hide that the bay is still checking in), `assetid` is the
resolved PC or `null`, `extra.queuecount` is how many rows were stored, and
`warnings` carries the soft problems above.
Schema source of truth: `get_collector_schema` in `plugins/printers/plugin.py`.
If you change the payload, change it there and re-check this table.
```
POST /api/collector/printers
X-API-Key: SECRET
Content-Type: application/json
{
"hostname": "WORKSTATION01",
"queues": [
{
"queuename": "Bay Label Printer",
"drivername": "Generic / Text Only",
"portname": "IP_192.0.2.40",
"portaddress": "192.0.2.40",
"isdefault": true
},
{
"queuename": "Office Laser",
"drivername": "HP Universal Printing PCL 6",
"portname": "IP_192.0.2.41",
"portaddress": "192.0.2.41",
"isdefault": false
}
]
}
```
#### The latest report replaces the previous one
This is CURRENT STATE, not history. Each report deletes every row previously
recorded for that hostname and writes the reported set in its place, inside the
dispatcher's transaction. Nothing accumulates, "what does this bay have" is
never a question about time, and a queue removed from a bay disappears from
ShopDB on the next cycle without anyone tidying up.
That has one sharp edge, and it belongs to the client: an empty `queues` array
is a legitimate report that wipes the host's rows. A client whose enumeration
FAILED must therefore send nothing at all rather than an empty list. Reporting
nothing loses one cycle; reporting `[]` after a WMI hiccup deletes real state
and reads as a bay that lost its printers. `Report-PrintersToShopDB.ps1` tracks
this with an `$enumerated` flag and exits without posting when both the cmdlet
and the WMI fallback failed.
#### Observed is not assigned
Stated plainly, because it is the whole point of keeping two tables:
**A collector report NEVER becomes an assignment.** Nothing on this path writes
a `usesprinter` or `defaultprinter` row. The moment a drifted bay's observed
state is treated as correct, enforcement stops meaning anything - a bay that
installed the wrong printer would make itself right simply by reporting it.
Observed state becomes assigned state only when a person asks for it, through
`printers.edit`), after reviewing the comparison. That route refuses rather than
guesses: it will not seed from an ambiguous host, will not seed queues that
match no printer unless told to, and will not write an empty set.
#### Matching, and what "unknown" means
Nothing is matched at ingest - the raw observed strings are stored as reported,
and resolution to a printer asset happens at READ time, so a printer added to
ShopDB tomorrow matches yesterday's report without the bay reporting again.
At read time each queue is matched by PORT ADDRESS first (an IP or FQDN names
one device unambiguously), then by queue name. There is no fuzzier fallback: a
queue that matches nothing is reported as `unknown` rather than guessed, because
a wrong match seeds a wrong assignment, which is worse than no assignment.
#### Reading it back
| Endpoint | Purpose |
|---|---|
| `GET /api/printers/observed/<hostname>` | What the host last reported, every queue classified against the assignment it resolves to (`matching`, `drifted`, `extra`, `missing`, `unknown`), with `observedat`, a summary, and a `seedcandidate` preview. Read-only. Needs `printers.view`. |
| `POST /api/printers/assignments/seed-from-observed/<assetid>` | The one path from observed to assigned, human-triggered. Needs `printers.edit`. Normally posted against the MACHINE, so the assignment survives a reimage. |
#### Key delivery for the printers reporter
The reporter reads its server and its key from the registry the enforcement
client already provisions (`HKLM:\SOFTWARE\GE\ShopDB`, values `BaseUrl` and
`CollectorKey`), or from `-ApiUrl` / `-ApiKey` in the manifest entry's `Args`.
Nothing is baked into the script or the manifest JSON on the share - the same
rule as the computers reporter, for the same reason.
Server side, the key resolves per-plugin first: set `COLLECTOR_API_KEY_PRINTERS`
if you scope keys per collector, or rely on the shared `COLLECTOR_API_KEY`. A
site that has scoped the computers key per-plugin and set no printers key gets
401 on this endpoint until one of the two is in place. A `collector.ingest`
managed token works here exactly as it does for computers.
@@ -658,8 +877,13 @@ fleet's key is scoped to the computers collector only.
| `No collector registered for plugin computers` | 404 | The computers plugin is disabled or not loaded on that instance. Enable it. |
| `No collector registered for plugin computers` | 404 | The computers plugin is disabled or not loaded on that instance. Enable it. |
| `hostname is required` / `No data provided` | 400 | Empty body, non-JSON body, or missing identity field. Check `-ContentType 'application/json'` and that `hostname` is set. |
| `hostname is required` / `No data provided` | 400 | Empty body, non-JSON body, or missing identity field. Check `-ContentType 'application/json'` and that `hostname` is set. |
| `Internal error processing collector payload` | 500 | Generic by design - the server does not leak the cause to the caller. The real error (DB, unexpected exception) is in the server log (`Collector upsert failed for computers`). Check there. |
| `Internal error processing collector payload` | 500 | Generic by design - the server does not leak the cause to the caller. The real error (DB, unexpected exception) is in the server log (`Collector upsert failed for computers`). Check there. |
| `No collector registered for plugin printers` | 404 | The printers plugin is disabled or not loaded on that instance. Enable it. A lean site without it simply does not collect observed queues. |
| `queues is required; send an empty array for a host with no queues` | 400 | The printers payload omitted `queues` entirely. Absent and empty are not the same thing here - see "The latest report replaces the previous one". |
| A bay's observed queues all vanished | 200 | It reported `queues: []`, which legitimately clears the host. Check the client log for `posting NOTHING` (a failed enumeration correctly sends nothing) versus `reporting an empty set`. If the bay really has queues, the enumeration on that host is the thing to fix. |
| Observed queues show as `unknown` | 200 | The queue matched no printer asset by port address or by name. Add the printer to ShopDB (with its IP) and re-read - matching happens at read time, so the bay does not need to report again. Never seeded automatically, by design. |
| `warnings` present but `action` is created/updated | 200 | Soft issues only; the row WAS written. Common: unmapped `pctype` (fix the pctypemap setting), unknown `osname` (add it to the `operatingsystems` vocab), unknown app name, `pcsubtype` not stored. No action needed unless the warning matters to you. |
| `warnings` present but `action` is created/updated | 200 | Soft issues only; the row WAS written. Common: unmapped `pctype` (fix the pctypemap setting), unknown `osname` (add it to the `operatingsystems` vocab), unknown app name, `pcsubtype` not stored. No action needed unless the warning matters to you. |
Client-side log for the fleet reporter: `C:\Logs\Shopfloor\collector-YYYYMMDD.log`.
Client-side log for the fleet reporter: `C:\Logs\Shopfloor\collector-YYYYMMDD.log`;
for the printer-queue reporter: `C:\Logs\Shopfloor\report-printers-YYYYMMDD.log`.
Server-side: the Flask app log (the collector logs upsert failures and per-plugin
Server-side: the Flask app log (the collector logs upsert failures and per-plugin
| `setup_complete` | `false` | Set true once the first-run wizard finishes; gates the `/setup` route. |
| `setup_complete` | `false` | Set true once the first-run wizard finishes; gates the `/setup` route. |
| `employee_directory_mode` | `selfhosted` | `selfhosted` (tables in this app) or `external` (a separate HR database, see `EMPLOYEE_DB_*`). |
| `employee_directory_mode` | `selfhosted` | `selfhosted` (tables in this app) or `external` (a separate HR database, see `EMPLOYEE_DB_*`). |
| `usb_directory_mode` | `selfhosted` | `selfhosted` or `external` (a separate `cmmc_usb` database, see `CMMC_USB_DB_*`). |
| `usb_directory_mode` | `selfhosted` | `selfhosted` or `external` (a separate `cmmc_usb` database, see `CMMC_USB_DB_*`). |
| `site_base_url` | (empty) | Public base URL (scheme + host) for QR codes and absolute links. Blank = use the browsing origin. |
| `audit_retentiondays` | `0` | Delete audit entries older than this many days. 0 keeps everything forever. Saving a period deletes nothing by itself: purge from this page or run `flask audit purge --apply`. |
| `site_base_url` | (empty) | Public base URL for QR codes and absolute links. Scheme + host, optionally including the subpath a site is served under (`https://host/shopdb`). Blank = use the browsing origin. |
| `facility_name` | (empty) | Facility name in the dashboard header. Blank = frontend falls back to `ShopDB`. |
| `facility_name` | (empty) | Facility name in the dashboard header. Blank = frontend falls back to `ShopDB`. |
| `pc_access_domain` | `device.geaerospace.net` | Domain appended to a PC hostname for remote-access links. Blank = hostname as-is. |
| `pc_access_domain` | `device.geaerospace.net` | Domain appended to a PC hostname for remote-access links. Blank = hostname as-is. |
| `employeeid_pattern` | `^\d{9}$` | Regex a search term must match to be treated as an employee id. Invalid regex falls back to the default and never 500s. |
| `employeeid_pattern` | `^\d{9}$` | Regex a search term must match to be treated as an employee id. Invalid regex falls back to the default and never 500s. |
@@ -196,12 +197,21 @@ The batch generator at `/print/asset-label-batch/<assettype>` (reached from the
### map
### map
**SUPERSEDED as of 0.11.0.** A blueprint and its pixel dimensions are properties
of a LEVEL now, not of the site (ADR-017), because a site can have more than one
building and a building more than one floor. Manage them in
**Settings > Buildings and levels**, or through `/api/maplevels`.
These four keys are still present so that downgrading finds the blueprint it had,
and the levels migration copied their values onto the first level. Editing them
changes nothing on the map.
| Key | Default | Notes |
| Key | Default | Notes |
|-----|---------|-------|
|-----|---------|-------|
| `map_blueprint_light` | `/static/images/floorplan-placeholder.svg` | Floor-map blueprint (light theme). Re-upload your own in Settings > Floor Map. |
| `map_blueprint_light` | `/static/images/floorplan-placeholder.svg` | Superseded by `maplevels.blueprintlight`. Retained for downgrade only. |
| `saml_allow_local_login` | `true` | Allow local username/password login when SAML is on. |
| `saml_allow_local_login` | `true` | Reserved, and locked on. Local login cannot be turned off while it is the only way in. |
| `saml_auto_create_users` | `true` | Auto-create users on first SAML login. |
| `saml_auto_create_users` | `true` | Reserved. Auto-create users on first SAML login. |
| `saml_admin_group` | (empty) | SAML group name that grants the admin role. |
| `saml_admin_group` | (empty) | Reserved. SAML group name that grants the admin role. |
**Personal API tokens.** Besides login JWTs and SAML, a user may create
**Personal API tokens.** Besides login JWTs, a user may create
personal API tokens (PATs) for scripts and integrations, from Settings > API
personal API tokens (PATs) for scripts and integrations, from Settings > API
Tokens (or `POST /api/apitokens`). A PAT is sent like a JWT
Tokens (or `POST /api/apitokens`). A PAT is sent like a JWT
(`Authorization: Bearer shopdb_pat_...`), authenticates as its owning user
(`Authorization: Bearer shopdb_pat_...`), authenticates as its owning user
@@ -313,6 +332,21 @@ intersected with the owner's live permissions at use time, and suspends the
admin bypass, so it is denied on role-gated (admin-only) endpoints and on import
admin bypass, so it is denied on role-gated (admin-only) endpoints and on import
mode. Use an unscoped token for admin-only work and imports.
mode. Use an unscoped token for admin-only work and imports.
### tools
| Key | Default | Notes |
|-----|---------|-------|
| `securesend_allow_guests` | `false` | Let people with no account create Secure Send links. Public (readable before login) so the create page can show a guest the form instead of a sign-in wall. Off by default: a guest secret has no creator, so the audit trail can say when and from where but never who. Receiving a secret never needs an account either way. |
Turning this on makes creation an unauthenticated write, bounded at 10 links per
address per 10 minutes. A signed-in user still needs the `tools.secrets`
permission, so a member without it cannot create one while a guest can - opening
the door to guests is a decision about the door, not about revoking what a role
holds. The setting fails closed if it cannot be read.
See [SECURE-SEND.md](SECURE-SEND.md) for the whole feature, including why the
server cannot recover a secret and what the size limit is doing.
### identifiers (dynamic)
### identifiers (dynamic)
One boolean key per asset identifier per asset type, keyed
One boolean key per asset identifier per asset type, keyed
@@ -8,12 +8,14 @@ the live code, not aspiration. The authoritative hook reference is
## Current version
## Current version
The plugin contract is at **0.13.0**, declared in `shopdb/__init__.py` as
The plugin contract version is declared in `shopdb/__init__.py` as
`__contract_version__`. It is pre-1.0, which under semver means any 0.x minor
`__contract_version__`, and the current value is in
bump is allowed to break the contract, and this project has used that latitude.
[PROJECT-MAP.md](PROJECT-MAP.md), which is generated. It is pre-1.0, which under
semver means any 0.x minor bump is allowed to break the contract, and this
project has used that latitude - see 0.19.0 below.
The product release version (`__version__`, currently 0.7.0) is a separate
The product release version (`__version__`) is a separate series with its own
series with its own bump rules; see [ADR-007](adr/ADR-007-product-versioning-and-releases.md).
bump rules; see [ADR-007](adr/ADR-007-product-versioning-and-releases.md).
Do not pin against it for compatibility - pin against `__contract_version__`.
Do not pin against it for compatibility - pin against `__contract_version__`.
### 0.x history
### 0.x history
@@ -31,10 +33,32 @@ Recorded in the comment block in `shopdb/__init__.py`:
| 0.11.0 | Added `service_token_authorized(scope)` to `shopdb.api` so a plugin's unattended endpoints (e.g. the GE-Enforce fetch API) can authorize a scoped managed service token without importing core token internals | additive surface (minor) |
| 0.11.0 | Added `service_token_authorized(scope)` to `shopdb.api` so a plugin's unattended endpoints (e.g. the GE-Enforce fetch API) can authorize a scoped managed service token without importing core token internals | additive surface (minor) |
| 0.12.0 | Added the mailer helpers (`send_email`, `send_alert`) to `shopdb.api` | additive surface (minor) |
| 0.12.0 | Added the mailer helpers (`send_email`, `send_alert`) to `shopdb.api` | additive surface (minor) |
| 0.13.0 | Added the `User` model to the `shopdb.api` surface | additive surface (minor) |
| 0.13.0 | Added the `User` model to the `shopdb.api` surface | additive surface (minor) |
| 0.14.0 | Added `send_webhook` to the `shopdb.api` surface | additive surface (minor) |
| 0.15.0 | Added `authorized_service_token` / the `SupportTeam` model to the `shopdb.api` surface | additive surface (minor) |
| 0.16.0 | Added the `get_settings_defaults` hook so a plugin declares the Setting rows it owns; the framework seeds them at install, at enable, and on `flask plugin upgrade-all`, files a first-time write under the declared category, and honours `public: True` for pages that render before login | additive optional hook (minor) |
| 0.17.0 | Added `DashboardDefault` to the `shopdb.api` surface, so a plugin can resolve a display without reaching into core | additive surface (minor) |
| 0.18.0 | Added `DISPLAY_ROLES`, `DISPLAY_ROLE_PATHS` and `normalize_display_role`, and finally exported `DashboardDefault`, which 0.17.0 imported but left out of `__all__`. The role vocabulary became the kiosk's own - `Dashboard`, `Lobby`, `3DPrintRoom` - so a plugin holding its own copy of that map reads core's instead of drifting from it | additive surface (minor) |
| 0.19.0 | **BREAKING.**`get_dashboard_widgets` returns DATA AND SHAPE, not a component name. The old shape (`name` + `component` + `size`) named a Vue component per widget, which cannot survive a lean build - a plugin's component may never be staged into the frontend bundle (ADR-013) - and five plugins were declaring widgets that pointed at components nobody had written. A card now declares `id` / `title` / `endpoint` / `render` / `severity` / `permission` / `empty` / `position`; see PLUGIN-HOOKS.md. Also added `send_upload` so a plugin serving user-supplied bytes gets the headers that keep an SVG from running as script | **contract change (minor, pre-1.0)** |
| 0.20.0 | The `Asset` model gained `levelid` (which floor plan its `mapx`/`mapy` are pixels of) and `mapverifiedat`; `Location` gained `levelid` too, and `resolve_asset_position` now returns the `levelid` belonging to whichever source supplied the coordinates. A plugin that writes a map position MUST write the level with it - a position without one renders as "level unknown" rather than being drawn on the default drawing, because a marker on the wrong floor plan looks correct (ADR-017) | additive surface (minor) |
| 0.21.0 | Added the `get_guest_grantable_permissions` hook: a plugin names the WRITE permissions a site may grant to the seeded `guest` role, which is who anyone without a token now holds. Reads (`.view`/`.read`) are grantable without a plugin saying anything; anything else is refused. Also exposed `require_any_permission` on `shopdb.api` for a read two different rights should reach | additive optional hook + surface (minor) |
| 0.22.0 | Added the `register_import_hook` surface: a plugin owns a VIRTUAL COLUMN on an importable CSV table - a header the bulk importer accepts but never writes - plus a post-write row hook that turns the value into a plugin concept. The computers plugin uses it for `machineassetnumber` on `computers.csv`, which builds the PC->machine controls link, so a site writes the machine number in the PC's own row instead of hand-building a separate relationships sheet. Core never learns what the column means. `RowWarning` is exported with it so a hook can report a soft failure without aborting the import | additive surface (minor) |
| 0.23.0 | Added `mint_service_token`, `list_service_tokens` and `revoke_service_tokens` so a plugin can provision, list and revoke a single-scope managed service token without importing core token internals. The endpoints plugin uses them for a UI-managed, regenerable agent key scoped to `collector.ingest.endpoints` alone - narrow on purpose, so rotating or revoking it cannot disturb the computers or GE-Enforce collectors, and revocation is an instant kill switch on agent ingest | additive surface (minor) |
| 0.24.0 | Added `collector_authorized(pluginname)` so a plugin can gate an agent-facing route that lives OUTSIDE the generic collector blueprint on the same per-plugin key and kill switch the ingest route uses. The endpoints command channel needs it: the agent long-polls `/api/endpoints/commands/next` and posts results back, and those are agent callbacks rather than JWT routes. Without it a plugin would reach into core collector internals to authorize them | additive surface (minor) |
| 0.25.0 | Added `caller_holds(permission)`, which answers whether the current caller holds a permission WITHOUT refusing the request - the complement to `require_permission` for a route that serves everyone but should only offer what the caller can act on. The endpoints services panel is the case: it emitted Start/Stop row actions to every reader, because the emission never consulted the caller. Clicking them was always refused server-side, so nothing was exploitable, but offering a read-only viewer controls over a shop-floor PC is the wrong signal, and it becomes a real problem once a site grants that panel to guests. Guests are included in the answer, since a signed-out visitor holds whatever the guest role was granted | additive surface (minor) |
| 0.26.0 | Dashboard cards may declare `shortname`. The dashboard merged its per-source cards into one worst-first queue, and a card's `title` is written as a heading ("Endpoints with crashes or shutdowns") which as a row badge ran wider than the hostname it qualified. Optional, defaults to the title, so a plugin that does not declare one is unaffected | additive field (minor) |
| 0.27.0 | Added `Script` so a plugin can read the ShopDB-hosted script library that backs endpoint `run_script` delivery without importing core model internals. The endpoints plugin resolves a delivered script by name and serves its body by digest to the agent | additive surface (minor) |
| 0.29.0 | Added an optional `group` to `get_asset_panels`, naming which detail-page tab a panel belongs to: `details` (the default) / `monitors` / `software` / `activity`. The detail pages are tabbed, and a page that placed panels by knowing their ids would defeat the hook. Additive: an absent or unrecognised group resolves to `details` server-side, so every existing plugin keeps working with no change | additive surface (minor) |
| 0.28.0 | Added `client_ip` and `strip_port` so a plugin does not hand-roll caller-IP extraction. Two plugins had, and both read the first `X-Forwarded-For` hop, which the caller supplies - so a rate limit keyed on it was defeated by rotating the header. The helper resolves from `remote_addr`, which ProxyFix has already filled from the header when `TRUSTED_PROXY_COUNT` says a proxy is really in front | additive surface (minor) |
| 0.31.0 | The asset-panel vocabulary a plugin may declare grew four optional things: two more tab groups, `serviceprocess` and `reports`, beside the 0.29.0 set; a `format: 'datetime'` on a column or field, so a panel stops shipping pre-formatted date strings the browser cannot re-localise; a `map.body` key, so a keyvalue field can name where its value lives in a nested payload; and a `tabs` section that carries `columns`/`rows`/`empty` and renders as a table, so one card can hold two tables that share no key (volume capacity and physical drive health are the case). All optional and all resolved server-side when absent, so a plugin declaring none of them is unaffected | additive surface (minor) |
The source comment block documents 0.3.0, 0.4.0, 0.6.0, 0.7.0, 0.9.0, 0.10.0, and 0.11.0 (its
3. `venv\Scripts\flask db upgrade` (if new migrations).
4. Recycle the app pool.
## Troubleshooting
| Symptom | Cause / fix |
|---|---|
| Site 502 / process won't start | Check `APP_ROOT\logs\httpplatform*`. Usually a bad `processPath`, missing waitress, or `wsgi:app` not importable (set `PYTHONPATH`). |
| Boots but SQL echoes / debug on | `FLASK_ENV` not `production` (web.config env var or `.env`). |
| `flask db upgrade` error 1071 | MySQL 5.6 `[mysqld]` flags in step 3 not applied / server not restarted. |
| ConfigError on boot | A required var (SECRET_KEY / JWT_SECRET_KEY / DATABASE_URL / CORS_ORIGINS) missing or left at a dev default in `.env`. |
| Login works, CORS errors in browser | `CORS_ORIGINS` does not match the exact origin (scheme + host + port) the browser used. |
| Audit logs show 127.0.0.1 | Expected without the URL Rewrite X-Forwarded-For rule (step 6). |
docker compose exec -u root api chown -R shopdb:shopdb /app/instance
docker compose restart api
```
The `chown` is not optional. `docker compose cp` writes the files with the
copying user's numeric uid, which is only `shopdb` by coincidence if your host
account happens to be uid 1000. Get it wrong and the site reads its restored
files fine and cannot write new ones.
Then run the migrations and confirm the plugins came back:
```bash
docker compose exec api flask db upgrade
docker compose exec api flask plugin upgrade-all
docker compose exec api flask db current # must match `flask db heads`
docker compose exec api flask plugin list # the site's plugins, enabled
```
Restore `instance/` BEFORE `plugin upgrade-all`: that command works from
`plugins.json`, so running it against an empty instance directory upgrades
nothing and reports success.
The symptom of having missed this is a site that comes back with its plugins
disabled and image URLs that 404: the MySQL rows survived, the files did not.
Re-enabling by hand works, but `flask plugin apply-profile <profile.json>` puts
the same list back in one command and is the thing to keep in version control.
The framework's `__contract_version__` may have moved. Check `docs/adr/` for any new ADRs since the last update. If an ADR introduces a breaking change, the upgrade may require coordinated work; the ADR's "Consequences" section documents it. See [docs/UPGRADE.md](UPGRADE.md) for the full upgrade procedure, including re-seeding and the v0.5+ floor-plan note.
The framework's `__contract_version__` may have moved. Check `docs/adr/` for any new ADRs since the last update. If an ADR introduces a breaking change, the upgrade may require coordinated work; the ADR's "Consequences" section documents it. See [docs/UPGRADE.md](UPGRADE.md) for the full upgrade procedure, including re-seeding and the v0.5+ floor-plan note.
On Windows, install all of them with winget (accept each license, then
On Windows, install all of them with winget (accept each license, then
@@ -293,7 +293,7 @@ section of the plugin lab for the full review checklist.
| --- | --- |
| --- | --- |
| Frontend loads but every API call fails / CORS error | backend not on 5001 (`flask run --port 5001`), or `CORS_ORIGINS` missing `http://localhost:5173`. |
| Frontend loads but every API call fails / CORS error | backend not on 5001 (`flask run --port 5001`), or `CORS_ORIGINS` missing `http://localhost:5173`. |
| App refuses to boot in production config | a required `.env` var (`SECRET_KEY`, `JWT_SECRET_KEY`, `DATABASE_URL`, `CORS_ORIGINS`) missing or a dev default. |
| App refuses to boot in production config | a required `.env` var (`SECRET_KEY`, `JWT_SECRET_KEY`, `DATABASE_URL`, `CORS_ORIGINS`) missing or a dev default. |
| `flask db upgrade` error 1071 (key too long) | MySQL 5.6 without the `innodb_large_prefix`/Barracuda flags; use MySQL 8 for dev. |
| `flask db upgrade` / `flask plugin upgrade-all` error 1071 (key too long) | MySQL 5.6 on the default Antelope file format, where an index prefix caps at 767 bytes - one utf8mb4 `VARCHAR(255)` is 1020. Start mysqld with `--innodb-file-format=Barracuda --innodb-large-prefix=ON` (both are also settable live with `SET GLOBAL`), keep `?charset=utf8mb4` on `DATABASE_URL`, and note that tables CREATED before those flags were on are silently `ROW_FORMAT=Compact` rather than `DYNAMIC` - 5.6 downgrades the row format with a warning instead of an error, so they keep the 767-byte cap until rebuilt with `ALTER TABLE <t> ROW_FORMAT=DYNAMIC`. MySQL 5.7+ and 8.0 need none of this. |
| Nav missing Machines/PCs/... | plugins not installed/enabled (step 2b), or the backend not restarted after enabling. |
| Nav missing Machines/PCs/... | plugins not installed/enabled (step 2b), or the backend not restarted after enabling. |
| "No time zone found with key America/New_York" | `tzdata` not installed - `pip install -r requirements.txt` includes it. |
| "No time zone found with key America/New_York" | `tzdata` not installed - `pip install -r requirements.txt` includes it. |
| npm/git/pip SSL error (`UNABLE_TO_GET_ISSUER_CERT_LOCALLY`, `unable to get local issuer certificate`) | corp proxy (Zscaler) intercepts HTTPS - point each tool at the corp root CA. See section 0b. |
| npm/git/pip SSL error (`UNABLE_TO_GET_ISSUER_CERT_LOCALLY`, `unable to get local issuer certificate`) | corp proxy (Zscaler) intercepts HTTPS - point each tool at the corp root CA. See section 0b. |
ShopDB is a server. Around it are four things that run on shop-floor PCs, each
documented in its own page, and each of those pages assumes you already know how
it relates to the others. This page is that relationship.
## The shape
```
+-------------------------------+
| ShopDB (one per site) |
| Flask API + Vue UI + MySQL |
+-------------------------------+
^ ^ ^ |
reports | | | | serves
(X-API-Key) | | | v
+----------------+ | | | +------------------+
| asset reporter |---+ | | | kiosks, displays |
| on every PC | | | | (browser, HTTPS) |
+----------------+ | | +------------------+
| |
+----------------+ | | +------------------+
| backup |----------+ +----| EventSaver |
| collectors | config revisions | screensaver |
+----------------+ +------------------+
pulls the slide feed
+---------------------------+
| GE-Enforce client | pulls its manifest, applies it,
| scheduled task, SYSTEM | reports the cycle back
+---------------------------+
^ |
| v
manifest the PC's
(share or actual state
HTTPS)
```
Every arrow is one direction on purpose. **Nothing on the server reaches out to
a PC.** The server holds state and answers questions; the fleet asks. That is
what lets a bay behind a firewall, asleep, or on a different network segment be
merely out of date rather than broken - and it is why every one of these tools
degrades to "stale" instead of "failed".
## What each piece is for
| Piece | Direction | What it does | Page |
|---|---|---|---|
| **Asset reporter** | PC to server | Says what this PC is: hostname, serial, type, user, IPs, machine number. Creates the PC record the rest of the system hangs off. | [ADOPTING-AT-ANOTHER-SITE](ADOPTING-AT-ANOTHER-SITE.md) |
| **GE-Enforce client** | both | Pulls the manifest for this PC's type, makes the PC match it, reports what it installed, skipped or failed. | [GE-ENFORCE](GE-ENFORCE.md) |
| **Backup collectors** | PC to server | Post machine configuration (NTLARS/DNC, CMM, part marker, UDC) as revisions, so a controller can be restored and a change can be seen. | [BACKUP-RESTORE](BACKUP-RESTORE.md) |
| **EventSaver** | PC to server | Pulls the slide playlist and shows it when a bay is idle. | [EVENTSAVER](EVENTSAVER.md) |
| **Kiosks and displays** | PC to server | A browser in kiosk mode on a ShopDB route. No client software beyond the browser. | [GE-ENFORCE-DISPLAY](GE-ENFORCE-DISPLAY.md) |
## The three ways a PC talks to the server
**The collector API**, `POST /api/collector/*`, with `X-API-Key`. Used by the
asset reporter and the backup collectors. The key is a managed token scoped to
`collector.ingest` and nothing else, so one recovered off a bay cannot read the
asset register. Idempotent by design: the same report twice changes nothing.
See [COLLECTOR-INTEGRATION](COLLECTOR-INTEGRATION.md) and ADR-006.
**The GE-Enforce API**, `/api/geenforce/*`, with a token scoped to the fetch
scope. Manifests out, cycle reports back. Most PC types still take their
manifest from the SMB share and use this only to report; the display cohort
takes everything over HTTPS, because those PCs have no share.
**Plain public reads.** The slide feed and the kiosk dashboard routes need no
credentials at all, because a screensaver and a wall display have nowhere to
keep one. They are read-only and carry nothing a floor visitor should not see.
## Where the credentials live
Three documents each named a different place, which is what happens when a
credential is described from whichever end the author was working on. Both ends
are here.
| Credential | On the server | On the PC | Rotate by |
|---|---|---|---|
| Collector key | A managed token scoped to `collector.ingest` only. `COLLECTOR_API_KEY` in `.env` also works and is simpler for a pilot, but cannot be revoked per fleet. | `HKLM:\SOFTWARE\GE\ShopDB` value `CollectorKey` | Replacing the token file the fleet reads from, so every PC picks it up on its next cycle. Never put it in the manifest - manifests sync broadly. |
| GE-Enforce fetch token | A managed service token scoped to the GE-Enforce fetch scope | `HKLM:\SOFTWARE\GE\ShopDB` value `ApiToken` | Same path as the collector key |
| Server URL | n/a | `HKLM:\SOFTWARE\GE\ShopDB` value `BaseUrl` | Setting it once at install; nothing should overwrite a bay's own value |
| Application secrets | `.env`: `SECRET_KEY`, `JWT_SECRET_KEY`, `DATABASE_URL`, integration tokens | n/a | See [BACKUP-RESTORE](BACKUP-RESTORE.md) - losing these invalidates every issued token |
Two rules behind that table. A shop-floor PC is not a trusted place, so what it
holds is scoped to exactly what it does: a collector key that leaks buys the
reader the ability to file a PC report, not to read the asset register. And a
credential is delivered, never typed - a value that has to be entered per machine
is a value that is wrong on some machine.
## What owns what
The question that decides most arguments: **who is allowed to change this?**
- **ShopDB owns the record.** What a PC is, what it drives, what it should have.
- **The manifest owns the PC's configuration.** If GE-Enforce enforces a value,
editing it on the PC is temporary - the next cycle puts it back. That is the
feature, and it is the single most common surprise.
- **The PC owns its own identity.** Serial, hostname, installed software: the
server records what it is told rather than deciding.
Where two systems both converge state - GE-Enforce and Intune DSC, say - decide
which owns which layer before deploying both, or they will fight in a way that
is very hard to see. [ADOPTING-AT-ANOTHER-SITE](ADOPTING-AT-ANOTHER-SITE.md) has
the recommended split.
## When something looks wrong
| Symptom | The piece to look at first |
|---|---|
| A PC is missing from ShopDB entirely | Asset reporter: never ran, or no collector key |
| A PC is there but its details are old | Asset reporter: stopped running. The dashboard's "PCs not reporting" card is this |
| A PC has the wrong software | GE-Enforce: check its last cycle report, then its manifest scope |
| A bay shows the wrong thing on screen | EventSaver config, or the kiosk dispatcher - both are per PC type |
| A machine has no recent config backup | Backup collector on the controlling PC. "Backups that have stopped" on the dashboard |
| Two PCs claim one machine number | Neither: it is a claim in progress, or a typo. The dashboard card names both |
Each row is a different program on a different schedule. Knowing which one to
Small, and the reasons are worth stating so a site does not over-provision.
ShopDB serves one plant's own staff, not the internet: a busy instance handles a
few dozen concurrent users and a fleet reporting a few hundred PCs a day, and
both are trivial loads. What actually consumes resources is MySQL and the
uploads.
| | Minimum | Comfortable | Why |
|---|---|---|---|
| Disk | 5 GB free (the installer refuses below this) | 40 GB | The application and Python are about 1 GB. The rest is MySQL, config-backup revisions, and uploads - branding, floor-plan blueprints, application installers and warranty proofs, which are the only ones that grow without bound. |
| RAM | 4 GB | 8 GB | MySQL wants most of it. The application itself is a Python worker per request. |
| CPU | 2 cores | 4 cores | Reports and the map are the heaviest reads and are cached. |
A VM is entirely normal. The one thing worth insisting on is that the database
directory and the backup directory are not on the same disk as anything you
would be sad to fill up.
### Getting the installer
The `.exe` is a release asset on the GitHub repository, not a file in it - it is
around 240 MB, which is well past what a repository accepts and well inside what
a release asset does. Take it from the release page for the version you want,
and keep it: you need the previous one to roll a release back, and you cannot
download it from your own server.
Every build is stamped with its version and ships a CycloneDX SBOM onto the
server, so `shopdb-admin.ps1 verify -Path <name>` answers "does this server
carry component X" without a network call - which is the question that arrives
on a Friday afternoon after a CVE announcement.
### If you get stuck
Raise an issue on the repository. Include the output of
`shopdb-admin.ps1 check -Json`, which reports version, IIS and pool state,
database reachability, Python version and installed plugins, and contains no
secrets - it is the single most useful thing you can attach, and it is designed
to be pasted.
---
The installer **checks all of this before it changes anything**, and it will not
The installer **checks all of this before it changes anything**, and it will not
let you continue until the check passes. You do not have to get it right first
let you continue until the check passes. You do not have to get it right first
time.
time.
@@ -90,7 +131,7 @@ latin1 database mangles any non-ASCII text you store.
is not already running a website. Simplest.
is not already running a website. Simplest.
- **Under this server's existing address** - `http://yourserver/shopdb/`. Choose
- **Under this server's existing address** - `http://yourserver/shopdb/`. Choose
this when the server already serves something else and you do not want a second
this when the server already serves something else and you do not want a second
port or a new DNS name. This is what West Jefferson uses.
port or a new DNS name. This is what the reference site uses.
You cannot change your mind later without re-running the installer, because the
You cannot change your mind later without re-running the installer, because the
-`MOUNT_PATH` comes out unless you are preserving a subpath (item 2 above).
Leave the Windows site installed but stopped until the Docker site is verified.
Rolling back is then a matter of starting the app pool again.
### What does not need migrating
`plugins.json` comes across in the instance directory, so the site's enabled
plugin set follows it. The image bakes the whole catalog, so whatever was
enabled on Windows is available in the container. The schema is identical, so
`db upgrade` and `plugin upgrade-all` are no-ops unless the target is also a
newer release.
---
## Docker to Windows/IIS
The reverse works the same way and is worth knowing for a rollback. Dump from
the container, restore into the Windows MySQL server, unpack the instance
directory into `APP_ROOT\instance`, and grant the app pool Modify on it (step
7.3 of [INSTALL-WINDOWS-IIS](INSTALL-WINDOWS-IIS.md)) - the container's uid
means nothing to Windows, and a directory the pool cannot write produces
"internal error" on any upload or plugin toggle.
The one constraint is MySQL version: do not restore an 8.x dump into a 5.6 or
5.7 server.
---
## Cutover checklist
1. Announce the outage. The database is stopped for the dump.
2. Take the dump and the instance copy from the SAME quiet moment.
3. Stand the new stack up and restore both.
4. Run `db upgrade` and `plugin upgrade-all`, then `db current` against
`db heads`.
5. Log in. Load a floor map. Load a page with images. Toggle nothing.
6. Repoint DNS, or update the fleet's server address if the hostname changed.
7. Confirm one bay checks in and one collector report arrives.
8. Leave the old stack in place, stopped, until step 7 has happened at least
once on a working day.
## See also
- [BACKUP-RESTORE](BACKUP-RESTORE.md) - what a complete backup contains, and the
restore procedure each of these steps is built on
- [DEPLOY](DEPLOY.md) - the Docker stack, and the update procedure
- [DEPLOY-AIRGAP](DEPLOY-AIRGAP.md) - the same stack where nothing can be pulled
- [INSTALL-WINDOWS-IIS](INSTALL-WINDOWS-IIS.md) - the Windows end in detail
- [UPGRADE](UPGRADE.md) - moving between product versions, which is a different
question from moving between topologies
- [ADR-004](adr/ADR-004-deployment-topology.md) - why each site runs its own
stack
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
